Expectations, outcomes and challenges of modern code review
looks at motivations, challenges and outcomes by observing, surveying and interviewing Microsoft teams with diverse code review practices/culture
main motivation = finding defects
BUT less about defects than expected cos also about knowledge transfer, increased team awareness and alternative solutions
modern code review = informal, tool-based and regular
What are expectations for code review? (Do they change for managers vs developers vs testers?) What are its outcomes and do they match expectations? What are the main challenges?
helps practitioners figure out how code review might fit into their development process + when
helps researchers focus on the challenges to make them less challengey
not many actual comments in code reviews were about defects
understanding needs not met by current tool
compares expectations of managers and developers to code review outcomes; relates outcomes to understanding needs
Related work
Porter et al (1995) looked at effect of team size, type of review and number of sessions on code review
Votta found that 20% of time was wasted on scheduling with traditional code review cos involves looking at code line by line in a meeting
Work on developing a tool that allows asynchronous code reviews
Sutherland and Venolia found meat of code review is about articulating design rationale
Latoza et al found developers had difficulty finding rationale behind code changes and had to get knowledge from other developers
Findings
Why do programmers do code review?
What are the challenges of code review?
influence on defects but also wider team and development process eg. makes people less protective of their code; supports code conventions of the team
To find defects - find bugs, errors in design, flaws in the logic
To improve code - readability, consistency (with conventions), dead code removal, commenting etc are not about accuracy/defects
To create alternative solutions - ideas for better implementation in comments (mostly developers think this and only 2% of managers mentioned this motivation)
To transfer knowledge - learning about other parts of code base, best practices; developer education
To increase team awareness and transparency - everyone knows about changes
For shared code ownership - more open to critiques
The outcomes of code reviews
comments mostly on code improvement. defect finding was 4th
defect finding comments not being numerous suggest that expectations for code review don't match reality
understanding reason for code change is most difficult - descriptions of code is not enough (needs what was changed and why)
a lot of comments related to trying to understand (2nd most frequent)
complete understanding required for finding defects and alternative solutions though + better code reviews in general that are deeper, more directed/actionable/pertinent and touch on more subtle issues
tools don't provide for understanding further so developers email or go to talk to author