Expectations, outcomes and challenges of modern code review

looks at motivations, challenges and outcomes by observing, surveying and interviewing Microsoft teams with diverse code review practices/culture

main motivation = finding defects

BUT less about defects than expected cos also about knowledge transfer, increased team awareness and alternative solutions

modern code review = informal, tool-based and regular

What are expectations for code review? (Do they change for managers vs developers vs testers?) What are its outcomes and do they match expectations? What are the main challenges?

helps practitioners figure out how code review might fit into their development process + when

helps researchers focus on the challenges to make them less challengey

not many actual comments in code reviews were about defects

understanding needs not met by current tool

compares expectations of managers and developers to code review outcomes; relates outcomes to understanding needs

Related work

Porter et al (1995) looked at effect of team size, type of review and number of sessions on code review

Votta found that 20% of time was wasted on scheduling with traditional code review cos involves looking at code line by line in a meeting

Work on developing a tool that allows asynchronous code reviews

Sutherland and Venolia found meat of code review is about articulating design rationale

Latoza et al found developers had difficulty finding rationale behind code changes and had to get knowledge from other developers

Findings

Why do programmers do code review?

What are the challenges of code review?

influence on defects but also wider team and development process eg. makes people less protective of their code; supports code conventions of the team

To find defects - find bugs, errors in design, flaws in the logic

To improve code - readability, consistency (with conventions), dead code removal, commenting etc are not about accuracy/defects

To create alternative solutions - ideas for better implementation in comments (mostly developers think this and only 2% of managers mentioned this motivation)

To transfer knowledge - learning about other parts of code base, best practices; developer education

To increase team awareness and transparency - everyone knows about changes

For shared code ownership - more open to critiques

The outcomes of code reviews

comments mostly on code improvement. defect finding was 4th

defect finding comments not being numerous suggest that expectations for code review don't match reality

understanding reason for code change is most difficult - descriptions of code is not enough (needs what was changed and why)

a lot of comments related to trying to understand (2nd most frequent)

complete understanding required for finding defects and alternative solutions though + better code reviews in general that are deeper, more directed/actionable/pertinent and touch on more subtle issues

tools don't provide for understanding further so developers email or go to talk to author