• Code reviewing @ Microsoft — Lessons Learned

    by  • September 27, 2016 • Code review, Research, Software • 0 Comments

    Recently, I gave a keynote at QUATIC – the 10th international conference on quality of information and communication technologies. The topic was code reviewing. In particular,  lessons learned at Microsoft. In our team, we have a close eye on the code reviewing practices of the engineering teams across the company and regularly perform studies to evaluate how well this practice works for the different teams. Below a few of the key findings from our studies that generalize across teams and organizations:

    Most teams’ policy requires an approved code review before the checkin. Microsoft is a large organization, and even though most of the engineers (>90%) who perform code reviewing use one internal tool called CodeFlow, the practices and policies for code reviewing differ. But, for almost all teams an approved code review — next to green passes of test and analysis runs — is a prerequisite for a code change to be permitted to be checked into the shared code base.

    Feedback usefulness of code reviewers depends on the size of the change. The larger the change size (counted by number of files changed), the less useful the feedback of the code reviewers. During interviews, engineers explained that large reviews are just hard to understand, thus limiting their ability to contribute valuable feedback.

    Feedback usefulness depends on the code reviewer’s experience (with the code). A code reviewer that has either reviewed or changed the code before gives more valuable feedback. The main difference comes from a single review, or a single change, i.e., a reviewer who has reviewed the code (only) once before gives already 30% more useful comments, than a reviewer who sees the code for the first time. Also, the experience a reviewer has in the organisation influences how useful the code review comments will be. We could see that especially during the first year, the feedback usefulness increases with time.

    Receiving feedback in a timely manner is one of the top challenges. As one engineer puts it: “Usually you write up some code and then you send it out for review, and then about a day later you ping them to remind them… and then about half a day later you go to their office and knock on their door.”  To conquer this challenge, code change authors should aim for small, incremental changes. Also, many engineers give their reviewers a heads-up that a review is coming in order for them to schedule some time for the code review. Also, giving reviewers a quick walk through so that they are able to quickly give meaningful feedback can be helpful.

    Checkout the slides for more challenges and best practices. Do you recognize some of the best practices and challenges? What’s your secret to make it work, and/or what’s the main pain point during code reviewing?

    Tweet This


    Leave a Reply

    Your email address will not be published. Required fields are marked *