Interactive code review for systematic changes

Developers often inspect a diff patch during peer code reviews. Diff patches show low-level program differences per file without summarizing systematic changes---similar, related changes to multiple contexts. We present Critics, an interactive approach for inspecting systematic changes. When a developer specifies code change within a diff patch, Critics allows developers to customize the change template by iteratively generalizing change content and context. By matching a generalized template against the codebase, it summarizes similar changes and detects potential mistakes. We evaluated Critics using two methods. First, we conducted a user study at Salesforce.com, where professional engineers used Critics to investigate diff patches authored by their own team. After using Critics, all six participants indicated that they would like Critics to be integrated into their current code review environment. This also attests to the fact that Critics scales to an industry-scale project and can be easily adopted by professional engineers. Second, we conducted a user study where twelve participants reviewed diff patches using Critics and Eclipse diff. The results show that human subjects using Critics answer questions about systematic changes 47.3% more correctly with 31.9% saving in time during code review tasks, in comparison to the baseline use of Eclipse diff. These results show that Critics should improve developer productivity in inspecting systematic changes during peer code reviews.