Improving Design Reviews at Google
Improving Design Reviews at Google
How Google meaningfully accelerated the design review process.
This is the latest issue of my newsletter. Each week I cover the latest research and perspectives on developer productivity.
Join us: Laura and I are hosting a new monthly discussion about the latest developments in developer productivity, and current strategies we’re seeing from the companies we work with. Register to join us here.
This week I read Improving Design Reviews at Google, a paper that describes how Google’s research team identified a problem impacting developer productivity, created a solution, and tested it. The approach outlined in this paper, as well as the specific actions they took to improve design reviews, should be useful for other leaders focused on developer productivity.
My summary of the paper
Google’s engineering satisfaction survey revealed that design reviews were a major pain point. Like many companies, Google’s process involved the document author tagging reviewers in a Google Doc through comments. Reviewers would then add their own comments, and after some back-and-forth discussions, they would resolve issues, marking the document as approved. While this approach got the job done, it wasn’t efficient, and developers saw it as a serious drag on their productivity.
The Design Approval Companion (DAC) tool
They identified a few issues with the design review process: approvers often struggled to find and track documents or changes that needed their attention, and reviewers weren’t always sure if a document was fully approved.
To fix this, researchers developed a tool called the Design Approval Companion (DAC). DAC is an internal Google Docs add-on that helps authors formalize the design review process with minimal extra work. Here’s how it works:
An author installs the DAC add-on once. After writing a design doc, they add approvers using Google Docs comments and use the word #approver anywhere in a comment. Approvers then approve by resolving the assigned action item as usual.
DAC scans all comments for the #approver tag, generates an approvers table at the top of the document, and keeps it updated automatically.
The researchers also integrated the DAC add-on with Event Notifier, an internal Chrome extension used by all developers, so the tool can alert users to events that need their attention.
Evaluating the tool
The researchers evaluated the tool by tracking its adoption (e.g., number of authors and approvers using it and the number of design documents it was used with), gathering developer feedback, and measuring time-to-approval.
Adoption and usability feedback: Over several years, the tool gained widespread adoption organically, mostly through word of mouth. Key usability features contributed to this: it integrated seamlessly with Google Docs and required minimal effort from both authors and approvers. Surveys also showed that engineers preferred a structured and automated design review process when available.
Adoption metrics also confirmed that the more approvers there are, the longer design reviews take. On average, there are 2.6 reviewers on each document.
Time-to-approval (TTA) measures the total duration between the time the document was created and the time all approvers approved the document. Researchers first established a baseline TTA before the DAC tool was introduced, which was done by asking developers to identify comments that constituted as approvals in a sample set of documents. After DAC was widely adopted, TTA had improved by 25%.
The researchers also hypothesized that TTA would improve over time as developers got used to the tool, and they were right. The average TTA continued to get better with each use.
These findings confirmed that the DAC tool created for internal use at Google meaningfully improved the design review process.
Final thoughts
I enjoy sharing case studies in this newsletter when they show how developer productivity can be improved with a focused, structured approach. Google is especially good at this. They start by identifying issues through surveys, then carefully track the impact of their changes using system-based metrics. They also appear to make sure to track metrics that matter to the business, like velocity (measured with time-to-approval in this case).
Who’s hiring right now
Here is a roundup of Developer Productivity job openings. Find more open roles here.
SiriusXM is hiring a Staff Software Engineer - Platform Observability | US
Roku is hiring a Senior Manager - Developer Tools | Cambridge
Citi is hiring a Director - Engineering Excellence | Texas
KnowBe4 is hiring a Software Engineer - Developer Experience | Florida
Expedia is hiring a Senior Manager - Delivery Platform | Seattle
That’s it for this week. If you know someone who might like this issue, consider sharing it with them:
-Abi