This is the latest issue of my newsletter. Each week I cover the latest research and perspectives on developer productivity.
This week I’m summarizing What Predicts Software Developers’ Productivity? by Google developer productivity researchers Emerson Murphy-Hill, Ciera Jaspan, Caitlin Sadowski, and their colleagues. This study aimed to identify the factors that most strongly correlate with productivity, and learn whether these factors differ across companies.
While other similar papers have provided insight into the factors that affect productivity, this paper gives some insight into which factors may be more important to prioritize.
My summary of the paper
There are many paths an organization can take to improve developer productivity, for example investing in better tools, focusing on technical debt, or cutting meetings. While prior research has explored many of these strategies, the goal of this study was to help leaders prioritize their options by providing them with a better idea of what factors strongly correlate with productivity.
For this study, the researchers first created a survey that asked developers about their productivity, as well as factors that prior literature suggests correlate with productivity. They surveyed 622 developers across Google and two other companies in different industries. Then, to analyze the relationships between variables, the researchers ran multiple linear regression analyses for each factor within each company and in relation to the developers’ self-rated productivity.
Defining productivity: The researchers needed to first establish a definition for productivity. They chose to use survey questions as a subjective measurement of productivity for two reasons: “surveys are a straightforward and commonly used method to measure knowledge worker productivity,” and "surveys are flexible enough to allow responses from developers in a variety of roles and to allow the respondent to incorporate a variety of information into a self-rating of productivity.” The researchers asked developers about their productivity by rating their agreement with the statement: “I regularly reach a high level of productivity.”
Regarding causality: This study measured correlations between productivity factors and self-rated productivity but did not investigate whether each factor directly causes productivity changes. However, the researchers argue that “we can have some confidence that these factors cause increases or decreases in productivity” by relying on prior work.
Here are the key findings from the study:
The factors that correlate with developer productivity
The table below presents the results of the analysis:
How to read the table above:
The first column indicates the factor, in the form of the statement shown to software developers on the survey.
The minor column labeled estimate indicates regression coefficient, which quantifies the strength of the association between that factor and self-rated productivity, where the larger the number, the stronger the association. Note that estimate does not mean higher factor ratings, but instead means a higher correlation between a factor and self-rated productivity.
The final major column indicates the mean estimate across all three companies. The table is sorted by mean estimate. The first bar visualizes the magnitude of the mean estimate, and the second bar visualizes the magnitude of the standard deviation.
Estimate and standard error values with an asterisk (*) indicate that the factor was statistically significant in its model. For instance, job enthusiasm (F1) was statistically significant across all three companies, but meeting preparation (F17) was only significant in Google.
The most strongly correlated factors with self-rated productivity are those with the highest absolute mean estimate. However, these are not necessarily generalizable for all companies. To determine which of the factors leaders can have the most confidence in applying within their own organizations, the researchers identified the factors that are statistically significant across all three of the companies studied:
Job enthusiasm (F1)
Peer support for new ideas (F2)
Useful feedback about job performance (F11)
Apart from identifying these factors, one intriguing observation is that the top 10 productivity factors are non-technical. Consequently, the authors encourage further research into human factors of productivity, including what makes developers enthusiastic about their job and what kinds of job feedback are useful to developers.
How the factors differ across companies
By examining the second bar in the far-right column (lighter gray), we can see the factors with the lowest variance—that is, the factors that were most consistent across companies:
Use of remote work to concentrate (F40)
Useful feedback about job performance (F4)
Peer support for new ideas (F2)
The researchers posit that the stability of these factors makes them suitable for generalization, implying that companies beyond those studied are likely to get similar results as to theirs. The researchers hypothesize that “developers, regardless of where they work, are equally affected by remote working, job feedback, and peer support for new ideas.”
The factors that varied the most between companies were:
Use of best tools and practices (F15)
Code reuse (F25)
Accuracy of incoming information (F6)
Other companies are less likely to see similar results for these factors. The researchers consider several potential reasons for the high variance, all tied to companies having different codebases and processes. For example, when discussing why reusing existing code (F25) was strongly related to self-rated productivity at Google but less so at another company, the researchers suggest that this may be because code reuse is easier to achieve in Google due to its monolithic codebase.
Improving the factors correlated with productivity
The researchers conclude by suggesting strategies for improving the factors that are most strongly correlated with developer productivity. These strategies draw on prior research as well as their experience at Google:
Improving job enthusiasm: Leaders can promote job enthusiasm by facilitating developers' growth through learning opportunities and recognition of exceptional work. For instance, one company in this study experimented with publicly recognizing developers who adopted tools and practices for structured code navigation.
Increasing support for new ideas: One proven strategy for fostering support of new ideas is to share best practices. “At Google, one way knowledge is shared is through an initiative called Testing on the Toilet, where engineers write short newsletters about testing or other technical knowledge.”
Increasing feedback about job performance: Research encourages leaders to provide feedback that is non-threatening, behaviorally-focused, and outcome-oriented. “At Google, one way that feedback is provided in this way is through blameless postmortems, where after a significant negative outcome occurs like a service outage, engineers collaboratively write a report that focuses on the behaviors that contributed to a root-cause, without blaming individual engineers.”
Final thoughts
This paper adds to the growing body of evidence supporting the importance of developer satisfaction. Specifically, it underscores that factors such job enthusiasm and support for new ideas strongly correlate with developer productivity across companies. This reinforces findings from other studies linking developer productivity and satisfaction (Graziotin, Storey).
What I also think is insightful from this paper is the analysis on how the correlations differ among companies. Non-technical factors (e.g., getting useful feedback, job enthusiasm) are important factors for productivity across companies. However, when it comes to technical factors such as the use of tools, codebase experience, and information quality, their impact on productivity varies based on the unique circumstances of each company.
That’s it for this week! If you know someone who might like this issue, consider sharing it with them:
P.S. If you’re interested in more research, send me a connection request with the note “research” and I’ll share a downloadable bundle of my top 10 favorite papers of this year.
This strongly aligns to Dan Pink’s book Drive, which is handy because Drive is easy to use framework.