The Best Of Engineering Enablement 2023

The best research papers and interviews published so far.

This is the latest issue of my newsletter. Each week I cover the latest research and perspectives on developer productivity.

---

This newsletter was launched in the summer of 2022, meaning that 2023 has been the first, full calendar year of the newsletter. Thank you for being a reader! This has been an eventful year for developer productivity: the DevEx framework was introduced, McKinsey published a report that sparked debate, and meanwhile the impact of AI on productivity has increasingly become a topic of interest. 

In this end-of-year issue, I’ve put together a list of the best research papers and interviews published so far, organized by topic. This should serve as a curated guide to finding the most relevant resources on each topic.

---

What is developer productivity? 

1. Three Dimensions of Developer Productivity. This paper provides a set of outcomes that leaders can measure to capture productivity: originally Velocity, Quality, and Satisfaction, later revised to Speed, Ease, and Quality. 

2. The SPACE of Developer Productivity. This paper encourages leaders to think beyond just “activity” when attempting to measure and understand productivity. Productivity is more than just activity, it also encompasses satisfaction, performance, collaboration, and efficiency. 

3. Four Lenses of Productivity. The different lenses for viewing developer productivity: individual, team, organizational, and market. 

4. Why it’s Difficult to Measure Developer Productivity. This paper explains why measuring productivity is complex: because software engineering is creative, and because there are outside forces that influence how productive developers will be at a task.  

What impacts developer productivity? 

1. Maximizing Developer Effectiveness. This article describes the day in the life of a developer to illustrate what impacts productivity. Instead of tracking outputs to measure productivity, organizations would find more success if they focused on providing an effective working environment. 

2. DevEx: What Actually Drives Productivity. An approach to measuring and improving developer productivity which centers on the developer experience. 

3. Software Development Waste. The types of waste in software development and their causes. 

4. What Improves Developer Productivity at Google? Code Quality. Code quality is the strongest driver of developer productivity at Google, followed by the innovativeness of tooling and infrastructure, and then whether developers experience shifting project priorities. 

5. What Predicts Software Developers’ Productivity?. This study identifies correlations between various factors and developer productivity. The findings underscore the importance of job enthusiasm, support for new ideas, and feedback about job performance. 

6. The Bi-Directional Relationship Between Developer Satisfaction and Productivity. Higher levels of productivity drive job satisfaction, and vice versa. The paper also identifies the top impediments experienced by developers: poor software architecture, legacy code, finding relevant information, too many dependencies, and poor engineering tools. 

7. How Emotions Affect Perceived Productivity. Emotions are correlated with productivity. Positive emotions are triggered when developers feel in flow.  

Deeper dives into specific factors

1. Flow

   1. Barriers to Flow. Barriers to entering flow states include working on tasks that are boring, easy, or mundane, interruptions, time-related challenges, non user-friendly tools, and poor code quality.

   2. What Makes Developers Unhappy?. Developer unhappiness is caused by being stuck in problem solving, time pressure, and bad code quality and coding practice. 

   3. The Emotional Roller Coaster of Changing Requirements. Last-minute requirement changes (without extending the timeline) trigger negative emotions. 

   4. What Makes Some Interruptions More Disruptive Than Others?. The time of day when an interruption happens, whether the interruption was caused by the developer themselves or something external, and whether the interruption included context switching, are all predictors of how disruptive an interruption will be. 

2. Feedback loops

   1. How Flaky Tests Impact Developers. When there are too many flaky tests, developers are more likely to ignore potentially genuine test failures. 

   2. Understanding the Experience of Code Review. The code review process is made less efficient when reviews take longer than expected, when it’s not clear who should follow up on comments within a review, or when the reviewer lacks necessary information. 

   3. Cross-Team Collaboration at Scale. Communication between teams is affected by how well-aware teams are of each others’ work, how frequently they interact, and their attitudes towards the work other teams do. 

3. Cognitive Load 

   1. The Effects of Technical Debt on Morale. Tech debt reduces morale: it negatively impacts developers’ confidence and their sense of progress. Managing tech debt increases morale. 

   2. Technical Debt Cripples Productivity. Developers waste 23% of their working time due to technical debt. 

   3. The Cost of Architectural Complexity. Architectural complexity is linked to a much higher defect rate, a significant decline in productivity, and a much greater probability of turnover. 

   4. The Causes of Unreliable Software. Organizational complexity is the strongest predictor of bugs in a software module. 

   5. Characteristics of Code Quality. Readability, Structure, and Documentation are defining properties for code quality.

How to measure developer productivity 

1. Measurement approaches 

   1. How Google Measures Developer Productivity. An interview with two members of the Engineering Productivity Research team describing their approach to measurement. 

   2. How LinkedIn Measures Developer Productivity. LinkedIn takes a three-pronged approach to measuring productivity, which includes a quarterly, survey, real-time feedback tooling, and data collected from developer tools. 

   3. Shopify’s Developer Experience Survey Works. Shopify’s Developer Acceleration organization uses the survey to inform their roadmaps and measure the impact of their work. 

   4. How Peloton Measures Developer Experience With Surveys. What went into designing the survey, how it’s administered, and how insights are used. 

2. Defining, selecting, and implementing metrics 

   1. Three-Bucket Framework for Engineering Metrics. How to select which metrics to report on. 

   2. The Goals, Signals, Metrics Framework for Selecting Metrics. An exercise to guide the metrics selection process. 

   3. Defining and Tracking Productivity Metrics at LinkedIn. An interview with Grant Jenks from LinkedIn’s Developer Insights team that dives into the nuances of different metrics, and how they’re defined and tracked at LinkedIn. 

   4. Measuring Flow and Friction. The process Google uses for creating and validating metrics, and how they measure flow and friction. 

   5. How Google Measures and Manages Tech Debt. The 10 categories of tech debt, how Google measures tech debt, and strategies for managing it. 

   6. Using OKRs in Software Engineering. The common challenges teams face when adopting OKRs. 

   7. An Inside Look at DORA. What DORA is, how the metrics are intended to be used, and how not to use metrics. 

Strategies for improving productivity

1. Team-level improvements

   1. 2023 DORA Report. This year’s report found that teams that focus on the user have higher organizational performance. 

   2. Reducing Onboarding Time. Mentorship, documentation, team interaction, and well-chosen starting tasks are all factors that influence how quickly developers ramp-up. 

   3. Friction Between Product and Engineering. How to know whether this is becoming a bottleneck for a team, and strategies for addressing the problem. 

   4. Code Ownership and Software Quality. A case for either assigning ownership where there lacks a clear owner, or at least using weak ownership as an indicator of risk.

2. Larger coordinated improvements 

   1. Build Times and Developer Productivity. Even modest improvements to build times are beneficial. If teams are unable to make builds faster, they can make build times more predictable to improve productivity. 

   2. Etsy’s Multi-Year DevEx Initiative. An interview with former Etsy CTO Mike Fisher on why they decided to focus on developer experience and how their initiative worked. 

   3. How Google Measures and Manages Tech Debt. A list of strategies for managing tech debt. 

   4. Improving the On-Call Experience at Intercom. Key changes Intercom has made to their on-call experience over the years. 

   5. Moving to Remote Development Environments at Slack. Why and how Slack moved their entire development experience to remote environments. 

   6. Enabling Teams to Drive Their Own Productivity Improvements. The tactics used by Toast’s R&D Team Enablement function that enable engineering teams across the company to drive their own improvements. 

---

Thanks for reading. If you found this issue useful, you might be interested in joining a conversation I’m hosting with Laura Tacho on DORA, SPACE, and DevEx, and how to choose which framework to use. 

This is the final issue of the Engineering Enablement newsletter in 2023. Enjoy the holidays and I’ll see you on Friday, January 5th! 

-Abi