How top tech companies measure developer productivity

The top-level engineering productivity metrics used by 17 companies, including Spotify, Stripe, and Uber.

This is the latest issue of my newsletter. Each week I cover the latest research and perspectives on developer productivity.

---

This week I contributed a guest post to The Pragmatic Engineer, which shared the metrics used by Developer Productivity teams at 17 well-known companies. Today’s newsletter includes an excerpt from that piece.

Note: You can read the entire article on The Pragmatic Engineer. You can also download an extended version of the report, which defines the metrics listed in the report. 

Discussions around developer productivity metrics are often too high-level to be helpful. I wanted to provide something more concrete, so I interviewed several Developer Productivity (DevProd) and Developer Experience (DevEx) teams to learn what metrics they’re using. 

These teams, which are responsible for improving developer productivity at their companies, rely on developer productivity metrics. They need data to understand the problems developers are experiencing, prioritize projects appropriately, and track the impact of their work. This is why I think we can learn from a lot these teams about which metrics are useful.

To find out what these teams are measuring, I reached out to 17 leading tech companies, asking about the top-level metrics their Developer Productivity functions or teams use. Below is an overview of developer productivity metrics used across different companies, at time of publication. Here’s what they shared: 

Several interesting findings stand out to me after reviewing this list:

DORA and SPACE metrics are used selectively  

I expected the metrics from DORA and SPACE to appear more frequently, since they’re often thought of as “industry standard.” But only one company, Microsoft, mentioned embracing one of these frameworks wholesale, which makes sense since they authored the SPACE framework.

For other companies, some individual metrics from these frameworks were mentioned only as components in a broader, more holistic measurement strategy.

Broad adoption of qualitative metrics 

Across the board, all companies shared that they use both qualitative and quantitative measures. For example, Developer Engagement was shared as a metric by Intercom, Postman, and Peloton. Companies including Atlassian, Lattice, and Spotify similarly rely on self-reported measures.

This highlights a marked shift in how top companies are approaching developer productivity measurement. Five years ago, most of these companies were likely focused exclusively on quantitative metrics. Our recent DevEx paper provides a framework for leveraging qualitative metrics to measure developer experience. 

A big emphasis on “focus time”

I was surprised by how many companies track “focus time” as a top-level metric. Although research has shown “deep work” to be an important factor in developer productivity, I didn’t expect as much attention on it as I found. Stripe and Uber shared specific metrics, such as “Number of Days with Sufficient Focus Time,” and “Weekly Focus Time Per Engineer, while other companies mentioned deep work as a topic they measure in their developer survey programs. 

Unique metrics 

There’s quite a bit of overlap in what different developer productivity teams are measuring. But there are also a few unique metrics worth highlighting: 

• Adoption Rate (DoorDash, GoodRx, and Spotify.) How many developers actively use a product or service. Spotify’s version is a measure of how many developers have adopted its Golden Standards. 

• Design Docs Generated per Engineer (Uber.) Design docs are written by engineers for non-trivial projects before they start meaningful work—the idea is to get feedback early and ultimately decrease the time taken to complete a project. Their metric tracks how frequently developers are following this practice. 

• Experiment Velocity (Etsy.) Former CTO, Mike Fisher, says that at Etsy, each team designs and runs its own experiments to assess how users will respond to new features. This practice is a core aspect of their engineering culture, facilitating a culture of learning and helping teams stay focused on the customer. Etsy has developed an in-house experimentation platform to track the progress of these experiments. Metrics include how many experiments start each week, how many have been stopped, and how many have a positive hit rate. For context, the ultimate goal is to measure learning velocity. 

• Developer CSAT/NSAT (Chime and LinkedIn.) LinkedIn measures Developer NSAT (Net User Satisfaction,) which tracks how satisfied developers are overall with their development systems. Another metric listed on the report is a Developer Customer Satisfaction (CSAT) score for the tools and services that developers use. CSAT is captured via quarterly developer surveys. These metrics are different: the CSAT metric focuses on specific tools, whereas LinkedIn’s NSAT measures developers’ satisfaction with all tools, overall. Also, the CSAT metric is calculated as a percentage of positive responses, whereas LinkedIn’s is the percentage of satisfied responses subtracted by the percentage of dissatisfied responses.  

Final thoughts

This report sheds light on the metrics that DevProd teams are finding useful for understanding and improving developer productivity. If you’re just getting starting with metrics, I always recommend using Google’s Goals, Signals, Metrics (GSM) framework to guide which metrics you select. Start by defining your goals for measuring, then work backwards to choose metrics that support your goals. 

If you already have metrics in place, you might compare the ones you’re using to those listed here. Are there metrics you hadn’t considered that might useful for you? Are you leveraging both qualitative and quantitative measures to understand developer productivity? These may be useful questions to consider as you evaluate your existing approach.

Download the full report here or read the original article on The Pragmatic Engineer. 

---

That’s it for this week! If you have any feedback or questions, I’d love to hear from you on LinkedIn.

-Abi