Many teams struggle to use developer productivity data effectively because they don’t know how to use it to decide what to do next. They know that data is there to help them improve, but they may not know where to look. And even then, what do you actually do to put the wheels of change in motion? In this conversation, Laura and I talk about data-driven management and how to take a structured, analytical approach to using data for improvement.
Listen now on Apple, Spotify, and YouTube.
Some takeaways:
Common mistakes organizations make with developer productivity metrics:
Reverting to old habits: Simply adding the metrics to leadership reports without driving real change.
Overwhelming teams with data: Expecting teams to derive meaning from hundreds of measurements without providing adequate support or clear expectations.
Failing to connect metrics with decision-making: Collecting data that sits unused in dashboards rather than influencing team behavior and strategy.
Questions high-performing engineering teams will ask about productivity metrics:
Who is this data for? Are we diagnosing or improving? How will this data be used in decision-making?
Two primary use cases for metrics:
Engineering organizations use metrics to assess efficiency and drive improvement. Leadership uses this data to guide transformation efforts, ensure alignment with business goals, increase quality, and improve engineering velocity. Teams use metrics to make daily and weekly decisions to improve their performance and velocity.
Systems teams (Platform Eng, DevEx, DevProd): These teams use metrics to understand how engineering teams interact with internal systems and to assess the ROI of DevEx investments. These metrics are existential for these teams, who need to show the impact of their work in order to measure the success of their investment. These measurements are also crucial for setting future priorities.
Categories of metrics:
Diagnostic metrics: These are high-level, summary metrics that provide insights into trends over time.
Collected with lower frequency
Benefit from industry benchmarks to contextualize performance
Best used for directional or strategic decision-making
Examples: DX Core 4 primary metrics, DORA metrics
Improvement metrics: These metrics drive behavior change.
Collected with higher frequency
Focused on smaller variables
Are often in the teams’ locus of control
Other best practices for ensuring metrics lead to action:
Tell a story with data: Rather than presenting raw numbers, frame metrics in the context of progress toward key business goals.
Use industry benchmarks for context: Comparing your organization’s metrics to industry benchmarks can help make data actionable. You can download the full set of DX Core 4 benchmarks here.
Mix qualitative and quantitative data: Looking at quantitative data from systems can tell you what is happening, but only self-reported data from developers can tell you why. For improvement, the “why” is critical.
Timestamps:
(0:00) Intro
(2:07) The challenge we’re seeing
(6:53) Overview on using data
(8:58) Use cases for data: Engineering organizations
(15:57) Use cases for data: Engineering systems teams
(21:38) Two types of metrics - Diagnostics and Improvement
(38:09) Summary
Read the full transcript here.
Referenced:
Share this post