How Developers and Managers Define Productivity
What managers need to know about how developers define productivity. Plus, a shared definition for “quality” and how it can help conversations about making tradeoffs.
This week I read How Developers and Managers Define and Trade Productivity for Quality, a paper by Margaret-Anne Storey, Brian Houck, and Thomas Zimmerman. Reading this reminded me of an experience I had while working at GitHub. I asked engineering leaders across the company how they defined “productivity” and got wildly divergent answers that left me puzzled.
Dr. Margaret-Anne Storey and her colleagues have approached this question using rigorous research methods, helping us understand how developers and managers define productivity and quality, and how aligned they are in their views.
My summary of the paper
The terms “productivity” and “quality” are used liberally in industry settings, but there is a lack of consensus and awareness of what these terms actually mean. Alignment on how developers and managers define productivity and quality is essential if we are to meaningfully measure or improve these things.
Here’s what the research found:
1. Developers and managers define productivity differently.
The researchers bucketed definitions of productivity shared by developers and managers into five categories based on the SPACE Framework. SPACE includes five dimensions of productivity including Satisfaction (S), Performance (P), Activity (A), Collaboration (C), and Efficiency and Flow (E).
Most IC’s defined their own productivity in terms of activities and task completion (A), whereas most managers defined their team’s productivity in terms of the performance and quality of their team’s outcomes (P).
One takeaway here is that by enabling developers to complete more tasks, you will make them feel more productive, which in turns means they’ll be happier (more on this in an upcoming paper).
2. Developers often have incorrect beliefs about how managers define productivity.
Most IC’s believe that their managers define their team’s productivity in terms of activities and task completion (A), when in fact this is incorrect.
This misalignment has many potential implications. For example, developers may optimize for the wrong behaviors (eg. prioritizing completing their own tasks over the greater good of the team) due to misperceptions about what their managers value.
To address this, it’s important for engineering managers to clearly define the outcomes that matter, and constantly reinforce them to make sure people are aligned. Ideally, this should be further reinforced by conducting performance evaluations based around outcomes, as opposed to how many tasks or features were completed.
3. Developers and managers share a closer definition of “quality”.
Developers and managers are more aligned in their definitions of “quality” than they are for “productivity”. The researchers created a framework for software quality based on the definitions they collected, called TRUCE. (Timeliness of features delivered, Robustness of features, User delight, Collaboration needs met, and Evolution needs met.)
TRUCE provides a good basis for having conversations about software quality. For example, instead of broadly deciding to “trade quality” for productivity, teams can be specific about which aspects of quality they’re willing to trade to meet short-term needs.
With all the hubbub around measuring developer productivity, it’s sobering to realize that our industry isn’t even aligned on what productivity means. On that note, this paper is a great reminder to be careful about what you measure. For example, teams that measure the number of tickets or pull requests completed may be incentivizing developers to focus on task completion instead of driving toward outcomes.
If you’re a manager, this paper is a good reminder to be specific when discussing “productivity” with developers or other leaders. It’s also important to make sure that developers know exactly how you’ll assess their contributions, and by what indicators success will be measured by.
Other recommended reads
Recognize that attrition is an output
“You should focus on what you can control…. Besides cash and deferred compensation, look at how well your teams are executing. Are roles and responsibilities clear? Are decisions languishing? How quick is your and your team’s decision-making? How is execution?… I’m sure you will find plenty of reasons to improve. Those are what you can and must learn to control.”
Infra teams aren’t usually “chronically understaffed,” but it can feel that way during times of rapid company growth
Will Larson, CTO at Calm
“The challenge from the business perspective is that if you over-invest in the rapid growth era, you’re radically over-invested for the later eras… And so, it's about trying to figure out, what is it going to look like now, this painful growth state? What's it going to look like when the business stabilizes, and how far out are you from the actual stabilization?”
Resources for leading through crises
Lara Hogan, engineering leadership coach
A collection of resources for leaders to respond to crises and better support your teams. For quick reference, see Lara’s 1:1 questions.
That’s all for this week! Reach out with thoughts on Twitter at @abinoda, or reply to this email.