How Developers and Managers Define Productivity

What managers need to know about how developers define productivity. Plus, a shared definition for “quality” and how it can help conversations about making tradeoffs.

This week I read How Developers and Managers Define and Trade Productivity for Quality, a paper by Margaret-Anne Storey, Brian Houck, and Thomas Zimmerman. Reading this reminded me of an experience I had while working at GitHub. I asked engineering leaders across the company how they defined “productivity” and got wildly divergent answers that left me puzzled.

Dr. Margaret-Anne Storey and her colleagues have approached this question using rigorous research methods, helping us understand how developers and managers define productivity and quality, and how aligned they are in their views.

My summary of the paper

The terms “productivity” and “quality” are used liberally in industry settings, but there is a lack of consensus and awareness of what these terms actually mean. Alignment on how developers and managers define productivity and quality is essential if we are to meaningfully measure or improve these things.

Here’s what the research found:

1. Developers and managers define productivity differently.

The researchers bucketed definitions of productivity shared by developers and managers into five categories based on the SPACE Framework. SPACE includes five dimensions of productivity including Satisfaction (S), Performance (P), Activity (A), Collaboration (C), and Efficiency and Flow (E).

Most IC’s defined their own productivity in terms of activities and task completion (A), whereas most managers defined their team’s productivity in terms of the performance and quality of their team’s outcomes (P).

One takeaway here is that by enabling developers to complete more tasks, you will make them feel more productive, which in turns means they’ll be happier (more on this in an upcoming paper).

2. Developers often have incorrect beliefs about how managers define productivity.

Most IC’s believe that their managers define their team’s productivity in terms of activities and task completion (A), when in fact this is incorrect.

This misalignment has many potential implications. For example, developers may optimize for the wrong behaviors (eg. prioritizing completing their own tasks over the greater good of the team) due to misperceptions about what their managers value. 

To address this, it’s important for engineering managers to clearly define the outcomes that matter, and constantly reinforce them to make sure people are aligned. Ideally, this should be further reinforced by conducting performance evaluations based around outcomes, as opposed to how many tasks or features were completed.

3. Developers and managers share a closer definition of “quality”.

Developers and managers are more aligned in their definitions of “quality” than they are for “productivity”. The researchers created a framework for software quality based on the definitions they collected, called TRUCE. (Timeliness of features delivered, Robustness of features, User delight, Collaboration needs met, and Evolution needs met.)

TRUCE provides a good basis for having conversations about software quality. For example, instead of broadly deciding to “trade quality” for productivity, teams can be specific about which aspects of quality they’re willing to trade to meet short-term needs. 

Final thoughts

With all the hubbub around measuring developer productivity, it’s sobering to realize that our industry isn’t even aligned on what productivity means. On that note, this paper is a great reminder to be careful about what you measure. For example, teams that measure the number of tickets or pull requests completed may be incentivizing developers to focus on task completion instead of driving toward outcomes.

If you’re a manager, this paper is a good reminder to be specific when discussing “productivity” with developers or other leaders. It’s also important to make sure that developers know exactly how you’ll assess their contributions, and by what indicators success will be measured by.

---

That’s all for this week. Thanks for reading.