Characteristics of Code Quality

Developers define code quality in terms of readability, structure, and documentation.

This is the latest issue of my newsletter. Each week I cover the latest research and perspectives on developer productivity.

---

This week I read Developers talking about code quality by Jürgen Börstler and Kwabena E. Bennin, and their colleagues, for the journal Empirical Software Engineering (a journal where I regularly discover new papers). This study sought to identify the defining properties of code quality. The results may be helpful for leaders attempting to measure or improve code quality.

My summary of the paper 

Quality is an important aspect of understanding and driving developer productivity, however teams rarely share a definition of what “code quality” really means. Further, while prior research has explored ways to measure code quality, recent studies have found that common measures lose their predictive power when controlled for the size of the codebase artifact. The authors here also note that common metrics for code quality do not capture quality improvements as perceived by developers. 

For this study, researchers investigated how software developers perceive code quality by collecting interview data, then categorizing and analyzing the themes that emerged. The interviews focused on developers’ general views about code quality, as well as their views on code quality when examining specific code examples. 

Here are the key findings: 

How developers define code quality

Researchers first asked developers about their definition of code quality. These interviews surfaced 116 references to defining properties of code quality. 

After categorizing the data, the researchers found that readability and structure were the most commonly used defining properties for code quality. 82% of interviewees referred to either readability or structure, or both, when describing how they define code quality. 

Next, developers were asked to share their top recommendations for increasing the quality of code. When analyzing developers’ recommendations for increasing code quality, structure and readability were again found as the most commonly referred to properties. Documentation was an additional property that developers frequently mentioned as a target for improving code quality.

Finally, developers were asked to discuss specific code examples, describing in detail which properties or features they liked or disliked with the code and how those properties affect the quality of the code. The researchers found that developers again focused on structure, readability, and documentation.  

Final thoughts

This study serves as a helpful reminder that code quality is a fundamentally human property. There are have been many attempts to measure code quality using quantitative metrics related to complexity or the number of errors, but ultimately the only reliable way to measure code quality—or more specifically, whether code is readable and well-structured—is to ask developers. 

I’ll conclude by sharing a relevant excerpt of an article from Max Kanat-Alexander, the Tech Lead for the Developer Productivity Insights team at LinkedIn. Max wrote this for an issue of

The Pragmatic Engineer

, “Measuring Engineering Efficiency at LinkedIn”: 

At Google, I was a member of the Code Health group, and later led this collective. Within the group, we frequently had people show up to our meetings and ask:

“What metric should I use to measure code quality?” 

We would tell them the same thing, every time:

“Don't use quantitative metrics to measure code quality.” 

People would come and ask us about capturing more exotic measurements, for example, “cyclomatic complexity,” which measures the number of potential paths through a function. Over and again, we would explain these metrics are minimally useful, and that what matters for code quality are characteristics like simplicity, maintainability, and similar things. 

Over time, I started to develop a deeper and more nuanced approach to the question of which metrics to measure:

1. “Simple code” means “easy to read, understand, and correctly modify.” This was my first realization, as I built up a mental model of code quality.

2. Code quality is experienced by a human being, and isn’t a quantity to be measured. Code quality is not a quantity. There’s no “amount” of simple code. Instead, you have to find out from human beings what they are experiencing.

---

That’s it for this week! If you know someone who might like this issue, consider sharing it with them:

P.S. If you’re interested in more research, send me a connection request with the note “research” and I’ll share a downloadable bundle of my top 10 favorite papers of this year.