8 myths on software engineering and AI
What the latest research actually says about AI's impact on developers, and where leaders are still getting it wrong.
Welcome to the latest issue of Engineering Enablement, a weekly newsletter sharing research and perspectives on developer productivity.
This week I’m sharing a new paper I co-authored with Jenna Butler, Margaret-Anne Storey, Travis Lowdermilk, Steven Clarke, and Emerson Murphy-Hill: 8 Myths on Software Engineering and GenAI. Drawing on recent large-scale studies, developer interviews, and field observations, the paper unpacks eight of the most persistent misconceptions about AI in software engineering.
Engineering leaders and teams can use this paper to set realistic internal expectations and benchmark their own AI ROI against real-world data.
My summary of the paper
The eight myths fall into three groups: how developers actually spend their time, how to measure AI’s impact, and how AI gets adopted in real organizations. The through-line is that the gap between AI’s promise and its measured impact has less to do with the models and more to do with the surrounding system of work.
What I hope makes this paper worth reading isn’t that any single myth is new; most have been circulating in the developer productivity research community for the last couple of years. What’s new is putting them side by side and showing how they reinforce each other.
Time, bottlenecks, and lines of code (Myths 1–3)
A 2025 study that I co-authored found that developers spend only 14% of their time writing code, consistent with prior research showing coding hovers between 11% and 18% of a typical day. The rest is design, meetings, review, coordination and administrative tasks.
That makes AI-as-code-generator a smaller lever than the headlines suggest, which aligns with DX’s findings that the typical organization is seeing a 7.8% increase in code throughput. Accelerating one phase can shift load into the next, and 8 Myths found that for one internal AI coding agent, only about half of the PRs were ultimately accepted, with 15% abandoned and 15% stuck waiting on a human reviewer.
The paper posits that measuring impact by lines of code, including AI-generated lines, repeats a mistake the field has known about for a decade. As Bill Gates put it: “Measuring software productivity by lines of code is like measuring progress on an airplane by how much it weighs.” Volume metrics incentivize the wrong behaviors and inflate the very review and quality burden that’s already the bottleneck.
To bridge the gap between bad metrics and true impact, many teams look to PR Throughput as a more complete measure of “work”. Tracking completed PRs is a significant step up from lines-of-code, and is a metric I have long supported, when used appropriately. However, even throughput requires caution in an AI era, and is important to use in a basket of metrics, as it is in the DX AI Measurement Framework.
Effects vary more than the headlines suggest (Myths 4–5)
The research on AI development tools is genuinely mixed. Some studies show large gains, others show neutral effects, and one study of experienced open-source developers found AI tools even increased implementation time by 18% on average.
The variance isn’t random. Familiar tasks benefit more than unfamiliar ones. Developer experience, motivation, and problem-solving style all shape outcomes. Even the prompt matters: one study found semantically equivalent rewrites produced different code in 46% of cases and changed correctness in 28%.
This is further proof why the “10x developer” narrative doesn’t hold up. Productivity gains measured on isolated, toy-sized tasks rarely survive contact with real codebases and real teammates, and prior research suggests much of the variance between developers is a property of the task, not the person.
Adoption is a systems problem, not an individual one (Myths 6–8)
Despite the headlines, only 10% of developers in one Microsoft survey expressed concern that AI tools might take their jobs. Most describe AI as expanding their creative capacity. More time on architecture, mentorship, and brainstorming, less on lookups.
But adoption itself is harder than the market assumes. The paper showed that 80% of developers use AI tools, but only 29% trust their accuracy. Recent research also identifies a “competence penalty”—developers, particularly women and older engineers, receive harsher evaluations for AI-assisted work even when the output is identical.
And almost all of the existing research still studies a single developer paired with a single tool, placing the burden of productivity on the individual. Historically, real productivity gains haven’t come from individuals optimizing their own work, they’ve come from systematic changes at the organizational level. AI may be the first technology where organizations have spent millions on licenses without a clear plan for how to extract value from them.
Final thoughts
The through-line across these eight myths is that AI’s impact in software engineering is shaped more by the system around the developer than by the developer themselves. Coding is a small share of the work. Lines of code is a poor measure. Effects vary by task, person, and context. And adoption depends on trust and organizational support far more than on tool capability.
For engineering leaders, that points to a different set of questions than the ones the market tends to ask. Not “how much code did AI write?” but “where in our system is AI actually relieving friction, and where is it just shifting pressure downstream?” Not “how do we get developers to use it?” but “what would it take for our developers to trust it?”
Treating AI adoption as an engineering systems problem, not a productivity hack, is what separates the organizations seeing real value from those still chasing the myths.
This week’s featured DevProd job openings. See more open roles here.
Ashby is hiring an Staff Platform Engineer | Remote
BambooHR is hiring a VP of Engineering | Utah (Hybrid)
Cashea is hiring an Infrastructure & Developer Productivity Platform Engineering Manager | Remote
Figma is hiring a Staff Software Engineer, Developer Experience | Remote; US
Morgan Stanely is hiring an AI Platform Engineer - Vice President | New York
That’s it for this week. Thanks for reading.