Subscribe now

In case you missed it, we just released the DX Annual recording library. You can watch the recordings by clicking through the Sessions page here. We’ll release recordings on the podcast soon as well, stay tuned.

There are several assumptions about AI circulating in executive teams and boardrooms that are shaping strategy today. For example, some leaders believe AI will ultimately mean fewer developers, or at least fewer junior engineers. Some believe AI adoption must be mandated from the top in order to take hold.

These beliefs are rarely black and white. So at DX Annual, we convened a panel of senior engineering and research leaders to debate these questions in the open. Our aim was to surface the nuance behind these assumptions, give our audience exposure to contrasting viewpoints, and equip them to articulate their own position with greater clarity and confidence.

The panel:

• Rafe Colburn, Chief Product and Technology Officer at Etsy, where he’s led engineering for 14 years through multiple waves of technology change

• Jesse Adametz, Sr. Director of Engineering, Platform Engineering at Twilio, overseeing developer tooling and productivity for thousands of engineers

• Eirini Kalliamvakou, Research Advisor at GitHub, whose work on developer productivity measurement has shaped how the industry thinks about engineering effectiveness

• Collin Green, Senior Staff UX Researcher at Google, where he studies developer workflows and the factors that drive or block productivity

• Brian Houck, coauthor of the SPACE framework (and now Distinguished Researcher at DX), whose research on AI’s impact on engineering spans many thousands of developers and managers

Each panelist was read a series of statements and asked for their opinion. Below is a recap of each statement and the panelists’ reactions. You can watch the full session here.

Will AI mean fewer engineers?

Statement: An AI-first SDLC means fewer engineers.

Rafe: Thumbs down. A job is a bundle of tasks, and AI is a clean substitute for some of them. People might not be doing those things anymore. But the demand for software isn’t going away, it’s going up. The price of building software is going to go down. So I don’t think fewer engineers is a likely scenario in the near future. I bet against it.

Brian: I agree, it’s not about fewer engineers, it’s about doing more. But one nuance: in the future, what do we even define as a software engineer? The bundle of tasks we think of today that make up a software engineer may change. Writing code is a core part of the identity. The people who do that specific thing may go down, but the total number of makers and builders won’t.

Collin: You’ll probably be able to do the same amount of stuff with fewer people, but that doesn’t mean demand will go down. Small companies that want to accomplish something will be able to get over the barrier to entry faster. It may actually increase demand for engineering skills.

Is AI accelerating technical debt?

Statement: AI is currently creating technical debt faster than it is helping us refactor it.

Eirini: Because you said “currently,” yes. Things with code generation are moving at a pace where the rest of the system hasn’t evolved enough to match it. We’re also seeing the anticipation of technical debt throttling how much developers use agents. They’re trying to balance the risk of something with the velocity of something, and it’s mental math that has to happen in the moment. There are ways to manage against the accumulation, like event-driven or scheduled agents that do maintenance and hygiene for codebases. It’s not a perfect solution, but it’s working for now. The accumulation of technical debt is certainly a big issue.

Jesse: It’s too definitive of a statement. We look at our engineers in quartiles, and our highest-performing engineers are also our highest-performing AI engineers. AI is an amplifier. If you’ve got high-performing engineers, they weren’t putting garbage into the system before and they’re still not. It’s garbage in, garbage out.

Rafe: I’ve worked at Etsy for 14 years. Proportionally, are we creating more tech debt now than five or ten years ago? No. The old code is still there. We’re creating a greater volume of code, but I’m not sure a greater proportion of it is tech debt. Automated code reviews are helping, and you can ask a coding agent to look at patterns in the code. The greater volume probably brings more tech debt in absolute terms, but I don’t think it’s an accelerator for tech debt at a greater rate than it’s an accelerator for anything else.

Brian: Disagree with the others. Organizations are optimizing for PR velocity, not cleanliness. And cognitive debt, understanding our systems less as AI writes more of the code, is a form of technical debt that’s growing. In the short term, tech debt is going up relative to PR velocity.

Collin: Technical debt is a business decision, not strictly an engineering one. If businesses have the same tolerance for risk and the same market pressures, they’ll incur the same total amount. But viewing agentic development as low-cost increases the risk of imprudent decisions.

Eirini: The choice of what even are you building becomes so important.

Will most code be AI-generated within five years?

Statement: In five years, more than 50% of code in most organizations will be written by AI.

Collin: If we’re talking about new code, yes, maybe even sooner than five years. But it’s worth asking whether that’s code that would have been written by humans otherwise, or just a larger total volume. Also worth considering how much of it will be rapidly rewritten.

Jesse: I interpreted the statement differently. If the claim is 50% of all code in a codebase, that’s wild. Too many companies have too much legacy software, and there’s no advantage to rewriting all of it.

Rafe: Agreed it has to be new code for the statement to hold.

Is the future of engineering about managing agents?

Statement: The future of software engineering is more about managing agents rather than about writing code.

Brian: In an idealized world, yes. But humans aren’t great at multitasking. Microsoft’s internal research shows that even experienced developers under time pressure revert to sequential, single-threaded agentic workflows. People with prior management experience (delegation, context-switching) will be more successful, but it’s a different skillset that we’re not hiring for yet.

Rafe: Thumbs up, loosely. Even using Claude Code today, you’re already managing multiple agents, it’s just abstracted away. The hobby-project vision of orchestrating 15 agents simultaneously isn’t the future. AI will help manage that complexity. But will a large chunk of your work be mediated by at least one agent? Yes. Agent orchestration is not a full-time human job.

Jesse: As a director, my brain naturally goes to managing agents. But the identity of a software engineer matters. Not everyone wants to be a manager, and not everyone is good at it. A blanket statement that everyone will manage agents is too much.

Eirini: I interpreted “managing” as the real engineering work of setting agents up for success: defining intent, setting constraints and guardrails, providing context, and then verifying output. That’s real engineering at a different level of abstraction. If that’s what the statement means, it’s a strong thumbs up.

Collin: Micromanaging agents is a failure mode. That leads to task-switching bottlenecks.

Do leaders need to mandate AI adoption?

Statement: Leaders need to mandate AI usage to make sure adoption is moving along.

Jesse: Not what we’ve seen. Twilio did light enablement (install defaults, basic guidance), and adoption took off on its own. No top-down mandate needed.

Rafe: People outside of engineering orgs often push for mandates because they’re afraid of falling behind. But mandates lead to shallow adoption and “tokenmaxxing.” AI adoption is inevitable, so why mandate something inevitable? If someone had said a year or two ago that every single talk at the DX conference would be about AI, people would have said no way—but is there anything else to talk about? Telling software engineers AI is going to be part of their job just feels like you’re fighting last year’s battle this year. Leaders should focus on removing friction and making it easy to create value, not pushing people.

Brian: I’m running a study of ~600 engineers and managers. The biggest disagreement: the majority of engineering managers think AI usage is a reasonable individual performance metric. Engineers disagree. That’s a myth to dispel. Activity metrics are useful for understanding patterns, but should not be used as direct performance measures.

Jesse: If organizations aren’t crystal clear about how AI will be measured, people will fill the gap with their biggest anxieties. Twilio isn’t putting AI usage in career frameworks anytime soon, but the reality is you’re compared to your peers.

Rafe: If you went to a job interview today and said “I don’t use AI coding tools,” it would be like saying you’re a programmer who refuses to use a text editor. It would just sound strange. Adoption is inevitable without a mandate.

Is code review now the bottleneck?

Statement: The bottleneck of software delivery is no longer writing code, it’s now reviewing code.

Collin: As Brian noted earlier in the day, developers only spend about 14% of their time writing code, so code was never really the bottleneck. The real bottlenecks are decision-making, prioritization, product design, support, and operations.

Jesse: AI is an amplifier. Whatever your organization’s bottleneck used to be, it still is. For Twilio, deep work has been an issue for a long time and still is. The conversation has shifted to “agent experience,” but the fix is the same: improve developer experience. Meetings, coordination tax, decision-making, these have always been the problems.

Brian: Code review is still a bottleneck, and it’s increasing as more code is produced. But at Microsoft, some features take two or more years from planning to customer delivery. Going from two days to three days on review isn’t the long pole. Planning and prioritization are.

Rafe: The perceived bottleneck of code review is what’s interesting. When writing a PR took two days, a review delay was annoying but tolerable. When writing takes 10 minutes, the human process of finding a reviewer and waiting feels unbearable. The interruptions of human processes, unless they provide clear value, are grating.

Jesse: Twilio’s data showed their highest-quartile AI developers took a ~14-point hit in perceived code review turnaround, but their actual median merge time decreased by multiple hours. It’s pure perception.

Is risk the only thing holding back adoption?

Statement: The only thing holding engineers back on AI adoption is unnecessary worry about risk.

Eirini: Engineers feel deeply accountable for what ships. When something is high-risk or hard to undo, they pull back on how much they leverage agentic velocity. That’s not irrational, it’s human behavior that’s hard to get over.

Brian: Risk is a big reason, but not the only one. Over a fifth of developers in our research cited concern about introducing defects and vulnerabilities as a top barrier.

Jesse: I question whether AI is actually increasing risk, since engineers were already introducing bugs before AI.

Rafe: The risk landscape has completely shifted. If we had a butter knife and now you have a chainsaw, the risks are different. The risks that you won’t be able to cut down a tree have gone way, way down. But the risks that you might really make a mess are a lot higher. We have people using these tools who don’t really know how a chainsaw is different than a butter knife — “Look, someone just gave me something, cool.” And on the other hand, engineers who are like, “I don’t want to use the chainsaw unless I really understand everything about how it works.”

Collin: Beyond risk, there are basic enablement problems: helping people choose tools, configure them, and get started. Those aren’t AI-specific, they’re just adoption problems.

Are AI adoption problems really culture problems?

Statement: Most AI adoption problems are really culture and management problems, not tooling problems.

Rafe: Culture and management problems are real, but sometimes the next blocker is a tooling problem, or a process problem. Process is a form of tooling. At Etsy, with a 20-year-old codebase and deeply tenured engineers, stated and unstated processes create friction. It’s all of the above. But if you’re a leader and you’re not giving people time to learn, you are doing it wrong. I used to ask engineers, “Are you working as fast as you can?” “I don’t know.” “Are you learning anything from what you’re doing?” “No.” Then you’re working as fast as you can. You’re not taking any time for that. Making space for learning solves 90% of these problems.

Collin: Most problems are human problems. Tool impediments exist, but the bigger barriers are incentives, anxiety, learning, and space to learn. Leaders who give people time to learn demonstrate they understand how the world actually works.

Eirini: The transformation is massive. Bottlenecks have shifted, processes aren’t working as expected, metrics no longer mean the same thing, and people are having an identity crisis. It can’t be just a tooling problem. Culture, motivation, and infrastructure all need to adjust.

Jesse: There’s a compounding effect. For people who haven’t started, it’s getting harder to start. I told my VP a couple weeks ago, “I have to quit to be able to keep up with the industry.” That’s the day job. So if folks haven’t gotten started, I can only imagine how much more daunting it’s getting.

Brian: Everything is moving so fast that what you learned yesterday is irrelevant tomorrow. Long-term burnout and fear of falling behind are basic human challenges. No matter how complex the technology, it’s always a human problem at the end of the day.

Eirini: Teams that did two-week group learning sprints (everyone uses agents for everything, no exceptions) saw much better results in adoption, engagement, and outcomes than teams where individuals learned on their own.

—

There’s a common throughline in these discussions: AI is reshaping the task mix of software work, but not eliminating the need for engineers. Additionally, the biggest risks and opportunities sit in how leaders design roles, measure impact, and consider the full PDLC, not in whether AI can generate more code.

To go deeper, you can watch the full session here and explore the rest of the DX Annual recordings on the Sessions page.

This week’s featured DevProd job openings. See more open roles here.

• Ashby is hiring an Staff Platform Engineer | Remote

• BambooHR is hiring a VP of Engineering | Utah (Hybrid)

• Cashea is hiring an Infrastructure & Developer Productivity Platform Engineering Manager | Remote

• Figma is hiring a Staff Software Engineer, Developer Experience | Remote; US

• Leidos is hiring a Platform Engineer | Remote; US

• Morgan Stanely is hiring an AI Platform Engineer - Vice President | New York

• Weave is hiring a Senior Platform Engineer, Data Infrastructure | Remote; US

That’s it for this week. Thanks for reading.

Share

Subscribe now

Hi everyone 👋 I’m Brian - I’ve recently joined DX as an Applied Scientist. I previously led research for EngThrive, Microsoft’s company-wide initiative for measuring and improving developer experience. I’ve been part of this newsletter before and am now excited and honored to be a regular contributor.

For my first issue, I’m summarizing a new paper I co-authored with Rudrajit Choudhuri, Eirini Kalliamvakou, and Thomas Zimmermann, published in ACM Queue: The AI-Native Developer.

In this study, we explore how AI is reshaping software engineering by analyzing survey data from over 1,300 developers and interviews with 22 AI-fluent practitioners. We specifically look at what developers want from AI and how AI fluency develops, and discuss three possible futures for the developer role.

What developers actually want from AI

To understand where developers want AI involved—and where they don’t—we mapped their daily work across four dimensions: value, identity, accountability, and demands. This revealed three clusters. Core Work (coding, debugging, system design, testing) scored high on all four. Ops & Coordination (DevOps, documentation, stakeholder communication) scored high on value and demands, but weakly on identity. People & AI Building (mentoring, AI feature integration) scored moderately across the board, with relatively strong identity alignment.

These clusters map onto what we call an “AI opportunity space”, a plot of developers’ openness to AI support against their actual AI usage. Core Work and Ops & Coordination tasks both landed in the Build and Improve zones. Developers were open to deeper AI assistance for everything from cross-artifact debugging to environment provisioning, but reported that current tools hadn’t caught up. This suggests the barrier is less reluctance and more trust.

For production-facing work, developers consistently emphasized predictability, transparency, verifiability, and human oversight.

The De-prioritize zone was the most revealing. Relational work like stakeholder communication and customer interactions, landed here consistently. Developers saw little role for AI beyond preparation. They wanted to retain final voice and responsibility.

How AI fluency actually develops

In our interviews, we observed a consistent four-stage journey from initial resistance to full orchestration.

The Skeptic starts with rational resistance. For skeptics, AI suggestions interrupted flow and created extra work. What shifted skeptics wasn’t persuasion; it was urgency. Developers saw AI fluency becoming a competitive advantage and pushed through the frustration. One put it plainly: “Either you have to embrace the AI, or you get out of your career.”

The Explorer emerges when AI starts to genuinely help. A cryptic error decoded, a familiar task automated, mundane work handled. Each small win chips away at skepticism.

The Collaborator integrates AI into the workflow earlier rather than using it only as a fallback. They treat iteration as part of the workflow rather than a recovery mechanism when AI output fails.

The Strategist orchestrates multiple agents in parallel: one defining test cases, another implementing, a third reviewing for security. They set context, review output, and decide what to delegate next. Strategists don’t just prompt, they front-load context. Some even have the AI “interview” them to surface missing requirements and business goals before a single line of code is generated. They have moved from being code producers to architects of intent. Ask them about the future, and they’ll tell you AI will write 90–100% of code within two years. What’s striking is that this prospect energizes rather than threatens them.

The tensions that don’t have easy answers

Even the most optimistic strategists we interviewed raised concerns.

The deepest was what we call the learning paradox. Mid-career developers wondered whether they were getting better at software engineering or “just better at prompting.” What does mastery look like when the skill is orchestration rather than implementation?

Deskilling was a related concern. Senior developers learned their craft through years of writing, debugging, and refactoring code. If junior developers delegate from day one, what foundational skills might they skip? Several interviewees questioned whether current onboarding approaches would prepare juniors for AI-augmented work or leave them dependent on tools they don’t fully understand.

Accountability runs underneath both. Developers put their names on code they didn’t directly write. Review and verification become the primary mechanisms for maintaining understanding and accountability, but as velocity increases, can verification norms keep pace, or will the pressure to ship quietly erode the guardrails that make delegation safe?

Three futures, and the choices that decide which we get

The paper closes by extending current patterns into three plausible futures.

Human craft at AI speed. Developers consciously choose to remain hands-on, using AI as an accelerator rather than a substitute. Tasks compress, but the shape of the work remains recognizable.

Orchestration and blended work. Software engineering is defined less by writing code and more by orchestrating intelligent systems and exercising judgment. Verification, mentoring, and design decisions take primary place; implementation gets delegated.

The Clerical Coder. In the paper’s dystopian future, AI generates all production code. Humans rubber-stamp at an industrial scale, accountable for outcomes they don’t fully understand. This is the rise of the “Software Signatory”, where you are rewarded for the volume of approvals rather than the depth of your craft. As the paper warns: If we reward speed without craft, we get volume without depth. When failures happen, the human reviewer is blamed.

There is no single trajectory here. Different teams will land in different places depending on their context and incentives. The outcomes are shaped by what organizations choose to reward and protect.

Final thoughts

AI adoption has increased, and the gap between how developers actually work and how they want to work has not closed. If anything, it has widened.

That suggests the framing of AI as a productivity tool—one that reduces toil and frees up time for meaningful work—may be too simple. The tasks developers find tedious aren’t necessarily the ones they trust AI to handle. And the tasks they trust AI to handle aren’t necessarily the ones creating the gap.

The more useful question for leaders isn’t “what can we automate?” but “are our tools earning the trust to touch what developers find meaningful?” and “what about the work is worth protecting?”

In case you missed it, we just released the DX Annual recording library. You can watch the recordings by clicking through the Sessions page here. We’ll release recordings on the podcast soon as well, stay tuned.

That’s it for this week. Thanks for reading.

Share

Subscribe now

If you weren’t able to attend DX Annual last month, or if you want to revisit specific sessions, all recordings are now available at dxannual.com.

DX Annual brought together ~500 senior engineering leaders from companies like Microsoft, Airbnb, Uber, Vanguard, Dell, and BNY for a day of practitioner-led sessions on what’s working and what isn’t as AI is integrated across the software development lifecycle.

Below are the key themes that emerged across sessions, along with links to the recordings worth watching for each.

The bottleneck of software delivery is no longer writing code

Microsoft’s research shows engineers spend only ~14% of their time writing code. AI has accelerated that 14%, but the other 86%—code review, testing, documentation, deployment approvals—is where meaningful gains are waiting. Vanguard found engineers are 30% faster at coding with AI, but end-to-end cycle time hadn’t moved until they started applying AI across the full lifecycle. Twilio found that top-quartile AI users saw a decline in review turnaround time even though merge time decreased by hours.

Watch: Keynote with Abi Noda & Brian Houck | Vanguard session | Closing panel

AI token spend is top of mind and organizations are measuring it differently

One organization (300–400 engineers) shared their token spend reached $128,000/week and is climbing toward $150,000. They operate with no caps, believing the opportunity cost of slowing adoption outweighs the bill. Some individual developers spend up to $20,000/month. Despite the headline number, their cost per pull request is actually decreasing as throughput increases.

1Password framed the AI token bill as the new cloud bill—something that must be managed with the same rigor as AWS spend. Their advice: stop using high-powered models for simple tasks, and forward-project consumption to negotiate better per-token rates with providers.

Etsy cautioned against top-down usage mandates that incentivize “tokenmaxxing”: employees generating unnecessary activity just to hit metrics.

Microsoft argued that leaders have a responsibility to create headroom to learn, which includes spending tokens on experimentation that may not produce immediate value. The fastest learners will have a competitive advantage. (They acknowledged Microsoft is in a unique position here.)

Watch: Opening panel | Closing panel

Structured training and skill quality matter more than tool choice

Indeed put ~2,000 engineers through a structured, tool-agnostic AI course. Engineers who completed it saw a 36% reduction in coding time; those who didn’t saw zero change—despite using the same tools at 97% adoption. Another big tech company took a complementary approach: narrowed tool options to two, created one-click setup, and built a champion network of ~50 engineers running peer-led onboarding sessions.

Similarly, Microsoft observed that most engineers, when under pressure, revert to single-threaded, pairing-style workflows rather than orchestrating teams of agents.

Watch: Indeed session | Closing panel

Leaders are incorporating more outcome-focused metrics to measure AI impact

Uber is evolving their measurement to include feature velocity, which captures value delivered regardless of whether a human or agent wrote the code. Dropbox shared that they adopted feature velocity as their primary measure of engineering effectiveness for the same reason: it ties directly to outcomes, not activity.

Some organizations are measuring AI through three areas, covering cost management (spend by team and project), engineering effectiveness (how well AI is being used), and value impact (feature velocity, quality indicators).

Watch: Uber session | Dropbox session

Productivity gains require daily, sustained AI usage

Airbnb segmented engineers by daily AI usage and found the returns aren’t linear. Engineers using AI 4+ hours per day more than doubled their output versus pre-AI baselines. Casual users saw only modest improvement. Organization-wide, Airbnb is seeing 65% higher throughput and 59% AI-authored code with no mandate in place. Additionally, Intercom showed what sustained usage looks like at scale: 95.9% of PRs authored by Claude, throughput doubled in 9 months, defect backlog down over 50%.

Watch: Airbnb session | Intercom session

PMs and designers are shipping code, and organizations are adapting

Airbnb has 2× as many AI users as developers—PMs, data scientists, designers, even their finance team independently onboarded themselves to VS Code. Mercari saw a 45.7% drop in accounting tasks and 60% reduction in help desk workload from AI tools operated by non-engineering teams. Vanguard is embedding AI across PMs, designers, and QA, treating the full team as the unit of productivity improvement.

Watch: Airbnb session | Mercari session | Vanguard session

All sessions

For the full list of recordings, visit the Sessions page at dxannual.com.

DX Annual was an invite-only event. Stay tuned for 2027 dates to be released soon.

Share

Subscribe now

In an opening session at DX Annual, Abi hosted a discussion with Tim Bozarth (CVP, CoreAI at Microsoft), Nancy Wang (CTO, 1Password), and Taroon Mandhana (CTO of AI & Teamwork at Atlassian) to learn how they’re adapting to the impact AI is having now and in the future. The panel covered topics such as org design, managing costs, and how the developer role is changing.

Below is a lightly edited excerpt from the discussion. You can watch the recording here. Stay tuned for the full DX Annual recording library to be released next week.

What does an AI-native SDLC look like?

Tim: If we use a five-stage frame: plan, create, validate, deploy, operate. Create is already changing. AI is incredibly good at writing a line of code. Whether it’s good at building complex systems remains to be seen. Historically, roughly 80% of engineering time goes to operate, 10 to 15% to create, and the remainder splits across plan, validate, and deploy. In the most effective teams, that’s inverting: plan and validate now consume the majority of time, because create and operate are compressing. Those are the stages where humans are tastemakers, where understanding of craft matters most. But don’t delegate validate to AI yet. We still need humans in the loop for important systems. And don’t delegate security to AI. Use it for pen testing, use it for red teams, use it to battle-harden your system, but don’t trust it to deliver secure products on its own.

“Roughly 80% of engineering time goes to operate. In the most effective teams, that’s inverting: plan and validate now consume the majority of time.”

Nancy: We’ve stopped writing full-length PRDs at 1Password. Teams build prototypes and put them in front of customers instead. That’s eliminated almost half of the back-and-forth where engineering asks product, “How do you handle this edge case?” You’ve already answered that during the plan and validate phase. But think about the SDLC as a pipe: adding more to the front means you add more to the back. More makers writing PRs means more code reviews, more reliability and maintainability work. We’re running an experiment with a reinforcement learning lab to build a DevOps agent customized to our environment, trained on real incident data and how our engineers respond. The goal is to delegate a significant portion of operate to an agent.

Taroon: The area with the most untapped potential is operate. Our engineers spend an insane amount of time responding to alerts, customer issues, and incidents. We’re seeing agents start to respond to alerts and only wake up a human if it’s real. Post-incident reviews are getting automated. 50% of simple vulnerabilities at Atlassian, things like library version bumps, are now resolved using AI. Accessibility bugs that were taking a backseat are getting done much faster. A lot of this is being run in the background by central dev infra teams to give time back to engineers.

Has AI changed how your engineering orgs are structured?

Taroon: We haven’t restructured the org. What’s changed is how teams form. For zero-to-one projects, we’ve moved to squads of 3 to 4 people. That would have felt too small a year ago, but AI has compressed the building part enough that the bottleneck is now alignment and decision-making. You don’t need 8 engineers when 3 can move faster, as long as they have the context and the authority to make calls without waiting on a chain of approvals. The org chart looks the same. The way work actually happens inside it looks very different.

Tim: Similar story. We haven’t redrawn boxes on an org chart. What we’ve done is shift to eight-week cycles with small, mission-specific v-teams. The point of those teams isn’t sustained delivery. It’s speed of learning. You form a team around a specific question, give them 8 weeks, and then decide whether to continue, absorb the work into a larger team, or stop. That cadence lets us stay responsive without constantly reorganizing.

Nancy: At 1Password, the biggest shift has been in planning horizons. We used to plan 12 to 18 months out. Now we’re down to a single quarter. When the tools and the capabilities are changing this fast, anything beyond 90 days is guesswork. That compression hasn’t required a reorg, but it has required a completely different operating rhythm from leadership. You have to be comfortable making decisions with less certainty and revisiting them more often.

Do any of your organizations mandate AI usage?

Tim: No. We track daily active AI use across Microsoft, and we invest heavily in enablement, training, and incentives to make adoption easy.

“We don’t set targets for usage itself. The metrics we actually care about are speed, ease, and quality.”

Activity is a diagnostic signal. If adoption is low in a particular team, that tells us something is blocking them, and we go figure out what it is.

Taroon: Same philosophy. We have a central team that ensures easy access to tools and removes friction from onboarding. Beyond that, what’s worked is organic champions. In groups of 100 to 200 engineers, someone naturally emerges who’s excited about this stuff and starts showing others what’s possible. We amplify those wins across the company. That’s been far more effective than any top-down mandate.

Nancy: We built a guild of AI champions at 1Password. The thing that actually moves adoption is celebrating concrete examples. When someone uses AI to pull a launch date forward by two weeks, and you tell that story publicly, it does more than any mandate ever could. People see the result and want to figure out how to get there themselves.

How are you managing AI costs?

Taroon: I’m on my third budget forecast since January. Token costs are volatile, and the models and pricing are shifting underneath you constantly. It’s the hardest part of financial planning I’ve dealt with in a while. We literally started to think about managing it like AWS COGS cost because it requires that level of rigor and sophistication.

Nancy: We built an internal SaaS cost management tool that maps token spend by repo and project. Without that visibility, you’re flying blind. That lets us map token spend back to intent, so we know what tokens are actually being spent on. We’re also looking at guardrails and abstractions, like mapping token usage by build volume to a CI automation service, so if something goes out of whack, you can catch it. Having frameworks and hypotheses about how tokens are being spent in your organization can prevent surprise conversations with your CFO. My recommendation to other leaders: negotiate forward-projected commitments with your model providers. If you can commit to volume, you can significantly reduce your per-token cost.

“Treat your AI token bill the same way you’d negotiate a cloud contract.”

Tim: One thing I’d add is that not every token needs to produce direct value. Some of this spend is learning and experimentation. The fastest learners will win, and that means we are going to spend tokens on things that do not create value, but they create learning amongst the workforce. Teams need room to try things that don’t immediately pay off, and leaders need to create headroom for that when they’re reporting to boards.

How do you define the skills profile of a great engineer 3-5 years from now?

Tim: Projecting three to five years out right now is hallucinating. What we focus on is what’s durable. And the pattern we see consistently, inside Microsoft, with our customers, in the startup community, is the maker’s mindset. You’re not attached to a specific tool. You’re oriented toward an objective, toward building something that creates a business outcome, and you drive toward it with whatever you can. That’s what we’re looking for in new hires and encouraging in existing engineers through how we structure incentives, rewards, and how we talk about the speed of experimentation and learning.

Nancy: I’ve shifted between product and engineering roles throughout my career, and the lines between them are blurring fast. What I’d look for is generalists with strong product instincts. The best advice I give new grads right now is to try to span the entire product development lifecycle through the software development lifecycle. Don’t specialize too early. The engineers who are seeing the most impact from AI are the ones who can move across codebases, work across craft, and operate at a higher level of abstraction. They’re not just writing code faster. They’re making better decisions about what to build and how to ship it, because they understand the full picture.

Taroon: I’d add that agency is becoming as important as technical depth. The willingness to step up, have the right conversations, and make decisions without waiting to be told. When AI compresses the time it takes to build something, the differentiator becomes the person who can figure out what to build next and move on it. That’s not a technical skill. It’s a disposition. And it’s something we’re starting to look for explicitly in how we evaluate engineers.

Where are the boundaries when non-engineers contribute code?

Taroon: Designers at Atlassian are submitting PRs. That’s real, it’s happening today, and it’s genuinely useful. The fidelity of early conversations has gone up dramatically because instead of debating a static spec, you’re looking at something interactive. But I want to be honest about the friction. Engineers are escalating daily about quality issues in those contributions. Prototyping by non-engineers is a clear unlock. Shipping to production is a different bar. The teams that are most comfortable accepting contributions from non-engineering roles are the ones with robust test suites and deployment checks already in place. If your right-of-code processes are weak, AI-assisted contributions from anyone, engineers included, are going to cause problems.

More broadly, we’re seeing patterns of duplication and tech debt increasing as people quickly produce features. The maintainability of the code is suffering. It’s prompted us to go back to standardized approaches and more right-of-code quality checks.

Nancy: Our CX associates are now generating PRs for front-end test coverage across browsers and mobile clients. These are people who know the product deeply from a customer perspective, and they’re using AI to translate that knowledge into actual test code. Engineering’s role has shifted. Instead of writing those tests themselves, they’re building the testing harnesses and review processes to evaluate the contributions. It’s a different kind of engineering work, but it’s just as important, and arguably higher leverage.

Tim: There’s a second dimension to this that I think gets overlooked. Yes, more people can now take an idea in their head and turn it into something you can interact with. That’s the prototyping side. But the non-engineers who are getting the most out of AI aren’t contributing code to products. They’re using it to optimize how they work: how they gather and distill information, how they communicate, how they run their own workflows. That doesn’t go to production in the traditional sense, but it goes to production in how the business operates. And that’s happening right now.

You can listen to the full discussion here.

This week’s featured DevProd job openings. See more open roles here.

• Ashby is hiring an Staff Platform Engineer | Remote

• Cashea is hiring an Infrastructure & Developer Productivity Platform Engineering Manager | Remote

• Figma is hiring a Staff Software Engineer, Developer Experience | Remote; US

• Leidos is hiring a Platform Engineer | Remote; US

• Weave is hiring a Senior Platform Engineer, Data Infrastructure | Remote; US

That’s it for this week. Thanks for reading.

Share

Subscribe now

Over the past 16 months, DX has been running a longitudinal study on AI’s impact on engineering velocity across a sample of more than 400 engineering organizations. We found that as AI tool usage increased by an average of 65%, median PR throughput increased by just under 8%. Most organizations are landing in the 5–15% range—a meaningful gain, but far below the 3x or 10x expectations many leaders are being held to.

That gap raises some questions. Why aren’t the gains higher? Where is reclaimed time actually going? And how should leaders be thinking about where to invest next?

This is what and I discussed to kick off the mainstage program at the inaugural DX Annual, which brought together roughly 500 senior engineering and developer productivity leaders for a day focused on navigating the AI era. Brian is a well-known developer productivity researcher, and someone we’ve referenced often in this newsletter—from past interviews with him (here and here) to summaries of papers he has co-authored (like this one and this one). In this conversation, Brian asked me about the study methodology and what sparked this research, as well as what’s limiting gains, how to measure AI’s impact, and how leaders can unlock greater gains.

Today we’re releasing the full report. You can download it here: AI and Engineering Velocity: A Longitudinal Analysis.

What follows is a lightly edited transcript of my conversation with Brian at DX Annual.

Brian: To kick us off, Abi—you’re sitting on one of the largest datasets on how engineers work in the real world. This new report digs into what’s happening as AI adoption scales across the industry. Before we get into the findings, what motivated this research?

Abi: Thanks, Brian. A little background on this study: Nearly every conversation I’ve had with folks over the past few months has started the same way. My CEO, our executives, are expecting astronomical gains because there’s so much hype about AI in the media, but that’s not what we’re seeing on the ground. How do we close that gap or set realistic expectations? How do we even know what to aim for?

I’d bet everyone reading this has experienced that to some extent. So at DX, we started asking ourselves the same question. What can we see in the data about what companies are actually seeing? That’s what we’ve been digging into. This is just the beginning; there’s a lot more to unpack and a lot of nuance, but at least we now have a first look at what’s happened over the past 16 months as AI adoption has matured across companies.

Brian: I’d love to nerd out on research methodology for a minute. Walk us through how DX actually investigated this. How did you design the study and decide which companies to include?

Abi: Our customer network at this point is about 500 companies. For this study, we took a sample of those. We looked for companies that had reached a point of maturity in their developers’ adoption of AI: we defined that as over 75% monthly active usage of AI coding tools. In almost all cases, there’s been a significant rise in adoption over the past 12 months. We also narrowed the scope to companies with over 100 engineers and excluded companies that had gone through major liquidity events, M&A, IPOs, or regulatory changes that could have had an impact.

In terms of methodology, it’s really hard to single out causality. What is AI and what are confounding factors? One approach I see a lot is cross-sectional analysis: comparing developers who use AI to developers who don’t. There are flaws in that approach because oftentimes the developers using AI were already the ones coding the most. They always look better.

What we did instead is look at longitudinal data at the organizational level. As AI adoption has matured, what has been the impact on organizational velocity and throughput?

Brian: What were your hypotheses, and what did you actually find?

Abi: As a developer myself, I had some gut-feel hypotheses, but honestly, from a study standpoint, we really didn’t know. We were pushing the team, asking “what’s the answer?” We wanted to know just as much as you probably do.

What we ultimately found is that most organizations fall in the 10-15% range in terms of PR throughput increase. The actual median was around 8%, the mean was around 11%. So a real gain, but well below what most leaders expect.

It’s much more modest than many CEOs, who are talking to their CEO friends about the 10x gains they’re supposedly delivering, would expect. In our engagements with companies and customers, it wasn’t completely surprising. There are a lot of factors underlying that, which we’ll get into.

Brian: One of the central themes of the SPACE framework is that developer productivity is nuanced and about much more than counting activities. So given that, why did you choose PR throughput as your primary measure?

Abi: Measuring velocity or productivity is inherently challenging. We did look across a number of different metrics, including the full spectrum of SPACE and the DX Core 4. But for what we wanted to focus on and publish, we felt PR throughput was the most relevant and practical right now, mostly because that’s what we see most organizations talking about and focusing on. We wanted to meet the world where it’s at.

We also explored our proprietary metric TrueThroughput, which uses AI to weight each PR by relative size and complexity so it takes some of the noise out. We liked that signal, but again, for relevance and practicality, PR throughput is a more accessible metric. We have more data on other metrics and how those have been affected, and we’ll eventually publish those as well.

Brian: I suspect that, for many, a 5%, 7%, 10% increase in PR throughput matches what they’ve been feeling. But as you mentioned, when I talk to business leaders, they’re often expecting 20, 30, 40, 50% increases in productivity. So why do you think the gains are lower than what people outside the industry might expect?

Abi: Our team conducted follow-up interviews to really dig into that question. There were a number of factors. These are just the top themes, not the full list.

The top one probably wouldn’t surprise most of us: coding is not the primary bottleneck for engineers. Brian, at Microsoft, you published a study on where engineering time is spent, and only about 14% of developer time is spent coding. It’s a very small part of where our time and money goes. So if AI is only optimizing that slice, it partially explains why the enterprise velocity gains are limited.

We also heard how AI tools and new automation introduce new bottlenecks. Things like review burden, technical debt, cognitive debt—which is a newer concept we can talk about. We also heard about challenges with adoption—a lot of social friction and cultural clashes that are inhibiting full adoption. There were a few more, but those were the major themes.

Brian: You don’t have to go far out in the distribution before you see some organizations with substantially higher gains. Any time the median and mean are that different, it implies outliers. So what set apart some of the companies with disproportionately high gains from the more typical experience?

Abi: We don’t have a great answer to this yet. Our first focus was on why the gains aren’t higher. The next question is: for those where gains are higher, why?

There are two parts to how we want to approach that. One is that we’re working to tease out whether the companies at the top of the range are seeing real, material gains or superficial, inflated numbers. We want to peel back the onion.

Two, assuming the gains are real, we want to understand what strategies and factors are driving those results. I think a lot of it boils down to what I’ve seen anecdotally: an all-in culture, a fully bought-in organization with centralized rollout and championing of AI tools. It’s more of a cultural shift toward infusing AI into the entire SDLC, not just coding, with an aligned push to drive adoption.

Brian: That aligns with some recent research I published showing that even something like leadership advocacy significantly increased the gains people saw from AI. You said something earlier that I think is really important and often misunderstood: software engineering is so much more than coding. Coding may be central to our identity as developers, but it’s only about 14% of our day.

Brian: One of the things I’ve been seeing in my research is that, yes, we’re producing 5, 7, 10, 15% more pull requests, but we’re doing it substantially more efficiently. For hands-on keyboard time spent coding, we’re producing about 40% more PRs per hour. That hints that we are reclaiming some coding time. Do you have any idea where that time is getting reinvested?

Abi: This has been a similarly perplexing question. For the past six months, I’ve heard customers asking: we’re seeing meaningful self-reported time savings from developers, but we’re not seeing that show up in our output and throughput. Where’s the disconnect? Where’s that time going?

We don’t know yet—this requires more research. But one preliminary hypothesis is that if developers only spend 14% of their time coding, then it makes sense that the time they’re recouping is ratably distributed across their activities. Only 14% of the time savings go back into coding, which would explain why we don’t see one-to-one output gains mapping to time savings.

Another hypothesis ties back to why the gains aren’t higher: some of the time savings come with side effects that require more time, like overseeing the work, QAing it, reviewing it. There’s also the lag factor: what do developers do while they’re waiting for agents to produce the code?

So those are some of the hypotheses, but we don’t have a clear answer yet.

Brian: There’s a reason we don’t just measure lines of code as a good measure of productivity. Writing more code isn’t always the right answer. As we increase velocity, what are some of the potential unwanted side effects you’ve been seeing?

Abi: Quality and cost are the two I hear customers talking about constantly. Cost is something we’re all trying to wrangle or at least get visibility into. Quality is something we all understand is an underlying risk, but I think it’s still early days. We did see Amazon go very public with this, but there’s some delay between the technical debt we may be creating and the consequences that follow.

The biggest thing I’ve been talking to customers about is a cultural risk I’ve been calling false velocity. I see this in so many organizations right now. And to some extent, even developer productivity leaders can fall into this trap, being so focused on showing off how much faster and more prolific we are with AI, without asking whether it’s leading to meaningful improvement.

You probably have engineers in your organization showing off really impressive things they can do with Claude or whatever tool. But are we asking: is your team’s product velocity actually increasing? Leaders are talking about how many more PRs and lines of code they’re generating, but is their roadmap actually accelerating? Is the quality of their products and code sustainable? There’s a real risk of being so focused on demonstrating what we can do that we’re not paying attention to whether we’re actually getting better—whether we’re materially improving our businesses.

Brian: So what’s your recommendation to engineering leaders who want to apply AI beyond just code generation? Where should they be looking?

That’s the biggest question I’m hearing from leaders right now. A lot of folks are at a point where they’ve rolled out one or more popular coding tools. Adoption is in a fairly satisfactory place, and they’re asking: what next? Especially if you’re sitting at that 10% number and asking how to get further.

The things I’m hearing about and seeing are, first, looking left and right of code. We’ve accelerated coding to some extent, but if that’s only 14% of where our time and money goes, what about the other 86%? How do we accelerate and optimize that?

Second, beyond thinking about the SDLC, there’s the question of how we improve the developer experience more deeply. Things like deep work and documentation—the friction points developers experience. How can we leverage AI to materially improve those?

And the third theme is this idea—there are many names for it—autonomous engineering, async engineering, background agents. I think today most of us are focused on leveraging AI to accelerate human work. Humans are still in the cockpit, steering and monitoring the work of the agents. The question is: how do we complement that acceleration with augmentation? Meaning more autonomous agents that are truly augmenting your human workforce and working in parallel, not just underneath your developers. Those are the three big themes.

Brian: You and I are both huge fans of Dr. Margaret-Anne Storey’s work. She’s been talking a lot about cognitive debt and the human cost of these AI transformations. What have you been looking at in that area?

Abi: I’m a big fan of that research. I think it falls under what I was talking about earlier in terms of risks and the time delay; we’re not always thinking about what the consequences might be 12 months from now if we adopt certain ways of working.

The loss of human understanding of the systems we’re building is a really interesting risk. There are different takes on how material it is. Some people argue it doesn’t matter if humans don’t deeply understand the systems, because they can use AI to quickly regain that understanding when needed. Others argue: no, if you need to call the mechanic, they should have mastery over the machine. I don’t think we know yet, but it’s a valid question.

Brian: We’ve hinted at it throughout this conversation—are we actually delivering innovation faster, or are we just moving bottlenecks around? My first reaction to an unanswered question is always: how do we measure it? Any thoughts on how leaders should be thinking about measurement as AI transforms how we work?

Abi: Our approach to measurement—some things stay the same and some things need to evolve. How we think about the overall software organization, things like quality and velocity, I think those stay the same. And especially if we’re trying to understand how things are changing with AI, you need consistent measures pre, post, and during this transformation.

But there are also new tools, new ways of working, new workflows being born every week. Measuring these new ways of working requires new approaches.

I’ll touch on two things. One is that I think increasingly there’s a need to separate how you’re measuring and framing AI’s impact. You should be thinking about acceleration—how much faster are our humans—as one bucket. And augmentation—how much more capacity are we generating with agents—as a second bucket. Thinking of those as two separate components in your overall formula is useful.

The question of how to measure agents is especially interesting. We’ve been exploring this idea of agent hourly rate: the human-equivalent hours an agent delivers, divided by its cost. That gives you a return-on-investment figure you can compare to human hourly rate, which is an interesting comparison point.

Something really new—not even in our minds last September—is this idea of agent experience. In the same way DX was founded on the idea of measuring developer effectiveness by going to developers and getting signal from them, we’ve applied a similar approach to agents. How do you measure AI agent effectiveness? You go to the agents and get feedback from them on where their bottlenecks and constraints are. We’ve just rolled that out. We’re surveying agents, which is pretty wild.

Brian: That is wild. To wrap up our conversation, give me one quick finding. What’s one thing you’ve learned about making agents more effective?

Abi: You have to ask them, just like we ask our developers. It’s really early days, but we’ve initially begun measuring four factors. For example, we ask the agents: how were the requirements you were given? How easily were you able to understand the codebase you were working in? How well were you steered by the human you were pairing with? We get quantitative metrics from that as well as qualitative feedback—the agent explains where it had challenges or bottlenecks. That’s the idea.

Brian: I love it—the same principles of measurement apply. That is unfortunately all the time we have, but I think we covered a lot of ground.

Abi: A special thank you to Brian, and everyone who attended DX Annual. Stay tuned for the session recordings, coming soon.

This week’s featured DevProd job openings. See more open roles here.

• Cashea is hiring an Infrastructure & Developer Productivity Platform Engineering Manager | Remote

• BNY is hiring an SVP, Application Development Manager | Pittsburgh, PA

• Figma is hiring a Staff Software Engineer, Developer Experience | Remote; US

• Leidos is hiring a Platform Engineer | Remote; US

• Weave is hiring a Senior Platform Engineer, Data Infrastructure | Remote; US

That’s it for this week. Thanks for reading.

Share

Subscribe now

Last week we held our first-ever DX Annual event in San Francisco, bringing together nearly 500 engineering and platform leaders for a full day focused on developer productivity and AI. Thank you to everyone who joined us. Session recordings will be available in the coming weeks.

Abi: This week we have a guest post from Dr. Margaret-Anne Storey, a Professor of Computer Science and a Canada Research Chair in Human and Social Aspects of Software Engineering. Margaret-Anne is one of the most widely published researchers on developer productivity, having co-authored the SPACE and DevEx frameworks amongst many other works.

Margaret-Anne: Earlier this year I published two posts exploring how generative and agentic AI may be quietly shifting where the most significant risks in software development lie, away from technical debt and code quality, and toward something harder to see and measure: the erosion of shared understanding across teams. This is what I refer to as cognitive debt. The response to these posts surprised me, as practitioners confirmed that cognitive debt was a significant challenge they were facing. They also proposed concrete suggestions for recognizing and mitigating cognitive debt. I’m combining both posts below, with light edits, as they tell a connected story.

For readers who want to go deeper, two papers extend these ideas. In Theory of Troubleshooting, co-authored with Arty Starr, we ground the cognitive debt concerns in cognitive science, showing how making sense of unexpected system behavior places considerable demands on working memory and attention, and how prolonged troubleshooting leads to cognitive fatigue with real implications for developer well-being. In From Technical Debt to Cognitive and Intent Debt I propose a Triple Debt Model that adds a third dimension to this framework: intent debt, the erosion of externalized rationale that both developers and AI agents need to work with to safely maintain and evolve a codebase.

How generative and agentic AI shift concern from technical debt to cognitive debt

The term technical debt is often used to refer to the accumulation of design or implementation choices that later make the software harder and more costly to understand, modify, or extend over time. Technical debt nicely captures that “human understanding” also matters, but the words “technical debt” conjure up the notion that the accrued debt is a property of the code and effort needs to be spent on removing that debt from code.

Cognitive debt, a term gaining traction recently, instead communicates the notion that the debt compounded from going fast lives in the brains of the developers and affects their lived experiences and abilities to “go fast” or to make changes. Even if AI agents produce code that could be easy to understand, the humans involved may have simply lost the plot and may not understand what the program is supposed to do, how their intentions were implemented, or how to possibly change it. Where cognitive load is what developers experience in the moment, cognitive debt is a project-level property, capturing how a team loses understanding over time.

Cognitive debt is likely a much bigger threat than technical debt, as AI and agents are adopted. Peter Naur reminded us some decades ago that a program is more than its source code. Rather a program is a theory that lives in the minds of the developer(s) capturing what the program does, how developer intentions are implemented, and how the program can be changed over time. Usually this theory is not just in the minds of one developer but fragments of this theory are distributed across the minds of many, if not thousands, of other developers.

I saw this dynamic play out vividly in an entrepreneurship course I taught recently. Student teams were building software products over the semester, moving quickly to ship features and meet milestones. But by weeks 7 or 8, one team hit a wall. They could no longer make even simple changes without breaking something unexpected. When I met with them, the team initially blamed technical debt: messy code, poor architecture, hurried implementations. But as we dug deeper, the real problem emerged: no one on the team could explain why certain design decisions had been made or how different parts of the system were supposed to work together. The code might have been messy, but the bigger issue was that the theory of the system, their shared understanding, had fragmented or disappeared entirely. They had accumulated cognitive debt across their team faster than technical debt, and it paralyzed them.

This dynamic echoes a classic lesson from Fred Brooks’ Mythical Man-Month. Adding more agents to a project may add more coordination overhead, invisible decisions, and thus cognitive load. Of course, agents can also be used to manage cognitive load by summarizing what changes have been made and how, but the core constraints of human memory and working capacity will be stretched with the push for speed at all costs. The reluctance to slow down and to do the work that Kent Beck calls “make the hard change easy” is what will lead to cognitive debt and cognitive load in the future.

In a breakout session at a recent Future of Software Engineering Retreat (arranged by Martin Fowler and Thoughtworks) we discussed how developers need to slow down and use practices such as pair programming, refactoring, and test-driven development to address technical debt AND cognitive debt. By slowing down and following these practices, cognitive debt can also be reduced and shared understanding across developers and teams rebuilt.

But what can teams do concretely as AI and agents become more prevalent? First, they may need to recognize that velocity without understanding is not sustainable. Teams should establish cognitive debt mitigation strategies. For example, they may wish to require that at least one human on the team fully understands each AI-generated change before it ships, document not just what changed but why, and create regular checkpoints where the team rebuilds shared understanding through code reviews, retrospectives, or knowledge-sharing sessions.

Second, we need better ways to detect cognitive debt before it becomes crippling. Warning signs include: team members hesitating to make changes for fear of unintended consequences, increased reliance on “tribal knowledge” held by just one or two people, or a growing sense that the system is becoming a black box. These may be signals that the shared theory is eroding.

Finally, this phenomenon demands serious research attention. How do we measure cognitive debt? What practices are most effective at preventing or reducing it in AI-augmented development environments? How does cognitive debt scale across distributed teams or open-source projects where the “theory” must be reconstructed by newcomers? As generative and agentic AI reshape how software is built, understanding and managing cognitive debt may be one of the most important challenges our field faces.

I explored these questions further in a recent keynote at the ICSE Technical Debt Conference and Panel. Cognitive debt tends not to announce itself through failing builds or subtle bugs after deployment, but rather shows up through a silent loss of shared theory. As generative and agentic AI accelerate development, protecting that shared theory of what the software does and how it can change may matter more for long-term software health than any single metric of speed or output.

Discussion: What I’m hearing about cognitive debt (so far)

After publishing the original post (above) on cognitive debt, it sparked thoughtful discussion across different communities. I’ve synthesized what I’m hearing below, and am connecting it to other reflections I’ve been reading.

There’s a growing concern about shared understanding

Several practitioners, including Simon Willison and others on a Hacker News discussion of a Martin Fowler article, describe experiencing cognitive debt directly. They talk about getting lost in their own projects and finding it harder to confidently add new features. They can move faster, but they lose the deeper sensemaking that connects decisions to intent, and intent to code.

This is not just about code quality. It is about whether individual developers and product teams can maintain a coherent mental model of what the system is doing and why.

Across these discussions, one theme is consistent: velocity can outpace understanding.

Cognitive debt hurts developers, not just the software

Technical debt lives in the code. Cognitive debt lives in people.

When shared understanding erodes, the pain shows up in:

• Loss of confidence when making changes

• Heavier review burden

• Debugging friction

• Slower onboarding

• Stress and fatigue

The software may be “working”, but the theory of the system becomes harder to access and keep track of. The cost is not only structural. It is experiential.

Siddhant Khare has written about AI fatigue. Steve Yegge reflects on burnout emerging from AI-accelerated development. Annie Vella eloquently writes about the emotional and cognitive experience of uncertainty when systems become harder to reason about. These perspectives reinforce that this is not just an engineering discipline issue, but one that affects how developers feel and function.

Cognitive debt, like technical debt, must be repaid

Martin Fowler notes that, like technical debt, cognitive debt must eventually be repaid. I agree.

But rebuilding lost knowledge requires restoring the distributed theory of the system. That includes capturing intent, the rationale behind decisions, key constraints, and how the architecture supports change. That theory is not stored in code alone. It is distributed across people, documentation, tests, conversations, tooling, and increasingly, AI agents.

Repayment means maintaining all of these, not just refactoring code or updating architecture documents. Under pressure to move quickly, whether in startups racing to learn or in large organizations pushing AI adoption, that repayment can feel expensive and easy to defer.

“This is just engineering”, but incentives are changing

Several commenters, including Michael Würsch, argue that cognitive debt reflects a failure of good engineering discipline. Clear specifications, rigorous reviews, extensive testing, and explicit architecture documentation should prevent knowledge loss.

In principle, I agree. But in practice, the incentives are shifting. AI lowers the cost of producing structure. It becomes easier for structure to evolve faster than shared understanding can stabilize. Even disciplined teams must consciously throttle or shape their practices to keep understanding aligned with change.

Specifications and documents are not sufficient if they are not living artifacts that teams actively engage with.

Emerging mitigation strategies

Encouragingly, many readers shared how they are mitigating cognitive debt.

They describe:

• More rigorous review practices

• Writing tests that capture intent

• Updating design documents continuously

• Treating prototypes as disposable

Some also describe using AI to reduce the cost of these practices, and even to support cognitive tracking, dependency management, and explanation.

Used deliberately, AI may help make cognitive work more visible rather than obscuring it.

The open question: How will high-performing teams adapt?

High-performing teams have always managed technical debt intentionally. As AI is adopted by startups and large companies, the question becomes how teams will manage cognitive debt.

How will they shape socio-technical practices and tools to externalize intent and sustain shared understanding? How will they use Generative and Agentic AI not only to accelerate code production, but to maintain their collective theory?

As AI reduces technical friction, shared understanding may become the bottleneck on performance.

I am continuing to watch how this evolves. If you are seeing mitigation practices that work in real teams, I would love to learn from them. As mentioned above, check out the article that goes deeper into how to recognize and mitigate cognitive debt and also proposes using the concept of intent debt to capture when decisions and the why behind a system are not captured for future humans and agents to refer to.

This week’s featured DevProd job openings. See more open roles here.

• Cashea is hiring an Infrastructure & Developer Productivity Platform Engineering Manager | Remote

• BNY is hiring an SVP, Application Development Manager | Pittsburgh, PA

• Figma is hiring a Staff Software Engineer, Developer Experience | Remote; US

• Leidos is hiring a Platform Engineer | Remote; US

• Reddit is hiring a Staff Software Engineer, Developer Experience | Remote; US

• Weave is hiring a Senior Platform Engineer, Data Infrastructure | Remote; US

That’s it for this week. Thanks for reading.

Share

Subscribe now

📅 Live research readout: Sign up here to join Abi for a readout and interactive discussion about the impact of AI on engineering velocity.

Since releasing our Q4 2025 AI Impact Report, reasoning models have improved, agentic workflows have gained traction, and teams are finding new ways to integrate AI into how they build.

Each quarter, we analyze the prior quarter’s data to benchmark AI adoption rates, productivity impact, and how teams are using these tools. For this edition, we expanded our dataset by over 40% more data points to give you a more representative picture.

We’ve compiled this analysis into our AI-Assisted Engineering: Q1 2026 Impact Report, which we are releasing today.

The full report covers industry adoption rates (now at 93%), tooling benchmarks, and the growing risks of “Shadow AI.” You can read the full report here. Below are two previews of interesting shifts we’ve observed since last quarter.

Engineering managers are shipping 4x more code than six months ago

Perhaps the most dramatic structural shift we observed is the changing nature of the engineering manager role.

In Q4, we noted that engineering managers who used AI daily were shipping twice as many pull requests over the period of time the sample was taken. That figure has doubled again: engineering managers are now shipping 4x as much code.

Why this matters: The definition of a developer is expanding. As agentic tools and AI assistants drastically lower the barrier to context-switching and boilerplate generation, managers are finding it far easier to get back into the codebase without sacrificing their leadership responsibilities. Coupled with an industry-wide trend of flattening organizational structures, AI is enabling the true return of the player-coach, allowing EMs to stay technically sharp and contribute directly to the product.

The question then is: if EMs can meaningfully contribute code again, should org structures evolve to reflect that? From what we’re hearing, it’s top of mind for many leaders, but most are still in “watch and learn” mode. No one we’ve spoken to has made structural changes based on this shift yet, but we’ll continue to monitor this trend.

Junior engineers now save more time with AI than Staff+ engineers

In our Q4 2025 report, Staff+ engineers who used AI daily were regaining more time than any other tenure band. Just three months later, the data tells a different story.

Junior engineers who use AI daily are now saving 4.9 hours per week, edging out daily Staff+ users who are saving 4.8 hours.

While time savings alone doesn’t tell a full story—in fact, this shift should come with heightened attention to quality, maintainability, and security—it is interesting to see this change in work impact among junior engineers.

Why this matters: Staff+ engineers, who naturally spend more time mentoring and reviewing rather than writing net-new code, are still seeing benefits, but junior engineers have officially taken the lead in total time saved. We provide a look at AI’s impact on quality in the full report; we’ll continue to pay close attention to quality measures amongst junior developer cohorts in the future.

What else is in the report?

The full Q1 2026 report provides comprehensive data and visual benchmarks on:

• Rust-specific improvements: How improvements in model reasoning and agentic loops have caused Rust to leapfrog other modern languages in AI-driven time savings.

• Quality volatility: Why some companies are seeing change failure rates swing by 2% (a 50% increase in defects), and why automated testing is no longer optional.

• The shadow AI risk: How developers are bypassing enterprise guardrails, and why acceptable use policies are critical for ensuring safe use of AI

• Small vs. large enterprises: Why companies with fewer than 200 developers are outpacing large enterprises in efficiency gains.

Final thoughts

As always, a reminder: there is no “average” experience with AI impact. Looking at industry trends and averages like those in this report can help contextualize your performance and do some pattern-matching, but I want to caution against using these averages as a guide to what your performance should look like. The truth is that AI impact is very asymmetrical and looks very different for each organization. Some are doing very well when it comes to throughput, but quality is suffering; some have done a great job of using AI for migrations, but are struggling to incorporate AI in testing and release processes. Averages abstract away the nuances of each company’s transformation journey. The best way to use this information is to enhance—not replace—your own metrics and AI strategy.

This week’s featured DevProd job openings. See more open roles here.

• Cashea is hiring an Infrastructure & Developer Productivity Platform Engineering Manager | Remote

• BNY is hiring an SVP, Application Development Manager | Pittsburgh, PA

• Figma is hiring a Staff Software Engineer, Developer Experience | Remote; US

• Leidos is hiring a Platform Engineer | Remote; US

• Reddit is hiring a Staff Software Engineer, Developer Experience | Remote; US

• Weave is hiring a Senior Platform Engineer, Data Infrastructure | Remote; US

That’s it for this week. Thanks for reading.

Share

In this episode, I’m joined by Eleanor Millman, Senior Staff Product Manager, and Mina Tawadrous, Associate Director of Platform Engineering at SiriusXM, to discuss how platform teams can scale prioritization without relying on revenue.

We talk through how SiriusXM moved beyond RICE to build a custom framework for internal platforms, using weighted factors like developer speed, reliability, cost, and trust to guide decisions across teams.

We also explore their concept of “assumptions as code,” in which teams store and reuse assumptions in a central repository to reduce misalignment and improve decision-making, with AI helping to surface and validate those assumptions.

We close with how this system is shaping SiriusXM’s 2026 prioritization approach and what it signals about a broader shift toward builder-driven product development.

Some takeaways:

Prioritization breaks without a shared system

• Prioritization does not scale naturally across teams. What works for one team breaks down at the org level with multiple stakeholders and competing requests

• Platform teams lack a clear revenue signal. Unlike product teams, they must prioritize based on indirect impact

• A shared framework aligns decisions. Without it, prioritization defaults to local optimization and noise

RICE is a starting point, not a solution

• Standard frameworks miss key dimensions for platform teams. Urgency and indirect impact are not captured well

• “Impact” needs to be decomposed. SiriusXM broke it into developer speed, reliability, cost, security, and more

• The framework must evolve over time. Iteration was critical to making it useful in practice

Weighting forces real tradeoffs

• You cannot prioritize everything at once. Increasing one dimension (like cost) necessarily deprioritizes others

• Assigning weights makes decisions explicit. Leaders must commit to what matters this quarter

• The output drives alignment across teams. A single prioritized list reduces cross-team conflicts

Data and conversation work together

• The framework creates a place to attach data. Metrics like reliability scores inform prioritization decisions

• Disagreements surface quickly. Teams can see where assumptions or inputs differ

• Conversations, not just scores, drive alignment. The value comes from debating inputs, not just ranking outputs

Assumptions are the real bottleneck

• Most disagreements come from hidden assumptions. Teams often believe they are aligned when they are not

• Assumptions can be conflicting, invisible, or stale. All three create friction in decision-making

• Making assumptions explicit improves clarity. It becomes easier to validate or challenge them

Storing assumptions as code scales learning

• Assumptions are stored in a central repository. User research and data become reusable across teams

• This reduces duplicated effort. Teams don’t need to rediscover the same insights repeatedly

• It creates a shared source of truth. Assumptions become visible, versioned, and easier to update

In this episode, we cover:

(00:00) Intro

(01:17) Mina’s role and path into platform engineering

(02:03) Eleanor’s background and shift into product

(03:15) Scaling prioritization across platform engineering teams

(05:41) Aligning platform priorities with stakeholders

(09:08) Evolving RICE into a platform-specific prioritization framework

(11:33) Iterating on the prioritization framework over time

(16:57) How the framework, data, and conversations drive alignment

(19:06) Storing assumptions as code in a central repository

(26:47) Resolving assumption conflicts with user interviews

(30:47) How stored assumptions integrate with AI workflows

(35:30) Standard mode and different user personas

(37:20) The industry shift towards builders

(41:04) The challenges of platform engineering

(43:36) How SiriusXM is prioritizing in 2026

Referenced:

• Measuring AI code assistants and agents

• SiriusXM

• VMware

• How SiriusXM revamped their platform and developer experience

• RICE Scoring Model | Prioritization Method Overview

• The evaporating cloud: A tool for resolving workplace conflict

Subscribe now

📅 Live research readout: Sign up here to join Abi for a readout and interactive discussion about an upcoming report that looks at the impact of AI on engineering velocity.

Onboarding new hires has always been an expensive and time-consuming process, and an area where AI has the opportunity to have a meaningful impact. In Q4 2025, when we looked at Time to 10th PR (a measure we use to track ramp-up time), we saw AI already having a dramatic effect. In some companies, Time to 10th PR was cut in half: from 91 days with no AI usage to 49 days with daily AI use.

We revisit this data quarterly. In our most recent analysis, we’re seeing that trend continue: onboarding time is faster today than it was in Q4 of last year.

For this latest cut, we analyzed data from a random sample of 400 companies during the period from October 2025 to February of 2026. We measured the average number of days between a developer’s start date and their 10th merged PR. We specifically looked at an aggregate of engineers who showed daily use of AI and compared how ramp-up time changed from October 2025 to February 2026.

The dataset includes large, global organizations with 500+ developers, spanning both tech and non‑tech companies.

As of April 2026, the current average Time to 10th PR across our dataset is 33 days, down from 39 days in Q4 2025. That represents a further ~15% decrease quarter-over-quarter, and more than a 50% reduction since Q1 2024, before AI usage was widespread. The change this quarter isn’t as dramatic as the 50% reduction we saw last year, but it is still meaningful. This trend begs the question: will onboarding times continue to fall, or are we approaching a natural floor where many of the remaining steps are not easily compressed by AI? As more agentic solutions are devised for onboarding processes, we may see this figure continue to decline.

Part of the improvement we’re seeing since last quarter comes from the fact that AI is no longer an optional, individual choice in many organizations. Developers are guided to use AI to ask questions about the codebase, to explain architectural decisions, and to propose draft changes that senior engineers can review. The result is that the “figuring things out” phase is shorter.

At the same time, Time to 10th PR is a narrow measure. It tells us how quickly someone reaches a specific milestone, but it does not, on its own, tell us about the quality of those changes, the amount of rework they generate, or the depth of understanding new hires have of the systems they’re touching. That will be something we explore more in future analyses.

Stay tuned for the full report coming soon.

This week’s featured DevProd job openings. See more open roles here.

• Awardco is hiring an Engineering Manager, Cloud Platform | San Francisco, CA

• BNY is hiring an SVP, Application Development Manager | Pittsburgh, PA

• Figma is hiring a Staff Software Engineer, Developer Experience | Remote; US

• Leidos is hiring a Platform Engineer | Remote; US

• Reddit is hiring a Staff Software Engineer, Developer Experience | Remote; US

• Weave is hiring a Senior Platform Engineer, Data Infrastructure | Remote; US

That’s it for this week. Thanks for reading.

Share

In this special episode of Engineering Enablement, I welcome back Jesse Adametz, this time as host.

In our conversation, we explore how AI is showing up across the SDLC, not just in code generation, and how it is shifting bottlenecks across the development process. We unpack what “AI readiness” actually means in practice, and why it often comes down to developer experience fundamentals like documentation, environments, and feedback loops.

We also discuss why enablement matters more than tool choice, how teams are thinking about measuring ROI, and what changes as background agents become more common. Finally, we explore how the role of the engineer may evolve, what questions teams are still trying to answer, and the challenges of non-engineers contributing to codebases.

Some takeaways:

AI is expanding beyond coding into the full SDLC

• The focus has shifted from code generation to the entire software lifecycle. Teams are applying AI to planning, prototyping, review, and documentation—not just writing code.

AI readiness is a developer experience problem

• The biggest blockers to AI adoption are long-standing DX gaps. Missing documentation, inconsistent environments, weak CI, and unclear system boundaries all limit effectiveness.

• Tool choice is not the primary driver of success. Models and tools are evolving too quickly for this to be a durable advantage.

• Some organizations are formalizing AI enablement as a function. Dedicated teams are emerging to drive adoption and share practices.

Measuring AI ROI is messy and still evolving

• Correlation vs causation makes attribution difficult. High AI usage often correlates with already high-performing engineers.

• Longitudinal analysis is more reliable than snapshots. Tracking changes over time gives better insight into impact.

• Token spend introduces real cost considerations. AI creates a direct, variable cost that organizations must evaluate.

AI impact falls into two buckets: amplification and augmentation

• Amplification improves human productivity. This includes higher throughput, time savings, and better developer experience.

• Augmentation extends capacity beyond humans. Agents begin to act as additional “headcount,” completing work independently.

• These require different measurement approaches. Amplification focuses on human output, while augmentation focuses on agent output relative to cost.

Background agents shift how work gets done and where bottlenecks appear

• Agents enable work to happen outside the human loop. Tasks can be completed asynchronously and proactively.

• This changes the developer role. Engineers move toward reviewing, guiding, and orchestrating agent output.

• Human workflows can become the bottleneck. If agents produce work faster than humans can process it, the constraint shifts.

• This reframes productivity. The question becomes where human involvement adds the most value.

Specs and documentation are becoming critical infrastructure

• AI makes documentation a core dependency. It directly impacts the quality of outputs.

• Poor documentation leads to poor results. Agents can duplicate systems or make incorrect assumptions without context.

• Documentation is shifting from optional to essential. It is now foundational for both human and AI productivity.

In this episode, we cover:

(00:00) Intro

(02:12) Where AI is showing up across the SDLC

(05:53) AI readiness and its link to developer experience

(08:23) Why enablement, education, and experimentation matter more than tool choice

(13:05) The case for a dedicated enablement team

(14:50) Measuring AI ROI: challenges and tradeoffs

(19:46) Background agents and token spend

(24:12) Measuring agent output with PR throughput

(26:58) How the engineer role might change

(31:01) Specs and documentation in the age of AI

(33:11) Non-engineers writing code

(35:30) What’s changing in the SDLC and open questions

Referenced:

• Measuring AI code assistants and agents

• Lessons from Twilio’s multi-year platform consolidation

• The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win

• How Claude remembers your project - Claude Code Docs

• specIsJustCode : r/ProgrammerHumor

Subscribe now

📅 Live research readout: Sign up here to join Abi for a readout and interactive discussion about an upcoming report that looks at the impact of AI on engineering velocity.

Each quarter, DX publishes data on how AI is being used at 500+ organizations and the impact it’s having. One metric we’ve been following is the percentage of code that’s written by AI—today’s newsletter shares a preview of what we’re seeing.

Currently, we measure the “percentage of AI-generated code” by asking developers directly how much of their merged code they believe is written by AI. This measure is captured quarterly by DX. In Q1 2026, developers at more than 500 organizations reported their average percentage of AI-authored code, along with how frequently they use AI tools (daily, weekly, or monthly). We aggregated these self-reported percentages to calculate the mean share of merged AI-generated code for each usage segment and overall.

We ran the previous Q4 analysis in the same way. In the future, we will collect this data and compare against a new system-based measure that automatically tracks the percentage of AI-generated code.

For this quarter, here’s a preview of what we’re seeing. Stay tuned for the full report later this month.

AI-generated code reaching production sees a slight increase

Since last quarter, the average share of merged code authored by AI has moved from 22% to 27.4%. While that’s directionally up, it’s not as meaningful of a change as we had expected.

The stability around 30% could be read two ways. On one hand, it may simply mean that current models and tooling are producing roughly the same proportion of merge-ready code as they were last quarter. However, there’s reason to believe something else is going on. The second half of 2025—particularly November and December—is widely regarded as a turning point for AI-assisted development. Models like Opus 4.5 represented a significant leap in capability, and some have gone so far as to say that anything before November 2025 shouldn’t even be used as a baseline, given how much changed in a short window.

If that’s true, then the more likely explanation for the stability around 30% is that most teams haven’t yet fully adapted to take advantage of those improvements. New models don’t automatically translate to more merged AI code; developers still need to update their workflows, build trust in the output, and find the right use cases for more capable tools.

There’s some evidence to support this interpretation. The daily users segment saw the largest increase in AI-generated code, moving from 24.1% to 30.8%. Meanwhile, weekly and monthly users, who are less likely to have adjusted their habits, saw smaller increases.

Final thoughts

This analysis focuses on how much merged code is AI-generated, but it doesn’t answer questions like whether a higher percentage of AI-generated code is associated with changes in quality. It also begs the question of whether there are certain types of organizations, or groups of developers, that are merging more vs. less AI-generated code.

We’ll explore these questions, and others like them, in the full Q1 AI Impact Report.

Additionally, in future reports we’ll compare this self-reported measure against a telemetry-based metric that tracks AI-generated code directly from version control systems. This will not only provide another lens on AI’s impact, but also give us a more granular view of how patterns are changing over time and where those changes are concentrated.

Stay tuned.

This week’s featured DevProd job openings. See more open roles here.

• Amplitude is hiring an Infrastructure Engineer | Lindon, UT

• Awardco is hiring an Engineering Manager, Cloud Platform | San Francisco, CA

• BNY is hiring an SVP, Application Development Manager | Pittsburgh, PA

• Figma is hiring a Staff Software Engineer, Developer Experience | Remote; US

• Leidos is hiring a Platform Engineer | Remote; US

• Lob is hiring a Staff Platform Engineer | Remote; US

• Weave is hiring a Senior Platform Engineer, Data Infrastructure | Remote; US

That’s it for this week. Thanks for reading.

Share

Subscribe now

Last week I hosted a live AMA about measuring the impact of AI and where organizations are with their rollout today. (Special thanks to Jesse Adametz for hosting the discussion.) One of the questions that came up had to do with measuring ROI: specifically, how should we think about correlation vs. causation? Leaders want to be careful about whether they should attribute changes in metrics to AI tools.

I answered the question live—you can download and watch the full recording here—or read this week’s newsletter for my take.

Many organizations make this mistake: They compare developers who use AI heavily to those who use it less, notice that the heavy users have higher throughput, and then conclude that AI must be the cause. The problem with this kind of analysis—“do people who use AI more have higher code throughput?”—is that in many cases, the developers who use AI more are the ones who were already coding more in the first place.

That’s why most of the time, longitudinal analysis (looking at how things change over time) is more informative. But it’s also harder to do. You need clean data over a long enough period, and you still have to account for confounding factors. For example, one big confound right now is the heightened pressure in many companies to increase throughput. Leaders are simultaneously pushing for more output and more AI usage. Because of this, some of the increase in throughput is likely due to this pressure itself—a classic case of Goodhart’s law, where once a metric becomes a target, it stops being a good measure.

When thinking about the ROI of AI more broadly, it’s useful to break it into two buckets:

• Amplification: How much more productive are humans thanks to AI? Here we look at:

  • Throughput (are engineers shipping more by using AI?)

  • Time saved (how much time do developers feel they’re saving in specific workflows?)

  • Developer experience scores (are AI tools improving the overall developer experience?) We can then convert improvements in developer experience into time savings, for example using something like a developer experience index.

• Augmentation: To what extent are you actually extending your engineering capacity by using agents, as if they were additional headcount? One unit we like to use here is human-equivalent hours: how much work are agents delivering, how much would that have taken a human, and then you divide the cost by those human-equivalent hours. Dividing the cost by the human-equivalent hours gives you an effective agent hourly rate (dollars per equivalent hour). If that rate is low, it indicates a high-ROI place to invest.

This amplification/augmentation framing is also helpful when talking to executives. You can say: to some extent we’re amplifying our humans, and to some extent we’re augmenting our workforce with agents.

Another common and related question that tends to come up in this discussion: if we agree that lines of code is a poor metric, what should we use instead to measure AI’s impact?

LOC is a noisy metric for a simple reason: A low-effort change can involve many lines of code, and a high-effort change can involve very few lines of code. That was already true before AI, and it becomes even worse with AI-generated code, which tends to produce more lines than a human might. This makes LOC even more inflated and noisy.

For these reasons—both pre- and post-AI—we’ve preferred metrics like PR throughput. It’s still imperfect, but it gives a more normalized view of “change throughput”: how many atomic changes are we pushing through the system? That makes it a less noisy high-level signal than raw LOC.

At DX, we’ve also developed a metric called TrueThroughput, which is a weighted version of PR throughput. It incorporates lines of code as one of several inputs to weight PRs, but LOC is not the sole or primary metric. Even so, all of these metrics are imperfect; you’re really choosing between different degrees of imperfection. In that space, LOC is significantly less useful than PR throughput as a primary signal. This lines up with broader industry experience as well. Many large tech companies that have studied this problem in depth have also converged on some form of change throughput as their preferred signal for tracking impact, rather than relying on raw lines of code.

To summarize my perspective on the correlation vs causation question: a simpler way to frame AI’s ROI is to look at how much it amplifies your existing developers, and how much it augments your organization with agent-driven capacity.

Download and rewatch the full AMA discussion here. My response to this specific question starts at 16:50.

This week’s featured DevProd job openings. See more open roles here.

• Amplitude is hiring an Engineering Manager, Cloud Platform | San Francisco, CA

• BNY is hiring an SVP, Application Development Manager | Pittsburgh, PA

• Figma is hiring a Staff Software Engineer, Developer Experience | Remote; US

• Lob is hiring a Staff Platform Engineer | Remote; US

• Mastercard is hiring a Vice President, Software Engineering | O’Fallon, MO; Boston, MA

• Plaid is hiring a Software Engineer - Platform | New York, NY

• Vercel is hiring a DX Engineer | Hybrid (Austin, New York City, San Francisco)

That’s it for this week. Thanks for reading.

Share

Subscribe now

Last week, we shared data showing that the current productivity gains from AI in engineering are more modest than many headlines imply. To better understand this result, we’ve been interviewing developers across a range of companies to hear what they’re experiencing, and to dig into the reasons behind these more modest gains.

This newsletter summarizes the key themes we’re hearing in those conversations so far, along with direct anonymous quotes from the interviews we performed. This work is ongoing, so if you have something to add, please share your perspective in the comments or on LinkedIn. We’d love to hear from you.

Why are the real‑world gains from AI more modest than many people expect?

Developers we spoke with pointed to several reasons. The most common: coding is a relatively small share of how engineers spend their time, so even meaningful acceleration there has limited impact on overall throughput. Beyond that: new bottlenecks are being created, and organizational and social friction hampers AI’s potential impact.

Here’s what we heard.

1. Coding isn’t the main bottleneck

The most common theme we heard: AI does accelerate developers’ ability to write code, but coding only represents a percentage of the work engineers actually do. Even if AI cut coding time in half, it would only affect a fraction of the total time spent delivering a feature.

• A lot of the slowdown isn’t from our tools, but from either other teams, services or processes.

• Many senior devs say: “writing code is the easy part”... The bottleneck is the human side: alignment, planning, scoping, code reviews, etc.

• AI can help developers write code fast, but writing faster code is not always more productivity. Most developers code only a small proportion of their time. There might be an expectation that this percentage is higher from those who write the headlines, but the reality is that most of the time developers spend is on navigating process and quality, change management, regulation, etc.

• The easy tasks are a little easier. The tedious tasks are a little less annoying. Larger/more complicated tasks — I may be able to shave off hours to a day… but that’s not happening every day, and that doesn’t necessarily mean I ship 3x more PRs. Just means a 4d task can take 3.

2. Partial SDLC automation creates new bottlenecks, so AI gains cap out

AI has sped up code generation, but that has created or exposed bottlenecks downstream, particularly in code review and validation. More code is being produced, but the processes for checking that code haven’t scaled with it, and developers report that the time saved writing is often consumed by the extra scrutiny that AI-generated code requires.

• AI can very significantly speed up initial engineering time, but often that saved time is spent on extended reviews, fact checking or issue remediation, resulting in net-zero productivity gain.

• The opportunities for LLMs to save large amounts of time on new projects is offset by time spent, sometimes ineffectively, attempting to use them on existing projects whose complexity and intricacies lead the LLM to make changes that are at minimum difficult to review even for experienced owners of the code.

• People do not have all parts of SDLC AI-enabled, wherever it does not exist becomes a bottleneck (planning, code review, verification). When people are trying to remove those bottlenecks, there are often lack of guardrails that creates misdirection and churned motions.

3. Social friction is a barrier to adoption

Polarization between pro- and anti-AI engineers, ambiguity about when and how AI should be used, and the absence of peer champions all slow the rate at which teams develop effective workflows with these tools.

• There’s misalignment amongst engineers – the camps are pretty polarized in terms of pro-AI and anti-AI. When anti-AI engineers have status and loud voices it is challenging for everyone to know how to interact with ai tooling.

• AI tooling isn’t discussed enough. Engineers are often unsure whether it’s high status or low status to be talking about using AI, so they just don’t. And people are reluctant to state their true views. If people say what they might really believe, they might seem crazy. In many organizations it is not socially acceptable to suggest that we re-evaluate changes to the code review process for example.

• Having at least one other teammate who is bought in to using AI tools is essential, being an isolated solo-adopter does not allow you to materialize the gains in a meaningful way. Software development is a team sport.

4. Tooling and skill gaps are limiting gains

These two factors are hard to separate: immature tools make the learning curve steeper, and developers who are early in that curve get less out of the tools. Both are limiting gains, though developers who shared this view generally see it as a temporary problem.

On the skill side, using AI effectively is its own discipline. On the tooling side, AI assistants don’t always slot cleanly into existing developer workflows. The integrations that would make them feel native to how teams already work aren’t there yet, and getting agents to operate autonomously on real infrastructure remains an unsolved problem for many.

On skill gaps:

• Someone just starting out with these tools is not as effective as someone who has incorporated them into their workflows for much longer. I have 1000+ hours of purely agentic coding experience and I have so much to learn. Many beginners may have just a few hours or tens of hours. Learning to agentically code is a skill.

• Most of the task isn’t the actual coding, it’s articulating a fuzzy problem in a clear, easily understandable way.

• This is more short term, but the pace of change in the tooling (e.g., the emergence of skills) means I’m spending a fair amount of time getting to know how to use the tools appropriately; which usually means going down various rabbit holes to discover how NOT to use them. That creates some (temporary?) friction that means I might fall back to my own skills when there’s production pressures.

Note: Leaders can distribute DX’s prompting guide and advanced prompting guide internally with developers as a way to help them build a foundational practice with AI.

On tool maturity:

• The tools aren’t mature enough to fit into existing systems cleanly, and most developers haven’t figured out how to use them well yet. Not because they’re bad engineers — the workflow is just genuinely new and nobody’s handed them a playbook.

• The difference between frontier models and models even a few months back can be significant.

• Most developers are used to working inside well-defined tools — your IDE, your terminal, your CI pipeline. Everything has a clear place and a clear trigger. AI assistants don’t work that way… So you end up with higher adoption among developers who naturally gravitate toward that kind of open-ended tooling, and much lower adoption among developers who just want something that fits cleanly into what they’re already doing.

5. Most AI tools lack important context

AI tools perform well on problems that are self-contained and well-documented. But most real engineering work isn’t like that. The context that matters—why a system was designed a certain way, what the implicit rules are, what business constraints apply—typically isn’t written down. It lives in people’s heads. Until that knowledge is made explicit and accessible, AI will keep hitting a ceiling on the kinds of problems it can reliably help with.

• The bigger issue I keep running into is that most codebases and systems aren’t set up for AI to actually help. Not an architecture problem — more that the knowledge of how things work lives in people’s heads. Why this service behaves this way, what the implicit contract between these two systems is, why that design decision got made three years ago — none of that is written down anywhere. An AI assistant can’t reason over a Slack thread from an archived channel or the mental model of the engineer who built it. Until that context gets surfaced into something concrete, you’re always going to hit a ceiling on what the tooling can do.

• AI doesn’t have the context I do on making those decisions, as LLMs are built of massive stats, not direct knowledge of the niche ends of what I’m accomplishing.

• I would (anecdotally) put forth that a lot of my work with AI is trying to get it setup to do the work with proper guardrails. It’s not safe to let run loose on our code-bases or production facing infrastructure as its not deterministic.

• AI models are generally good at understanding what we are trying to achieve, especially when they have access to the workspace or codebase. They can often provide useful solutions for specific use cases, particularly when the problem is related to foundational aspects of a technology or programming language. However, what they often lack is a deeper understanding of the business context and the various factors that influence business logic. That broader context is important to fully understand the bigger picture of a project. In some cases, this limitation can be mitigated by carefully crafting prompts, but when the business logic becomes complex or the prompts become too large for relatively simple tasks, people may step back and revert to more traditional approaches.

Other observations about the current state of AI tools

While speaking with developers, we heard a few other interesting themes about what they’re currently experiencing.

1. Documentation robustness is improving, but its quality is yet to be determined.

• The biggest quality gain I’m seeing is in improved documentation. Developers hate writing docs, so it will always be the last thing they tackle, or the first to get cut... if they tackle it at all. At least for me, I found our documentation getting much more robust thanks to AI being able to take that task off our plate. And it’s not just great for the developers to have access to these docs. Giving the AI agents access to it provides a quicker context boost to know how to tackle the next project.

• Using it to write code documentation leads to terribly incorrect information, an active detriment to future work, creating more work. This is really painful. It’s just not its forte.

2. AI is especially powerful in greenfield projects.

• Could you whip up an app from scratch 3x faster with AI? I would believe that. And if I had to just guess, that’s where I think a lot of the hype comes from. Someone says “look I created this thing from scratch in 1 day!” But, I think it is known that AI isn’t quite as proficient when working on a mature, large codebase...with legacy code and lots of nuance. Which is where most ICs are living in day to day, I think.

• While multi-tasking on my main efforts, I was able to build an entire new microservice from scratch to tests an idea and provide a proof of concept of a new paradigm (this has massive comprehension debt, I haven’t even looked at the code, but the service does what I expect it to).

• It has helped a lot with doing things like fast PoCs to check if something will work.

3. There’s concern that developers are offloading critical thinking.

• I would say I’m losing muscle memory on how to do things (e.g. terminal commands like checking git history for when a file changed, checking what version of a dependency is installed, etc.). I’ve grown dependent on the AI to remember the right commands for me.

• I think if someone blindly uses AI for everything, he/she slowly forgets day to day things.

• AI helps speed up some tasks but may result in folks offloading their thinking. Particularly amongst junior developers, the ‘improved initial speed’ can come at the expense of skill development. Overall speed improvement is negligible.

• I have also found myself having to explain to more junior engineers why what AI said about a problem isn’t the correct answer for us in a particular situation. AI empowers more junior engineers to work longer without asking engineering questions of the more senior folks. That is a sharp double-edged sword that I am still learning how to work with when mentoring people.

• [For me, AI’s] handiness is augmented by me knowing when it’s wrong due to my engineering experience.

Final thoughts

Two takeaways stand out to me from these conversations. First, the gains from AI code assistants are structurally limited by how much of the job is actually coding. Microsoft’s research puts coding at around 16% of a developer’s time. Even dramatic acceleration to code generation can only move the overall needle so much.

Second, it’s the human aspects of the software development lifecycle that are the biggest bottlenecks. Handoffs between teams, the skill required to effectively prompt and manage agents, and the review burden that AI-generated code creates—these are where time is going, and where the next wave of productivity gains may come from.

For leaders, this raises a couple of important questions. Which parts of the SDLC beyond coding are candidates for AI assistance? And are the human conditions for capturing AI’s upside in place?

The 10% figure is not the ceiling. But closing the gap will likely require focusing on the parts of the job that AI hasn’t touched yet.

If you have something to add, please share in the comments or on LinkedIn.

This week’s featured DevProd job openings. See more open roles here.

• Amplitude is hiring an Engineering Manager, Cloud Platform | San Francisco, CA

• BNY is hiring an SVP, Application Development Manager | Pittsburgh, PA

• Figma is hiring a Staff Software Engineer, Developer Experience | Remote; US

• Lob is hiring a Staff Platform Engineer | Remote; US

• Plaid is hiring a Software Engineer - Platform | New York, NY

• Vercel is hiring a DX Engineer | Hybrid (Austin, New York City, San Francisco)

• Zillow is hiring a Senior Product Manager, Developer Experience | Remote; US

That’s it for this week. Thanks for reading.

Share

Subscribe now

🗓 Join Abi on March 19th for a live Q&A session. He will address some of the more pressing questions we’ve received around measuring AI impact, the impact of tool choice, and more. Register here.

Edit: We've updated the data in this post to reflect the latest results from our longitudinal study.

Social media and vendor marketing have set high expectations for AI, suggesting as much as 2-3x productivity gains. But from the data we’re seeing, the reality on the ground is far more modest.

At DX, we’re currently conducting a longitudinal study to measure the long-term impact of AI adoption on key engineering productivity metrics. As part of this study, we analyzed data from a random sample out of 400 companies between November 2024 through February 2026 to track whether teams are shipping more pull requests as AI adoption increases.

We found that, during this time, AI usage increased significantly—by an average 65%. However, PR throughput only increased by 7.76%.

Note: This figure is particularly robust because we’ve filtered out potential gamification effects by excluding teams that set PR throughput targets for individual engineers, which could drive metric inflation rather than genuine output.

What this means for leaders

A ~10% gain is consistent with what we’re hearing from engineering leaders more broadly: most organizations are landing in the 5–15% range. It is a real improvement, but it’s a long way from the 2–3x gains many executives and boards have come to expect. AI is moving the needle, but leaders may need to reset expectations internally.

Why gains aren’t higher

To understand what’s driving this, we spoke with developers across several of these organizations. The explanation we heard most consistently: writing code was never the bottleneck.

As one senior developer put it: “The easy tasks are a little easier. The tedious tasks are a little less annoying. A four-day task might take three. But that doesn’t mean I’m shipping 3x more PRs.”

AI may be accelerating the coding portion of the job. But coding represents a relatively small slice of how engineers actually spend their time. Planning, alignment, scoping, code review, and handoffs—the human parts of the SDLC—remain largely untouched.

What’s next

We’re continuing to investigate the long-term effects of AI in engineering teams. The full study will explore why some teams are capturing more of the upside than others, and what leaders can do to close that gap. More to come.

This week’s featured DevProd job openings. See more open roles here.

• Amplitude is hiring an Engineering Manager, Cloud Platform | San Francisco, CA

• Figma is hiring a Staff Software Engineer, Developer Experience | Remote; US

• Plaid is hiring a Software Engineer - Platform | New York, NY

• UserTesting is hiring a Software Engineer, Developer Experience (Platform) | Spain

• Vercel is hiring a DX Engineer | Hybrid (Austin, New York City, San Francisco)

That’s it for this week. Thanks for reading.

Share

Subscribe now

🗓 Next month, you can join me for a live Q&A session. I’ll address some of the more pressing questions we’ve received around measuring AI impact, the impact of tool choice, and more. Register here.

This week on the Engineering Enablement podcast, we were joined by Uma Namasivayam, Senior Director of Engineering Productivity at Dropbox. Uma’s team owns engineering productivity across Dropbox’s roughly 1,000 engineers: everything from CI/CD systems and telemetry infrastructure to the company’s AI tooling rollout.

During the conversation, Uma shared specifics on how Dropbox drove AI adoption from one-third of engineers to three-quarters, why they chose to deploy multiple AI coding tools rather than starting by standardizing on one, and how they ended up building their own internal AI platform.

Below is a lightly edited excerpt from this conversation. You can listen to the full episode here.

A lot of engineering orgs treat developer productivity as an engineering problem. You have a different take on that.

Uma: I think of productivity at Dropbox—or anywhere, really—as a sociotechnical problem. There absolutely has to be a strong investment in the technology itself: improving the reliability of your systems, improving speed, and reducing friction in your tooling. But then there’s the element of collaboration, of culture, of working with leadership, with developers themselves, and with the people team.

A concrete example: one of the dimensions of developer productivity is deep work—can developers actually code uninterrupted? That’s not necessarily an engineering problem. Working with our chief people officer, we had to literally think about how to restructure meeting times, how to carve out focus blocks for employees. That required a completely different set of partners than fixing a slow CI pipeline. That’s why having a common language across all of those groups, and bringing them together around that language, was so important. You have to attack productivity from multiple different angles.

How did the rollout of AI coding tools intersect with the DevEx work that was already underway?

Uma: I think of them as two parallel work streams that have a lot of overlap in the middle. DevEx is about incremental, systematic friction reduction. It requires aligning with leadership, defining the problems clearly, and making steady progress. AI is about speed. It’s about getting the best tools into developers’ hands quickly, experimenting fast, and staying on the cutting edge.

Before AI can really deliver on its promise, the foundational systems—build and test, telemetry, production observability—have to be in a strong place. Developers need to trust that when they push code through an AI-assisted workflow, the quality guardrails are actually there. Without that trust, you can’t go really hard on AI.

How did you actually get adoption moving? What drove the initial uptake?

Uma: Early on, roughly one-third of our engineers were using AI tools organically, so people were just finding tools they liked on their own. That’s a decent starting point, but it’s not a strategy. The real inflection point came when our exec team made AI a clear company priority. Within about three months of that top-down signal, we got to around three-quarters of engineers using tools on a weekly basis.

With that said, top-down mandates only take you so far. After that initial push, we had to go deeper, starting by looking at what was actually blocking adoption in different parts of the engineering population, and addressing those pockets specifically. That’s where a product mindset comes in: understanding your customers’ actual pain points rather than assuming one solution works for everyone.

You offer developers multiple AI coding tools rather than standardizing on one. What’s the thinking there?

Uma: It comes back to treating this like a product problem. Different teams have genuinely different needs. Our mobile developers, for example, couldn’t use the tools that work well for other parts of the codebase. We had to find something specific to their use case. So we deliberately chose not to be a single-tool shop.

We also learned quickly that some tools that work at a smaller scale fall apart at Dropbox’s scale. We were piloting an AI code review tool and it just didn’t hold up. That pushed us toward building some of this in-house. We now have an internal platform that handles the backend complexity specific to Dropbox’s monorepo, and that other teams can build on. The build-versus-buy decision turns out to be really critical when you’re operating at this scale and at this speed.

One practical thing that also helped: we worked with our procurement and legal and security teams to dramatically reduce the time it takes to evaluate and approve new AI tools, getting that review process down to around three days. When the market is moving this fast, your ability to experiment is only as good as how quickly you can get tools in front of developers.

Once you’ve expanded beyond code completion, what does AI usage actually look like across the SDLC?

Uma: Code completion was the obvious starting point. Once we felt we’d gotten what we could from that, we started looking at the rest of the development lifecycle: code review, testing, and debugging. Every stage of the SDLC is on the table.

What we discovered is that a lot of the commercially available tools for these adjacent use cases didn’t hold up at our scale or for our specific codebase. That’s what led us to build in-house. One of our developers took it upon himself to figure out what kind of platform we could build, using Claude and Claude Code as the foundation, that would work within Dropbox’s environment and that others could build on top of. That platform now handles the backend complexity: deployment, monorepo scale, and testing guardrails. If a team wants to build an AI-assisted code review product, they start from that platform rather than from scratch. It’s one of the things I’m most proud of from the past year.

What’s the hardest unsolved problem you’re carrying into 2026?

Uma: Connecting developer productivity improvements to actual business outcomes. We can show that DXI improved, that AI adoption is up, and that developers are saving hours. But the arc from “developers are more productive” to “we shipped more value to customers faster”—that instrumentation isn’t there yet, and I don’t think anyone in the industry has fully cracked it.

What we’re seeing is that the capacity unlocked by AI is naturally flowing toward migrations and tech debt reduction. That’s actually pretty cool—give engineers more capacity, and they automatically invest it in the right things. But as a leader, I need to be able to answer the CFO and the exec team: where is this capacity going, and how does it connect to revenue? We’re working toward that. If anyone listening has cracked that code, I’d genuinely love to talk.

You can listen to the full conversation with Uma on the Engineering Enablement podcast.

This week’s featured DevProd job openings. See more open roles here.

• American Express is hiring a Sr. Manager, Digital Product Management - DevProd | Hybrid - London UK

• CoreWeave is hiring a Sr. Software Engineer - Developer Experience | Livingston NJ; New York, NY

• DoorDash is hiring an Engineering Manager - Developer Experience | San Francisco, CA; Sunnyvale, CA; Seattle, WA; Los Angeles, CA; New York, New York

• Figma is hiring a Staff Software Engineer, Developer Experience | Remote; US

• Plaid is hiring a Software Engineer - Platform | New York, NY

• UserTesting is hiring a Software Engineer, Developer Experience (Platform) | Spain

That’s it for this week. Thanks for reading.

Share

Subscribe now

🗓 Next month, you can join us for a live Q&A session with DX CEO Abi Noda. He’ll address recent questions around measuring AI impact, the impact of tool choice, and more. Register to join here.

In just the past year, AI has set into motion several major shifts in how engineering organizations are structured, including how teams are composed, where responsibilities sit, and what skills matter. These changes are still unfolding, but patterns are emerging.

To help name some of those shifts, I interviewed two analysts who spend their time talking with engineering leaders across industries: Ruchi Goyal, Global AI Practice Leader at Accenture, and Dr. Kate Holterhoff, Senior Industry Analyst at RedMonk. Below I’ve summarized the observations they shared—with the hope that naming these patterns is a useful reference point for leaders to recognize and navigate the same changes in their own organizations.

Hiring priorities are emphasizing AI fluency and code quality

The software industry is still working through a hiring slump, but the vacuum is being filled by specific AI-centric roles.

• The “AI Engineer” trend: We are seeing a surge in “AI Engineer” titles on LinkedIn. These are roles that are less about training foundational models and more about building the glue between LLMs and production code. (See example roles on Indeed here).

• Quality in focus: At the same time, a new skill is becoming central to engineering hiring: the ability to distinguish good code from mediocre AI-generated code. As Ryan Dahl and others have noted, AI removes the barrier to writing code, but it introduces a new one: dealing with slop.

• Because a focus on quality has become more important, some organizations are seeing friction emerge in the code review process.

Fragmented AI experiments are consolidating into centralized platforms

Early AI adoption in most organizations was scattered, with individual teams running their own experiments with different tools and no shared standards. That’s changing. Enterprises are moving toward centralized models such as a formal AI Center of Excellence or a hub-and-spoke structure that sets organization-wide direction. Goyal notes that “a lot of clients are looking into merging Developer Productivity and Internal Tools teams into one.” This can enable a number of new strategies for the organization:

• Consolidation: Organizations are folding AI initiatives into their internal developer platforms, creating governed paths for how AI gets used across the organization. Without that structure, teams may end up making independent decisions about tools and access, which can be hard to unwind later.

• Orchestration: Managing individual agents is becoming less of the challenge, and coordinating multiple agents working together is where the complexity now lives. Organizations that have already worked through DevOps maturity will recognize the pattern: the evolution tends to move from automation, to orchestration, to choreography. AI adoption is following a similar arc.

• AI is spreading across engineering organizations to the point where most developers are expected to use it as a matter of course. As Holterhoff put it, “AI is becoming water.” As that happens, Platform teams are becoming increasingly responsible for the governance and prompt hardening that makes this safe.

A new operational layer is emerging: LLMOps

As AI becomes part of the engineering infrastructure, someone has to own the operational layer that keeps it running reliably. In many organizations, that responsibility is coalescing into what’s being called LLMOps. Many Platform and DevProd teams are absorbing this responsibility.

• Prompt engineering to guideposts: Ad hoc prompting works for individuals but doesn’t scale across a team or organization. Turning prompts into reusable templates means you’re building institutional knowledge rather than leaving every developer to figure it out independently, and it creates a consistent quality baseline across the SDLC.

• The SWAT Team approach: In many orgs, a specialized team of engineers handles model integration and AI infra, often leveraging the heavy lifting provided by big cloud providers. Organizations that stand up a dedicated group for this work move faster and make fewer costly mistakes, while the rest of the engineering org can focus on building.

• Deployment convergence: We are seeing tighter integration between development environments and production. (For example, Expo’s integration with Replit allows developers to deploy directly to production, bypassing traditional friction points.) This creates real efficiency gains, but it also means the guardrails that used to exist between those environments need to be rethought, which is something Platform teams are becoming increasingly responsible for.

The human factor still matters the most

Despite the automation, the primary challenges are still cultural. As RedMonk has famously noted, “The developers are everything.” AI tools are being designed to support practitioners, not replace them.

• Murky ROI: Individual developers see the value of AI tools clearly enough that many are willing to pay for them personally. But at the organizational level, the picture is murkier. Leaders are measuring the impact of AI while reading headlines claiming 3x productivity gains… and most aren’t seeing numbers that match.

• Security as a skill: A major skill gap isn’t learning to prompt; it’s learning how to use AI securely. Training now focuses heavily on privacy, vetting specific LLMs, and ensuring that agentic workflows don’t bypass critical security guardrails.

• Software engineers as self-optimizers: Developers are naturally inclined to try new tools. That’s generally a good thing, but it means organizations need vetted, safe environments for that experimentation to happen in. The risk is that developers could adopt AI in ways the organization can’t see or govern.

These changes are moving quickly, and most organizations are still early in figuring out what they mean in practice. There’s no single right answer yet, which is part of what makes it valuable to hear from analysts with visibility across many organizations. A special thanks to Ruchi Goyal and Dr. Kate Holterhoff for sharing their perspective for this newsletter.

This week’s featured DevProd job openings. See more open roles here.

• American Express is hiring a Sr. Manager, Digital Product Management - DevProd | Hybrid - London UK

• CoreWeave is hiring a Sr. Software Engineer - Developer Experience | Livingston NJ; New York, NY

• DoorDash is hiring an Engineering Manager - Developer Experience | San Francisco, CA; Sunnyvale, CA; Seattle, WA; Los Angeles, CA; New York, New York

• Figma is hiring a Staff Software Engineer, Developer Experience | Remote; US

• Plaid is hiring a Software Engineer - Platform | New York, NY

• UserTesting is hiring a Software Engineer, Developer Experience (Platform) | Spain

That’s it for this week. Thanks for reading.

Share

Developer productivity is often treated as a tooling problem or a sentiment problem. In reality, it’s neither. It’s a socio-technical systems problem that spans engineering foundations, leadership alignment, organizational design, and culture.

In this episode, I’m joined by Uma Namasivayam, Senior Director, Engineering Productivity at Dropbox, to explore how Dropbox approaches developer experience at scale. We talk about why productivity needs to be framed as a business problem, how executive alignment creates the conditions for meaningful change, and what it takes to treat developer experience as a real product with developers as customers.

We also dig into Dropbox’s approach to AI adoption. Uma shares why strong foundations, such as build, test, and observability, are prerequisites for AI to actually accelerate work, how Dropbox encourages daily AI use without mandating a single tool, and where build-versus-buy decisions break down at scale.

We close with an honest look at what remains unsolved: how to connect gains in developer productivity and AI-driven capacity to real business outcomes, and where engineering leaders should focus next in 2026.

Some takeaways:

Developer productivity is a socio-technical problem

• Productivity cannot be solved through tooling alone; it spans engineering systems, leadership behavior, organizational structure, and people practices.

• Problems like build and test are engineering problems, while problems like focus time and interruptions are people problems, and both matter equally.

• Treating productivity as a system forces tradeoffs to be explicit, rather than hidden inside isolated tooling initiatives.

Executive alignment matters more than any single metric

• Top-down sponsorship creates permission to act, especially when productivity work cuts across org boundaries.

• A shared framework creates alignment, not answers; its value is giving leaders and engineers a common language.

• System metrics matter more than single metrics, because productivity improvements rarely move one dimension in isolation.

• Distributed accountability makes productivity a company problem, not a developer experience team problem.

Developer experience works best when treated as a product discipline

• Developers are customers, and their experience must be understood through both qualitative feedback and quantitative signals.

• Good system metrics do not guarantee good developer experience, which is why sentiment and perception matter.

• DX surveys surface where systems break differently for different teams, such as desktop, mobile, and web developers.

• Continuous feedback loops are essential, combining surveys, direct conversations, and usage data.

• Internal communication is part of the product, reinforcing to developers that their feedback leads to real change.

Prioritization requires structure, not intuition

• Finite capacity makes prioritization unavoidable, even in large, well-resourced engineering orgs.

• Segmenting developer populations clarifies tradeoffs, since different teams experience different bottlenecks.

• DX survey data provides a defensible starting point, but prioritization still requires judgment.

• Leadership-level stack ranking helps resolve conflicts, especially when multiple teams compete for attention.

• Frameworks make hard decisions easier to explain, even when they do not make them easy.

AI and developer experience must advance in parallel

• AI accelerates work, while developer experience reduces friction, and both are required for sustained gains.

• Foundational systems act as plumbing, enabling trust in speed, quality, and safety.

• Without strong CI, testing, and observability, faster code creation increases risk instead of value.

• Trust in guardrails enables confidence in AI-assisted development, especially at scale.

AI adoption succeeds through choice, not mandates

• Early organic adoption revealed real developer needs, rather than forcing a single tool.

• Different teams require different AI tools, particularly for mobile, desktop, and large-repo workflows.

• Supporting multiple tools increased adoption, rather than reducing it.

• Daily use depends on fitting AI into existing workflows, not adding extra steps.

• Habits matter more than access, which is why SDLC-level integration is critical.

Build vs. buy decisions change at scale

• Many AI tools fail when tested at large-company scale, despite working well in smaller contexts.

• Cost and performance become gating factors, not feature completeness.

• Internal platforms can abstract complexity, enabling teams to build AI workflows safely and consistently.

• Shared internal platforms unlock reuse, allowing teams to innovate without rebuilding infrastructure.

• Speed of iteration remains the primary differentiator, even when building in-house.

In this episode, we cover:

(00:00) Intro

(00:45) Dropbox’s engineering org

(01:59) Why developer productivity is a business problem

(04:08) The role of executive sponsorship in developer productivity

(06:02) How DX’s Core Four framework created a shared language

(08:13) Treating developer experience as a product

(11:30) How Dropbox prioritizes developer experience work

(14:20) The challenge of tying developer experience to business outcomes

(16:38) How AI and developer experience intersect at Dropbox

(18:35) The prerequisites for AI adoption to accelerate work

(20:26) How Dropbox encourages daily AI use

(23:12) AI use beyond code completion

(25:00) Managing AI tool demand at scale

(27:56) Early results from Dropbox’s AI efforts

(30:05) Progress on developer experience at Dropbox

(32:55) Advice for organizations investing in developer experience

(34:25) Capacity tradeoffs for developer experience

(35:59) The unanswered questions around AI and capacity in 2026

Referenced:

• DX Core 4 Productivity Framework

• Dropbox.com

Subscribe now

In 2025, we saw engineering leaders focus on rolling out AI coding assistants at scale across their organizations. As these tools became more widely used, it became clear that outcomes depended less on having access to AI and more on how teams were educated and enabled. In response, DX published the Guide to AI Assisted Engineering, outlining best practices and high-value prompting use cases to help engineering teams use AI effectively in their day-to-day work.

Now, as organizations move beyond pilots, the focus has shifted from adoption and enablement to operational improvements and more complex use cases. Successfully applying AI in these contexts requires more structured prompting practices than those used during early experimentation. To support that next step, we’ve created our first supplement to the original guide: Advanced Prompting Guide for AI Engineering.

This new guide follows the same format as the original, with clear Do and Don’t scenarios, full prompt examples, and code output examples. It is vendor-agnostic, with an emphasis on prompting structure, constraints, and context so the techniques can be applied across tools and architectures.

Inside, you’ll find prompt and code examples that focus on:

• Complexity management - For systems with cascading rules or conflicting requirements, the guide demonstrates graph-based prompting to reveal hidden dependencies, prioritize rules, and deal with changing state

• Governance and quality - Workflows that execute controlled validation loops, which result in higher accuracy, and can deal with more edge cases

• Risk mitigation - Dual-implementation strategies can yield more bulletproof outcomes, especially when dealing with critical transactions that require 100% accuracy

• Operational efficiency - Techniques like diff-only refactoring can reduce invasive changes to large, complex code repositories, as well as reduce tokens

These use cases are drawn from interviews, educational talks, and community interaction, and can be deployed across multiple scenarios. Whether you’re using coding assistants, building prompts for agents, or writing specs for spec-driven-development, you’ll find applicable methods in the guide.

One other important update: When the original guide was published, it was written primarily for developers. But 2025 was a pivotal year for elevating traditional non-builders. As highlighted in our Q4 AI Impact Report, engineering leaders are shipping more code, and designers and PMs have the ability to create deeper designs and prototypes. We still encourage engineering leaders to distribute this guide to their engineering teams, but this guide need not be exclusive to engineers. Whether engineer, designer, PM, or leader, if you are working on complex problems, this guide can provide useful perspectives.

Download the guide

This week’s featured DevProd job openings. See more open roles here.

• American Express is hiring a Sr. Manager, Digital Product Management - DevProd | Hybrid - London UK

• CoreWeave is hiring a Sr. Software Engineer - Developer Experience | Livingston NJ; New York, NY

• DoorDash is hiring an Engineering Manager - Developer Experience | San Francisco, CA; Sunnyvale, CA; Seattle, WA; Los Angeles, CA; New York, New York

• Experian is hiring a Software Engineering Manager - Security Platform | Remote

• Gusto is hiring a Sr. Platform Engineer | Denver, CO; San Francisco, CA; Atlanta, GA; Austin, TX; Chicago, IL; Miami, FL; Seattle, WA

• Notion is hiring a Senior Platform Engineer | Dubai, United Arab Emirates

• Plaid is hiring a Software Engineer - Platform | New York, NY

That’s it for this week. Thanks for reading.

Share

Subscribe now

🗓️ We recently announced DX Annual, our flagship conference for developer productivity leaders navigating the AI era. Go here to learn about the event and request an invite to attend.

One of the insights from the AI impact report released last quarter was that newer AI-native tools appeared to be outperforming others (see page 14). Even in side-by-side deployments, AI-native tools like agentic IDEs were associated with higher throughput compared to older or less specialized solutions.

Given how quickly the space is evolving, we decided to revisit this data to see whether those patterns still hold. In this latest analysis, we looked at PR throughput and median adoption rate across a slightly larger sample of companies (219, versus the 170 we examined last quarter). Here’s an updated look at what we’re seeing:

Comparing our latest Q1 2026 data to our Q4 2025 report we observe a performance jump consistently across all tools. This indicates that trends from last quarter’s data are continuing, and that stronger performance is being realized as tool maturity and adoption practice improves.

It’s notable that the throughput rankings themselves have not changed over the last three months, with newer, agentic tools still ranking highest in terms of velocity impact. Regardless of overall rank, we observed velocity increases across every tool, with a few major shifts since Q4 2025:

• Cursor has seen the largest increase in PR throughput. In Q4 2025, daily users merged a median of 2.8 PRs. Today, that number has jumped to 4.1, representing a 46% increase in throughput for frequent users.

• Claude leads for weekly and monthly users, with PR throughput exceeding 4.0. This is a significant move from the 2.6 (weekly) and 2.2 (monthly) reported just one quarter ago.

• GitHub Copilot daily users have also accelerated, moving from 2.5 PRs/week in Q4 to 3.61 today.

• Tabnine continues to show the lowest throughput at 1.83 for daily users. As noted in the previous report, this is likely because Tabnine is common in large enterprises where PR throughput is lower overall due to organizational complexity.

The last quarter of 2025 has been a landmark period for AI, with broader industry education and best practice converging with improved models, like Opus 4.5 and GPT-5.2, and better tool workflows such as those found in Cursor’s Agent Mode and Claude Code’s spec-driven focus.

Despite rapid advancement and innovation, many unsolved problems remain. Limitations such as memory and context constraints provide technical hurdles, while lack of organizational alignment on policies and practices continue to drive cultural challenges. With all this opportunity for improvement, I expect to see deliberate advancements continue, and productivity gains rise even further in the coming months.

While we were analyzing PR throughput data associated with different AI tools, we also looked at which tools are becoming daily drivers. (Note: adoption metrics don’t directly reveal productivity impact, as is mentioned in the DX AI Measurement Framework. But adoption metrics can give us a signal into which tools are being used for daily tasks versus those used only for specialized, less regular tasks.)

• GitHub Copilot remains the leader in overall stickiness, with the highest daily adoption rate at 9.76%. Its integration into the existing workflow may be helping it become a constant companion for developers. Additionally, for many businesses, Copilot is easy to procure since an existing purchasing agreement with Microsoft is likely to already be in place.

• Cursor is rapidly becoming a primary workspace, boasting an impressive 31.56% adoption rate among weekly users.

• Windsurf and Tabnine show a longer-tail adoption pattern. Windsurf, in particular, has the highest monthly adoption at 35.87%, suggesting it is being used as a powerful specialized tool for specific complex tasks rather than a total IDE replacement for most.

Summary

The data provides evidence for the belief that organizations will realize greater productivity gains as developers become more familiar with AI tools. Further, the significant jump we see in throughput shows how quickly the space is advancing. It’ll be fascinating to watch and see whether we see even higher throughput numbers next quarter, or gains will start to level over time.

I’ll conclude with a few reminders on how to use this data:

1. Correlate usage with throughput: Don’t just measure seat count. Ensure that your highest-cost tools are being used by your Daily or Weekly cohorts where the throughput gains are most visible.

2. Avoid vendor lock-in: Because the performance of these tools is evolving so quickly, we recommend a multi-vendor approach. Different tools excel at different frequencies (e.g., Copilot for daily assistance, Claude for weekly deep-dives). See our webinar on data-driven evaluations of AI tools for more information.

3. Measure to confirm ROI: Use these benchmarks to establish a baseline for your own organization. While AI metrics tell you what is happening, your core metrics, like PR throughput, confirm whether the investment is actually working.

This week’s featured DevProd job openings. See more open roles here.

• American Express is hiring a Sr. Manager, Digital Product Management - DevProd | Hybrid - London UK

• CoreWeave is hiring a Sr. Software Engineer - Developer Experience | Livingston NJ; New York, NY

• DoorDash is hiring an Engineering Manager - Developer Experience | San Francisco, CA; Sunnyvale, CA; Seattle, WA; Los Angeles, CA; New York, New York

• Experian is hiring a Software Engineering Manager - Security Platform | Remote

• Gusto is hiring a Sr. Platform Engineer | Denver, CO; San Francisco, CA; Atlanta, GA; Austin, TX; Chicago, IL; Miami, FL; Seattle, WA

• Notion is hiring a Senior Platform Engineer | Dubai, United Arab Emirates

• Plaid is hiring a Software Engineer - Platform | New York, NY

That’s it for this week. Thanks for reading.

Share

Subscribe now

🗓️ In case you missed it: we recently announced DX Annual, our flagship conference for developer productivity leaders navigating the AI era. Go here to learn more and request an invite to attend.

Business leaders understand the ability of centralized developer productivity and internal platform teams to promote better throughput and faster time to market for software features. But, how large should these teams be? What’s the best proportion of developer productivity specialists to other engineers within the company?

To find an answer, our team analyzed data from 39 companies with varying engineering team sizes across a number of industry verticals and looked for trends in organizational composition for teams explicitly dedicated to developer productivity and experience. We found that engineering leaders typically dedicate 2% to 6% of total headcount to centralized developer productivity. However, this ratio is not linear: as organizations scale beyond 1,000 engineers, the percentage of dedicated headcount steadily decreases due to tooling leverage and automation.

I’m excited to continue following these trends as AI efforts become more centralized and embedded in developer productivity teams, as we may see it lead to further hiring and expansion of responsibilities.

Defining the developer productivity umbrella

To ensure these benchmarks weren’t artificially inflated, we used a narrow definition of what counts as a team focused strictly on developer productivity. We only looked at teams that are unambiguously dedicated to internal platforms, developer productivity, and developer experience, and didn’t include teams focused on SRE or infrastructure, for example.

Specifically, we included the following teams in our analysis:

• Developer Experience and Productivity: Teams explicitly named “Developer Experience,” “DevEx,” “Developer Productivity,” “Developer Enablement,” or “Engineering Enablement”

• Engineering Effectiveness: Teams focused on “Engineering Productivity,” “Engineering Effectiveness,” or “Engineering Excellence”

• Internal Developer Platform Engineers: Groups building “Internal Developer Platforms”, “Internal Developer Portals,” “Developer Frameworks,” “Documentation Systems,” “Service Catalogs” or “Developer Tools”

• Build & Release Infrastructure: “CI Infrastructure,” “CI Platform,” “Build Infrastructure,” “Build Systems,” “Core Automation Platforms” focused on CI/CD, and “Release Engineering” teams

• Developer Education and Support: “Developer Education,” “Engineering Support Organization,” and developer onboarding teams

• Test Infrastructure: Teams dedicated to “Test Infrastructure” and “Test Frameworks” (not QA or product testing)

We excluded several categories that often get conflated with developer tooling, such as:

• General Cloud Infrastructure: Cloud operations, data center management, storage infrastructure, and network infrastructure teams

• Business Platform Teams: “Data Platform,” “ML Platform,” “Analytics Platform,” “Payment Platform,” “Content Platform,” and other product or customer-facing platforms

• Site Reliability Engineering (SRE): While SRE teams are critical, they focus on production reliability rather than developer tooling (except when explicitly named as developer-facing)

• Security Infrastructure: Security operations, compliance, and infrastructure security teams

• Product Infrastructure: Teams building infrastructure for customer-facing products rather than internal tools

• Lab Infrastructure: Physical and virtual infrastructure for product testing and validation

Most companies dedicate 4.7% of engineering headcount to developer productivity

In looking at our first sample of teams that have less than 1,000 engineers, we found that most teams dedicate between 2-6% of their overall engineering headcount to centralized developer productivity functions, with an average of 4.7% across the board. Some range past 8% on the high end, and others range below 2%.

For the average team then, per engineer, that means roughly one productivity engineer per seventeen developers at the high range, or one for every fifty engineers at the lower range. The distribution is shown in the chart below.

These numbers are not surprising, given the variance between company cultures, repository/build architectures, and general attitudes towards productivity. DORA’s State of DevOps reports suggest 10-20% investment between centralized teams and embedded productivity work – our study focuses on centralized, named roles. In general, we’ll find higher investment amongst engineering teams with more complicated deployments (microservices, polyglot stacks, monorepo), and lower investment amongst teams that focus mostly on CI/CD with more monolithic architectures.

When we looked at headcount ratios, we also analyzed team structure: even if two companies have 20 productivity engineers, they may organize them differently. Most organizations distribute their developer productivity team members across 2 to 6 distinct teams. However, some companies take a specialist model, spinning up as many as 15 specialized teams to tackle narrow problems like test optimization or internal portals, while others prefer a single, centralized developer experience or developer productivity unit that handles all enablement tasks under one roof.

Benchmarks by company size and industry vertical

While the 2–6% range is a benchmark, the data shows that investment doesn’t scale linearly. As engineering organizations grow beyond 1,000 people, the percentage of dedicated DevProd headcount starts to drop. This is likely due to diminishing returns in investment, as teams grow and scale, single developer productivity resources can take advantage of more automation, knowledge sharing, tooling, and teamwide collaboration to reduce the need for headcount.

We also saw clear differences in how different industries value DevProd investment:

• Technology (4.89%): These companies lead the pack, averaging about one productivity specialist for every 20 engineers. Because software is their primary profit center, shipping features faster directly impacts their bottom line.

• Fintech and Financial Services (4.36%): These organizations follow closely behind as they continue to modernize their infrastructure and prioritize engineering speed.

• Retail (3.8%) and Large Enterprise (3.32%): These sectors typically show lower ratios, though the latter (large enterprises) may have more to do with the diminishing returns noted in the trends of organizations with 1000+ engineers.

How to use these numbers

These numbers shouldn’t be used as a strict quota, but as a guide for your organizational strategy. The “right” team size is one that balances these industry averages with the specific bottlenecks your developers are facing.

If you are building a new “center of excellence,” use these ratios to set your initial headcount. To ensure you’re getting a real return on that investment, combine these benchmarks with a measurement framework like the Core 4 to track whether your new team is actually removing friction and improving the developer’s day.

This week’s featured DevProd job openings. See more open roles here.

• American Express is hiring a Sr. Manager, Digital Product Management - DevProd | Hybrid - London UK

• Capital One is hiring a Product Manager - Developer Experience | Plano TX; McLean VA; Richmond VA

• Gusto is hiring a Sr. Platform Engineer | Denver, CO; San Francisco, CA; Atlanta, GA; Austin, TX; Chicago, IL; Miami, FL; Seattle, WA

• Plaid is hiring a Software Engineer - Platform | New York, NY

• Reddit: Staff Software Engineer - Developer Experience | Remote - United States

That’s it for this week. Thanks for reading.

Share