In this episode, I’m joined by Eleanor Millman, Senior Staff Product Manager, and Mina Tawadrous, Associate Director of Platform Engineering at SiriusXM, to discuss how platform teams can scale prioritization without relying on revenue.

We talk through how SiriusXM moved beyond RICE to build a custom framework for internal platforms, using weighted factors like developer speed, reliability, cost, and trust to guide decisions across teams.

We also explore their concept of “assumptions as code,” in which teams store and reuse assumptions in a central repository to reduce misalignment and improve decision-making, with AI helping to surface and validate those assumptions.

We close with how this system is shaping SiriusXM’s 2026 prioritization approach and what it signals about a broader shift toward builder-driven product development.

Some takeaways:

Prioritization breaks without a shared system

• Prioritization does not scale naturally across teams. What works for one team breaks down at the org level with multiple stakeholders and competing requests

• Platform teams lack a clear revenue signal. Unlike product teams, they must prioritize based on indirect impact

• A shared framework aligns decisions. Without it, prioritization defaults to local optimization and noise

RICE is a starting point, not a solution

• Standard frameworks miss key dimensions for platform teams. Urgency and indirect impact are not captured well

• “Impact” needs to be decomposed. SiriusXM broke it into developer speed, reliability, cost, security, and more

• The framework must evolve over time. Iteration was critical to making it useful in practice

Weighting forces real tradeoffs

• You cannot prioritize everything at once. Increasing one dimension (like cost) necessarily deprioritizes others

• Assigning weights makes decisions explicit. Leaders must commit to what matters this quarter

• The output drives alignment across teams. A single prioritized list reduces cross-team conflicts

Data and conversation work together

• The framework creates a place to attach data. Metrics like reliability scores inform prioritization decisions

• Disagreements surface quickly. Teams can see where assumptions or inputs differ

• Conversations, not just scores, drive alignment. The value comes from debating inputs, not just ranking outputs

Assumptions are the real bottleneck

• Most disagreements come from hidden assumptions. Teams often believe they are aligned when they are not

• Assumptions can be conflicting, invisible, or stale. All three create friction in decision-making

• Making assumptions explicit improves clarity. It becomes easier to validate or challenge them

Storing assumptions as code scales learning

• Assumptions are stored in a central repository. User research and data become reusable across teams

• This reduces duplicated effort. Teams don’t need to rediscover the same insights repeatedly

• It creates a shared source of truth. Assumptions become visible, versioned, and easier to update

In this episode, we cover:

(00:00) Intro

(01:17) Mina’s role and path into platform engineering

(02:03) Eleanor’s background and shift into product

(03:15) Scaling prioritization across platform engineering teams

(05:41) Aligning platform priorities with stakeholders

(09:08) Evolving RICE into a platform-specific prioritization framework

(11:33) Iterating on the prioritization framework over time

(16:57) How the framework, data, and conversations drive alignment

(19:06) Storing assumptions as code in a central repository

(26:47) Resolving assumption conflicts with user interviews

(30:47) How stored assumptions integrate with AI workflows

(35:30) Standard mode and different user personas

(37:20) The industry shift towards builders

(41:04) The challenges of platform engineering

(43:36) How SiriusXM is prioritizing in 2026

Referenced:

• Measuring AI code assistants and agents

• SiriusXM

• VMware

• How SiriusXM revamped their platform and developer experience

• RICE Scoring Model | Prioritization Method Overview

• The evaporating cloud: A tool for resolving workplace conflict

Subscribe now

📅 Live research readout: Sign up here to join Abi for a readout and interactive discussion about an upcoming report that looks at the impact of AI on engineering velocity.

Onboarding new hires has always been an expensive and time-consuming process, and an area where AI has the opportunity to have a meaningful impact. In Q4 2025, when we looked at Time to 10th PR (a measure we use to track ramp-up time), we saw AI already having a dramatic effect. In some companies, Time to 10th PR was cut in half: from 91 days with no AI usage to 49 days with daily AI use.

We revisit this data quarterly. In our most recent analysis, we’re seeing that trend continue: onboarding time is faster today than it was in Q4 of last year.

For this latest cut, we analyzed data from a random sample of 400 companies during the period from October 2025 to February of 2026. We measured the average number of days between a developer’s start date and their 10th merged PR. We specifically looked at an aggregate of engineers who showed daily use of AI and compared how ramp-up time changed from October 2025 to February 2026.

The dataset includes large, global organizations with 500+ developers, spanning both tech and non‑tech companies.

As of April 2026, the current average Time to 10th PR across our dataset is 33 days, down from 39 days in Q4 2025. That represents a further ~15% decrease quarter-over-quarter, and more than a 50% reduction since Q1 2024, before AI usage was widespread. The change this quarter isn’t as dramatic as the 50% reduction we saw last year, but it is still meaningful. This trend begs the question: will onboarding times continue to fall, or are we approaching a natural floor where many of the remaining steps are not easily compressed by AI? As more agentic solutions are devised for onboarding processes, we may see this figure continue to decline.

Part of the improvement we’re seeing since last quarter comes from the fact that AI is no longer an optional, individual choice in many organizations. Developers are guided to use AI to ask questions about the codebase, to explain architectural decisions, and to propose draft changes that senior engineers can review. The result is that the “figuring things out” phase is shorter.

At the same time, Time to 10th PR is a narrow measure. It tells us how quickly someone reaches a specific milestone, but it does not, on its own, tell us about the quality of those changes, the amount of rework they generate, or the depth of understanding new hires have of the systems they’re touching. That will be something we explore more in future analyses.

Stay tuned for the full report coming soon.

This week’s featured DevProd job openings. See more open roles here.

• Awardco is hiring an Engineering Manager, Cloud Platform | San Francisco, CA

• BNY is hiring an SVP, Application Development Manager | Pittsburgh, PA

• Figma is hiring a Staff Software Engineer, Developer Experience | Remote; US

• Leidos is hiring a Platform Engineer | Remote; US

• Reddit is hiring a Staff Software Engineer, Developer Experience | Remote; US

• Weave is hiring a Senior Platform Engineer, Data Infrastructure | Remote; US

That’s it for this week. Thanks for reading.

Share

In this special episode of Engineering Enablement, I welcome back Jesse Adametz, this time as host.

In our conversation, we explore how AI is showing up across the SDLC, not just in code generation, and how it is shifting bottlenecks across the development process. We unpack what “AI readiness” actually means in practice, and why it often comes down to developer experience fundamentals like documentation, environments, and feedback loops.

We also discuss why enablement matters more than tool choice, how teams are thinking about measuring ROI, and what changes as background agents become more common. Finally, we explore how the role of the engineer may evolve, what questions teams are still trying to answer, and the challenges of non-engineers contributing to codebases.

Some takeaways:

AI is expanding beyond coding into the full SDLC

• The focus has shifted from code generation to the entire software lifecycle. Teams are applying AI to planning, prototyping, review, and documentation—not just writing code.

AI readiness is a developer experience problem

• The biggest blockers to AI adoption are long-standing DX gaps. Missing documentation, inconsistent environments, weak CI, and unclear system boundaries all limit effectiveness.

• Tool choice is not the primary driver of success. Models and tools are evolving too quickly for this to be a durable advantage.

• Some organizations are formalizing AI enablement as a function. Dedicated teams are emerging to drive adoption and share practices.

Measuring AI ROI is messy and still evolving

• Correlation vs causation makes attribution difficult. High AI usage often correlates with already high-performing engineers.

• Longitudinal analysis is more reliable than snapshots. Tracking changes over time gives better insight into impact.

• Token spend introduces real cost considerations. AI creates a direct, variable cost that organizations must evaluate.

AI impact falls into two buckets: amplification and augmentation

• Amplification improves human productivity. This includes higher throughput, time savings, and better developer experience.

• Augmentation extends capacity beyond humans. Agents begin to act as additional “headcount,” completing work independently.

• These require different measurement approaches. Amplification focuses on human output, while augmentation focuses on agent output relative to cost.

Background agents shift how work gets done and where bottlenecks appear

• Agents enable work to happen outside the human loop. Tasks can be completed asynchronously and proactively.

• This changes the developer role. Engineers move toward reviewing, guiding, and orchestrating agent output.

• Human workflows can become the bottleneck. If agents produce work faster than humans can process it, the constraint shifts.

• This reframes productivity. The question becomes where human involvement adds the most value.

Specs and documentation are becoming critical infrastructure

• AI makes documentation a core dependency. It directly impacts the quality of outputs.

• Poor documentation leads to poor results. Agents can duplicate systems or make incorrect assumptions without context.

• Documentation is shifting from optional to essential. It is now foundational for both human and AI productivity.

In this episode, we cover:

(00:00) Intro

(02:12) Where AI is showing up across the SDLC

(05:53) AI readiness and its link to developer experience

(08:23) Why enablement, education, and experimentation matter more than tool choice

(13:05) The case for a dedicated enablement team

(14:50) Measuring AI ROI: challenges and tradeoffs

(19:46) Background agents and token spend

(24:12) Measuring agent output with PR throughput

(26:58) How the engineer role might change

(31:01) Specs and documentation in the age of AI

(33:11) Non-engineers writing code

(35:30) What’s changing in the SDLC and open questions

Referenced:

• Measuring AI code assistants and agents

• Lessons from Twilio’s multi-year platform consolidation

• The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win

• How Claude remembers your project - Claude Code Docs

• specIsJustCode : r/ProgrammerHumor

Subscribe now

📅 Live research readout: Sign up here to join Abi for a readout and interactive discussion about an upcoming report that looks at the impact of AI on engineering velocity.

Each quarter, DX publishes data on how AI is being used at 500+ organizations and the impact it’s having. One metric we’ve been following is the percentage of code that’s written by AI—today’s newsletter shares a preview of what we’re seeing.

Currently, we measure the “percentage of AI-generated code” by asking developers directly how much of their merged code they believe is written by AI. This measure is captured quarterly by DX. In Q1 2026, developers at more than 500 organizations reported their average percentage of AI-authored code, along with how frequently they use AI tools (daily, weekly, or monthly). We aggregated these self-reported percentages to calculate the mean share of merged AI-generated code for each usage segment and overall.

We ran the previous Q4 analysis in the same way. In the future, we will collect this data and compare against a new system-based measure that automatically tracks the percentage of AI-generated code.

For this quarter, here’s a preview of what we’re seeing. Stay tuned for the full report later this month.

AI-generated code reaching production sees a slight increase

Since last quarter, the average share of merged code authored by AI has moved from 22% to 27.4%. While that’s directionally up, it’s not as meaningful of a change as we had expected.

The stability around 30% could be read two ways. On one hand, it may simply mean that current models and tooling are producing roughly the same proportion of merge-ready code as they were last quarter. However, there’s reason to believe something else is going on. The second half of 2025—particularly November and December—is widely regarded as a turning point for AI-assisted development. Models like Opus 4.5 represented a significant leap in capability, and some have gone so far as to say that anything before November 2025 shouldn’t even be used as a baseline, given how much changed in a short window.

If that’s true, then the more likely explanation for the stability around 30% is that most teams haven’t yet fully adapted to take advantage of those improvements. New models don’t automatically translate to more merged AI code; developers still need to update their workflows, build trust in the output, and find the right use cases for more capable tools.

There’s some evidence to support this interpretation. The daily users segment saw the largest increase in AI-generated code, moving from 24.1% to 30.8%. Meanwhile, weekly and monthly users, who are less likely to have adjusted their habits, saw smaller increases.

Final thoughts

This analysis focuses on how much merged code is AI-generated, but it doesn’t answer questions like whether a higher percentage of AI-generated code is associated with changes in quality. It also begs the question of whether there are certain types of organizations, or groups of developers, that are merging more vs. less AI-generated code.

We’ll explore these questions, and others like them, in the full Q1 AI Impact Report.

Additionally, in future reports we’ll compare this self-reported measure against a telemetry-based metric that tracks AI-generated code directly from version control systems. This will not only provide another lens on AI’s impact, but also give us a more granular view of how patterns are changing over time and where those changes are concentrated.

Stay tuned.

This week’s featured DevProd job openings. See more open roles here.

• Amplitude is hiring an Infrastructure Engineer | Lindon, UT

• Awardco is hiring an Engineering Manager, Cloud Platform | San Francisco, CA

• BNY is hiring an SVP, Application Development Manager | Pittsburgh, PA

• Figma is hiring a Staff Software Engineer, Developer Experience | Remote; US

• Leidos is hiring a Platform Engineer | Remote; US

• Lob is hiring a Staff Platform Engineer | Remote; US

• Weave is hiring a Senior Platform Engineer, Data Infrastructure | Remote; US

That’s it for this week. Thanks for reading.

Share

Subscribe now

Last week I hosted a live AMA about measuring the impact of AI and where organizations are with their rollout today. (Special thanks to Jesse Adametz for hosting the discussion.) One of the questions that came up had to do with measuring ROI: specifically, how should we think about correlation vs. causation? Leaders want to be careful about whether they should attribute changes in metrics to AI tools.

I answered the question live—you can download and watch the full recording here—or read this week’s newsletter for my take.

Many organizations make this mistake: They compare developers who use AI heavily to those who use it less, notice that the heavy users have higher throughput, and then conclude that AI must be the cause. The problem with this kind of analysis—“do people who use AI more have higher code throughput?”—is that in many cases, the developers who use AI more are the ones who were already coding more in the first place.

That’s why most of the time, longitudinal analysis (looking at how things change over time) is more informative. But it’s also harder to do. You need clean data over a long enough period, and you still have to account for confounding factors. For example, one big confound right now is the heightened pressure in many companies to increase throughput. Leaders are simultaneously pushing for more output and more AI usage. Because of this, some of the increase in throughput is likely due to this pressure itself—a classic case of Goodhart’s law, where once a metric becomes a target, it stops being a good measure.

When thinking about the ROI of AI more broadly, it’s useful to break it into two buckets:

• Amplification: How much more productive are humans thanks to AI? Here we look at:

  • Throughput (are engineers shipping more by using AI?)

  • Time saved (how much time do developers feel they’re saving in specific workflows?)

  • Developer experience scores (are AI tools improving the overall developer experience?) We can then convert improvements in developer experience into time savings, for example using something like a developer experience index.

• Augmentation: To what extent are you actually extending your engineering capacity by using agents, as if they were additional headcount? One unit we like to use here is human-equivalent hours: how much work are agents delivering, how much would that have taken a human, and then you divide that by how much it cost. Dividing the human-equivalent hours by the cost gives you an agent hourly rate. If that effective rate is low, it indicates a high-ROI place to invest.

This amplification/augmentation framing is also helpful when talking to executives. You can say: to some extent we’re amplifying our humans, and to some extent we’re augmenting our workforce with agents.

Another common and related question that tends to come up in this discussion: if we agree that lines of code is a poor metric, what should we use instead to measure AI’s impact?

LOC is a noisy metric for a simple reason: A low-effort change can involve many lines of code, and a high-effort change can involve very few lines of code. That was already true before AI, and it becomes even worse with AI-generated code, which tends to produce more lines than a human might. This makes LOC even more inflated and noisy.

For these reasons—both pre- and post-AI—we’ve preferred metrics like PR throughput. It’s still imperfect, but it gives a more normalized view of “change throughput”: how many atomic changes are we pushing through the system? That makes it a less noisy high-level signal than raw LOC.

At DX, we’ve also developed a metric called TrueThroughput, which is a weighted version of PR throughput. It incorporates lines of code as one of several inputs to weight PRs, but LOC is not the sole or primary metric. Even so, all of these metrics are imperfect; you’re really choosing between different degrees of imperfection. In that space, LOC is significantly less useful than PR throughput as a primary signal. This lines up with broader industry experience as well. Many large tech companies that have studied this problem in depth have also converged on some form of change throughput as their preferred signal for tracking impact, rather than relying on raw lines of code.

To summarize my perspective on the correlation vs causation question: a simpler way to frame AI’s ROI is to look at how much it amplifies your existing developers, and how much it augments your organization with agent-driven capacity.

Download and rewatch the full AMA discussion here. My response to this specific question starts at 16:50.

This week’s featured DevProd job openings. See more open roles here.

• Amplitude is hiring an Engineering Manager, Cloud Platform | San Francisco, CA

• BNY is hiring an SVP, Application Development Manager | Pittsburgh, PA

• Figma is hiring a Staff Software Engineer, Developer Experience | Remote; US

• Lob is hiring a Staff Platform Engineer | Remote; US

• Mastercard is hiring a Vice President, Software Engineering | O’Fallon, MO; Boston, MA

• Plaid is hiring a Software Engineer - Platform | New York, NY

• Vercel is hiring a DX Engineer | Hybrid (Austin, New York City, San Francisco)

That’s it for this week. Thanks for reading.

Share

Subscribe now

Last week, we shared data showing that the current productivity gains from AI in engineering are more modest than many headlines imply. To better understand this result, we’ve been interviewing developers across a range of companies to hear what they’re experiencing, and to dig into the reasons behind these more modest gains.

This newsletter summarizes the key themes we’re hearing in those conversations so far, along with direct anonymous quotes from the interviews we performed. This work is ongoing, so if you have something to add, please share your perspective in the comments or on LinkedIn. We’d love to hear from you.

Why are the real‑world gains from AI more modest than many people expect?

Developers we spoke with pointed to several reasons. The most common: coding is a relatively small share of how engineers spend their time, so even meaningful acceleration there has limited impact on overall throughput. Beyond that: new bottlenecks are being created, and organizational and social friction hampers AI’s potential impact.

Here’s what we heard.

1. Coding isn’t the main bottleneck

The most common theme we heard: AI does accelerate developers’ ability to write code, but coding only represents a percentage of the work engineers actually do. Even if AI cut coding time in half, it would only affect a fraction of the total time spent delivering a feature.

• A lot of the slowdown isn’t from our tools, but from either other teams, services or processes.

• Many senior devs say: “writing code is the easy part”... The bottleneck is the human side: alignment, planning, scoping, code reviews, etc.

• AI can help developers write code fast, but writing faster code is not always more productivity. Most developers code only a small proportion of their time. There might be an expectation that this percentage is higher from those who write the headlines, but the reality is that most of the time developers spend is on navigating process and quality, change management, regulation, etc.

• The easy tasks are a little easier. The tedious tasks are a little less annoying. Larger/more complicated tasks — I may be able to shave off hours to a day… but that’s not happening every day, and that doesn’t necessarily mean I ship 3x more PRs. Just means a 4d task can take 3.

2. Partial SDLC automation creates new bottlenecks, so AI gains cap out

AI has sped up code generation, but that has created or exposed bottlenecks downstream, particularly in code review and validation. More code is being produced, but the processes for checking that code haven’t scaled with it, and developers report that the time saved writing is often consumed by the extra scrutiny that AI-generated code requires.

• AI can very significantly speed up initial engineering time, but often that saved time is spent on extended reviews, fact checking or issue remediation, resulting in net-zero productivity gain.

• The opportunities for LLMs to save large amounts of time on new projects is offset by time spent, sometimes ineffectively, attempting to use them on existing projects whose complexity and intricacies lead the LLM to make changes that are at minimum difficult to review even for experienced owners of the code.

• People do not have all parts of SDLC AI-enabled, wherever it does not exist becomes a bottleneck (planning, code review, verification). When people are trying to remove those bottlenecks, there are often lack of guardrails that creates misdirection and churned motions.

3. Social friction is a barrier to adoption

Polarization between pro- and anti-AI engineers, ambiguity about when and how AI should be used, and the absence of peer champions all slow the rate at which teams develop effective workflows with these tools.

• There’s misalignment amongst engineers – the camps are pretty polarized in terms of pro-AI and anti-AI. When anti-AI engineers have status and loud voices it is challenging for everyone to know how to interact with ai tooling.

• AI tooling isn’t discussed enough. Engineers are often unsure whether it’s high status or low status to be talking about using AI, so they just don’t. And people are reluctant to state their true views. If people say what they might really believe, they might seem crazy. In many organizations it is not socially acceptable to suggest that we re-evaluate changes to the code review process for example.

• Having at least one other teammate who is bought in to using AI tools is essential, being an isolated solo-adopter does not allow you to materialize the gains in a meaningful way. Software development is a team sport.

4. Tooling and skill gaps are limiting gains

These two factors are hard to separate: immature tools make the learning curve steeper, and developers who are early in that curve get less out of the tools. Both are limiting gains, though developers who shared this view generally see it as a temporary problem.

On the skill side, using AI effectively is its own discipline. On the tooling side, AI assistants don’t always slot cleanly into existing developer workflows. The integrations that would make them feel native to how teams already work aren’t there yet, and getting agents to operate autonomously on real infrastructure remains an unsolved problem for many.

On skill gaps:

• Someone just starting out with these tools is not as effective as someone who has incorporated them into their workflows for much longer. I have 1000+ hours of purely agentic coding experience and I have so much to learn. Many beginners may have just a few hours or tens of hours. Learning to agentically code is a skill.

• Most of the task isn’t the actual coding, it’s articulating a fuzzy problem in a clear, easily understandable way.

• This is more short term, but the pace of change in the tooling (e.g., the emergence of skills) means I’m spending a fair amount of time getting to know how to use the tools appropriately; which usually means going down various rabbit holes to discover how NOT to use them. That creates some (temporary?) friction that means I might fall back to my own skills when there’s production pressures.

Note: Leaders can distribute DX’s prompting guide and advanced prompting guide internally with developers as a way to help them build a foundational practice with AI.

On tool maturity:

• The tools aren’t mature enough to fit into existing systems cleanly, and most developers haven’t figured out how to use them well yet. Not because they’re bad engineers — the workflow is just genuinely new and nobody’s handed them a playbook.

• The difference between frontier models and models even a few months back can be significant.

• Most developers are used to working inside well-defined tools — your IDE, your terminal, your CI pipeline. Everything has a clear place and a clear trigger. AI assistants don’t work that way… So you end up with higher adoption among developers who naturally gravitate toward that kind of open-ended tooling, and much lower adoption among developers who just want something that fits cleanly into what they’re already doing.

5. Most AI tools lack important context

AI tools perform well on problems that are self-contained and well-documented. But most real engineering work isn’t like that. The context that matters—why a system was designed a certain way, what the implicit rules are, what business constraints apply—typically isn’t written down. It lives in people’s heads. Until that knowledge is made explicit and accessible, AI will keep hitting a ceiling on the kinds of problems it can reliably help with.

• The bigger issue I keep running into is that most codebases and systems aren’t set up for AI to actually help. Not an architecture problem — more that the knowledge of how things work lives in people’s heads. Why this service behaves this way, what the implicit contract between these two systems is, why that design decision got made three years ago — none of that is written down anywhere. An AI assistant can’t reason over a Slack thread from an archived channel or the mental model of the engineer who built it. Until that context gets surfaced into something concrete, you’re always going to hit a ceiling on what the tooling can do.

• AI doesn’t have the context I do on making those decisions, as LLMs are built of massive stats, not direct knowledge of the niche ends of what I’m accomplishing.

• I would (anecdotally) put forth that a lot of my work with AI is trying to get it setup to do the work with proper guardrails. It’s not safe to let run loose on our code-bases or production facing infrastructure as its not deterministic.

• AI models are generally good at understanding what we are trying to achieve, especially when they have access to the workspace or codebase. They can often provide useful solutions for specific use cases, particularly when the problem is related to foundational aspects of a technology or programming language. However, what they often lack is a deeper understanding of the business context and the various factors that influence business logic. That broader context is important to fully understand the bigger picture of a project. In some cases, this limitation can be mitigated by carefully crafting prompts, but when the business logic becomes complex or the prompts become too large for relatively simple tasks, people may step back and revert to more traditional approaches.

Other observations about the current state of AI tools

While speaking with developers, we heard a few other interesting themes about what they’re currently experiencing.

1. Documentation robustness is improving, but its quality is yet to be determined.

• The biggest quality gain I’m seeing is in improved documentation. Developers hate writing docs, so it will always be the last thing they tackle, or the first to get cut... if they tackle it at all. At least for me, I found our documentation getting much more robust thanks to AI being able to take that task off our plate. And it’s not just great for the developers to have access to these docs. Giving the AI agents access to it provides a quicker context boost to know how to tackle the next project.

• Using it to write code documentation leads to terribly incorrect information, an active detriment to future work, creating more work. This is really painful. It’s just not its forte.

2. AI is especially powerful in greenfield projects.

• Could you whip up an app from scratch 3x faster with AI? I would believe that. And if I had to just guess, that’s where I think a lot of the hype comes from. Someone says “look I created this thing from scratch in 1 day!” But, I think it is known that AI isn’t quite as proficient when working on a mature, large codebase...with legacy code and lots of nuance. Which is where most ICs are living in day to day, I think.

• While multi-tasking on my main efforts, I was able to build an entire new microservice from scratch to tests an idea and provide a proof of concept of a new paradigm (this has massive comprehension debt, I haven’t even looked at the code, but the service does what I expect it to).

• It has helped a lot with doing things like fast PoCs to check if something will work.

3. There’s concern that developers are offloading critical thinking.

• I would say I’m losing muscle memory on how to do things (e.g. terminal commands like checking git history for when a file changed, checking what version of a dependency is installed, etc.). I’ve grown dependent on the AI to remember the right commands for me.

• I think if someone blindly uses AI for everything, he/she slowly forgets day to day things.

• AI helps speed up some tasks but may result in folks offloading their thinking. Particularly amongst junior developers, the ‘improved initial speed’ can come at the expense of skill development. Overall speed improvement is negligible.

• I have also found myself having to explain to more junior engineers why what AI said about a problem isn’t the correct answer for us in a particular situation. AI empowers more junior engineers to work longer without asking engineering questions of the more senior folks. That is a sharp double-edged sword that I am still learning how to work with when mentoring people.

• [For me, AI’s] handiness is augmented by me knowing when it’s wrong due to my engineering experience.

Final thoughts

Two takeaways stand out to me from these conversations. First, the gains from AI code assistants are structurally limited by how much of the job is actually coding. Microsoft’s research puts coding at around 16% of a developer’s time. Even dramatic acceleration to code generation can only move the overall needle so much.

Second, it’s the human aspects of the software development lifecycle that are the biggest bottlenecks. Handoffs between teams, the skill required to effectively prompt and manage agents, and the review burden that AI-generated code creates—these are where time is going, and where the next wave of productivity gains may come from.

For leaders, this raises a couple of important questions. Which parts of the SDLC beyond coding are candidates for AI assistance? And are the human conditions for capturing AI’s upside in place?

The 10% figure is not the ceiling. But closing the gap will likely require focusing on the parts of the job that AI hasn’t touched yet.

If you have something to add, please share in the comments or on LinkedIn.

This week’s featured DevProd job openings. See more open roles here.

• Amplitude is hiring an Engineering Manager, Cloud Platform | San Francisco, CA

• BNY is hiring an SVP, Application Development Manager | Pittsburgh, PA

• Figma is hiring a Staff Software Engineer, Developer Experience | Remote; US

• Lob is hiring a Staff Platform Engineer | Remote; US

• Plaid is hiring a Software Engineer - Platform | New York, NY

• Vercel is hiring a DX Engineer | Hybrid (Austin, New York City, San Francisco)

• Zillow is hiring a Senior Product Manager, Developer Experience | Remote; US

That’s it for this week. Thanks for reading.

Share

Subscribe now

🗓 Join Abi on March 19th for a live Q&A session. He will address some of the more pressing questions we’ve received around measuring AI impact, the impact of tool choice, and more. Register here.

Social media and vendor marketing have set high expectations for AI, suggesting as much as 2-3x productivity gains. But from the data we’re seeing, the reality on the ground is far more modest.

At DX, we’re currently conducting a longitudinal study to measure the long-term impact of AI adoption on key engineering productivity metrics. As part of this study, we analyzed data from a random sample out of 400 companies between November 2024 through February 2026 to track whether teams are shipping more pull requests as AI adoption increases.

We found that, during this time, AI usage increased significantly—by an average 65%. However, PR throughput only increased by 9.97%.

Note: This figure is particularly robust because we’ve filtered out potential gamification effects by excluding teams that set PR throughput targets for individual engineers, which could drive metric inflation rather than genuine output.

What this means for leaders

A ~10% gain is consistent with what we’re hearing from engineering leaders more broadly: most organizations are landing in the 8–12% range. It is a real improvement, but it’s a long way from the 2–3x gains many executives and boards have come to expect. AI is moving the needle, but leaders may need to reset expectations internally.

Why gains aren’t higher

To understand what’s driving this, we spoke with developers across several of these organizations. The explanation we heard most consistently: writing code was never the bottleneck.

As one senior developer put it: “The easy tasks are a little easier. The tedious tasks are a little less annoying. A four-day task might take three. But that doesn’t mean I’m shipping 3x more PRs.”

AI may be accelerating the coding portion of the job. But coding represents a relatively small slice of how engineers actually spend their time. Planning, alignment, scoping, code review, and handoffs—the human parts of the SDLC—remain largely untouched.

What’s next

We’re continuing to investigate the long-term effects of AI in engineering teams. The full study will explore why some teams are capturing more of the upside than others, and what leaders can do to close that gap. More to come.

This week’s featured DevProd job openings. See more open roles here.

• Amplitude is hiring an Engineering Manager, Cloud Platform | San Francisco, CA

• Figma is hiring a Staff Software Engineer, Developer Experience | Remote; US

• Plaid is hiring a Software Engineer - Platform | New York, NY

• UserTesting is hiring a Software Engineer, Developer Experience (Platform) | Spain

• Vercel is hiring a DX Engineer | Hybrid (Austin, New York City, San Francisco)

That’s it for this week. Thanks for reading.

Share

Subscribe now

🗓 Next month, you can join me for a live Q&A session. I’ll address some of the more pressing questions we’ve received around measuring AI impact, the impact of tool choice, and more. Register here.

This week on the Engineering Enablement podcast, we were joined by Uma Namasivayam, Senior Director of Engineering Productivity at Dropbox. Uma’s team owns engineering productivity across Dropbox’s roughly 1,000 engineers: everything from CI/CD systems and telemetry infrastructure to the company’s AI tooling rollout.

During the conversation, Uma shared specifics on how Dropbox drove AI adoption from one-third of engineers to three-quarters, why they chose to deploy multiple AI coding tools rather than starting by standardizing on one, and how they ended up building their own internal AI platform.

Below is a lightly edited excerpt from this conversation. You can listen to the full episode here.

A lot of engineering orgs treat developer productivity as an engineering problem. You have a different take on that.

Uma: I think of productivity at Dropbox—or anywhere, really—as a sociotechnical problem. There absolutely has to be a strong investment in the technology itself: improving the reliability of your systems, improving speed, and reducing friction in your tooling. But then there’s the element of collaboration, of culture, of working with leadership, with developers themselves, and with the people team.

A concrete example: one of the dimensions of developer productivity is deep work—can developers actually code uninterrupted? That’s not necessarily an engineering problem. Working with our chief people officer, we had to literally think about how to restructure meeting times, how to carve out focus blocks for employees. That required a completely different set of partners than fixing a slow CI pipeline. That’s why having a common language across all of those groups, and bringing them together around that language, was so important. You have to attack productivity from multiple different angles.

How did the rollout of AI coding tools intersect with the DevEx work that was already underway?

Uma: I think of them as two parallel work streams that have a lot of overlap in the middle. DevEx is about incremental, systematic friction reduction. It requires aligning with leadership, defining the problems clearly, and making steady progress. AI is about speed. It’s about getting the best tools into developers’ hands quickly, experimenting fast, and staying on the cutting edge.

Before AI can really deliver on its promise, the foundational systems—build and test, telemetry, production observability—have to be in a strong place. Developers need to trust that when they push code through an AI-assisted workflow, the quality guardrails are actually there. Without that trust, you can’t go really hard on AI.

How did you actually get adoption moving? What drove the initial uptake?

Uma: Early on, roughly one-third of our engineers were using AI tools organically, so people were just finding tools they liked on their own. That’s a decent starting point, but it’s not a strategy. The real inflection point came when our exec team made AI a clear company priority. Within about three months of that top-down signal, we got to around three-quarters of engineers using tools on a weekly basis.

With that said, top-down mandates only take you so far. After that initial push, we had to go deeper, starting by looking at what was actually blocking adoption in different parts of the engineering population, and addressing those pockets specifically. That’s where a product mindset comes in: understanding your customers’ actual pain points rather than assuming one solution works for everyone.

You offer developers multiple AI coding tools rather than standardizing on one. What’s the thinking there?

Uma: It comes back to treating this like a product problem. Different teams have genuinely different needs. Our mobile developers, for example, couldn’t use the tools that work well for other parts of the codebase. We had to find something specific to their use case. So we deliberately chose not to be a single-tool shop.

We also learned quickly that some tools that work at a smaller scale fall apart at Dropbox’s scale. We were piloting an AI code review tool and it just didn’t hold up. That pushed us toward building some of this in-house. We now have an internal platform that handles the backend complexity specific to Dropbox’s monorepo, and that other teams can build on. The build-versus-buy decision turns out to be really critical when you’re operating at this scale and at this speed.

One practical thing that also helped: we worked with our procurement and legal and security teams to dramatically reduce the time it takes to evaluate and approve new AI tools, getting that review process down to around three days. When the market is moving this fast, your ability to experiment is only as good as how quickly you can get tools in front of developers.

Once you’ve expanded beyond code completion, what does AI usage actually look like across the SDLC?

Uma: Code completion was the obvious starting point. Once we felt we’d gotten what we could from that, we started looking at the rest of the development lifecycle: code review, testing, and debugging. Every stage of the SDLC is on the table.

What we discovered is that a lot of the commercially available tools for these adjacent use cases didn’t hold up at our scale or for our specific codebase. That’s what led us to build in-house. One of our developers took it upon himself to figure out what kind of platform we could build, using Claude and Claude Code as the foundation, that would work within Dropbox’s environment and that others could build on top of. That platform now handles the backend complexity: deployment, monorepo scale, and testing guardrails. If a team wants to build an AI-assisted code review product, they start from that platform rather than from scratch. It’s one of the things I’m most proud of from the past year.

What’s the hardest unsolved problem you’re carrying into 2026?

Uma: Connecting developer productivity improvements to actual business outcomes. We can show that DXI improved, that AI adoption is up, and that developers are saving hours. But the arc from “developers are more productive” to “we shipped more value to customers faster”—that instrumentation isn’t there yet, and I don’t think anyone in the industry has fully cracked it.

What we’re seeing is that the capacity unlocked by AI is naturally flowing toward migrations and tech debt reduction. That’s actually pretty cool—give engineers more capacity, and they automatically invest it in the right things. But as a leader, I need to be able to answer the CFO and the exec team: where is this capacity going, and how does it connect to revenue? We’re working toward that. If anyone listening has cracked that code, I’d genuinely love to talk.

You can listen to the full conversation with Uma on the Engineering Enablement podcast.

This week’s featured DevProd job openings. See more open roles here.

• American Express is hiring a Sr. Manager, Digital Product Management - DevProd | Hybrid - London UK

• CoreWeave is hiring a Sr. Software Engineer - Developer Experience | Livingston NJ; New York, NY

• DoorDash is hiring an Engineering Manager - Developer Experience | San Francisco, CA; Sunnyvale, CA; Seattle, WA; Los Angeles, CA; New York, New York

• Figma is hiring a Staff Software Engineer, Developer Experience | Remote; US

• Plaid is hiring a Software Engineer - Platform | New York, NY

• UserTesting is hiring a Software Engineer, Developer Experience (Platform) | Spain

That’s it for this week. Thanks for reading.

Share

Subscribe now

🗓 Next month, you can join us for a live Q&A session with DX CEO Abi Noda. He’ll address recent questions around measuring AI impact, the impact of tool choice, and more. Register to join here.

In just the past year, AI has set into motion several major shifts in how engineering organizations are structured, including how teams are composed, where responsibilities sit, and what skills matter. These changes are still unfolding, but patterns are emerging.

To help name some of those shifts, I interviewed two analysts who spend their time talking with engineering leaders across industries: Ruchi Goyal, Global AI Practice Leader at Accenture, and Dr. Kate Holterhoff, Senior Industry Analyst at RedMonk. Below I’ve summarized the observations they shared—with the hope that naming these patterns is a useful reference point for leaders to recognize and navigate the same changes in their own organizations.

Hiring priorities are emphasizing AI fluency and code quality

The software industry is still working through a hiring slump, but the vacuum is being filled by specific AI-centric roles.

• The “AI Engineer” trend: We are seeing a surge in “AI Engineer” titles on LinkedIn. These are roles that are less about training foundational models and more about building the glue between LLMs and production code. (See example roles on Indeed here).

• Quality in focus: At the same time, a new skill is becoming central to engineering hiring: the ability to distinguish good code from mediocre AI-generated code. As Ryan Dahl and others have noted, AI removes the barrier to writing code, but it introduces a new one: dealing with slop.

• Because a focus on quality has become more important, some organizations are seeing friction emerge in the code review process.

Fragmented AI experiments are consolidating into centralized platforms

Early AI adoption in most organizations was scattered, with individual teams running their own experiments with different tools and no shared standards. That’s changing. Enterprises are moving toward centralized models such as a formal AI Center of Excellence or a hub-and-spoke structure that sets organization-wide direction. Goyal notes that “a lot of clients are looking into merging Developer Productivity and Internal Tools teams into one.” This can enable a number of new strategies for the organization:

• Consolidation: Organizations are folding AI initiatives into their internal developer platforms, creating governed paths for how AI gets used across the organization. Without that structure, teams may end up making independent decisions about tools and access, which can be hard to unwind later.

• Orchestration: Managing individual agents is becoming less of the challenge, and coordinating multiple agents working together is where the complexity now lives. Organizations that have already worked through DevOps maturity will recognize the pattern: the evolution tends to move from automation, to orchestration, to choreography. AI adoption is following a similar arc.

• AI is spreading across engineering organizations to the point where most developers are expected to use it as a matter of course. As Holterhoff put it, “AI is becoming water.” As that happens, Platform teams are becoming increasingly responsible for the governance and prompt hardening that makes this safe.

A new operational layer is emerging: LLMOps

As AI becomes part of the engineering infrastructure, someone has to own the operational layer that keeps it running reliably. In many organizations, that responsibility is coalescing into what’s being called LLMOps. Many Platform and DevProd teams are absorbing this responsibility.

• Prompt engineering to guideposts: Ad hoc prompting works for individuals but doesn’t scale across a team or organization. Turning prompts into reusable templates means you’re building institutional knowledge rather than leaving every developer to figure it out independently, and it creates a consistent quality baseline across the SDLC.

• The SWAT Team approach: In many orgs, a specialized team of engineers handles model integration and AI infra, often leveraging the heavy lifting provided by big cloud providers. Organizations that stand up a dedicated group for this work move faster and make fewer costly mistakes, while the rest of the engineering org can focus on building.

• Deployment convergence: We are seeing tighter integration between development environments and production. (For example, Expo’s integration with Replit allows developers to deploy directly to production, bypassing traditional friction points.) This creates real efficiency gains, but it also means the guardrails that used to exist between those environments need to be rethought, which is something Platform teams are becoming increasingly responsible for.

The human factor still matters the most

Despite the automation, the primary challenges are still cultural. As RedMonk has famously noted, “The developers are everything.” AI tools are being designed to support practitioners, not replace them.

• Murky ROI: Individual developers see the value of AI tools clearly enough that many are willing to pay for them personally. But at the organizational level, the picture is murkier. Leaders are measuring the impact of AI while reading headlines claiming 3x productivity gains… and most aren’t seeing numbers that match.

• Security as a skill: A major skill gap isn’t learning to prompt; it’s learning how to use AI securely. Training now focuses heavily on privacy, vetting specific LLMs, and ensuring that agentic workflows don’t bypass critical security guardrails.

• Software engineers as self-optimizers: Developers are naturally inclined to try new tools. That’s generally a good thing, but it means organizations need vetted, safe environments for that experimentation to happen in. The risk is that developers could adopt AI in ways the organization can’t see or govern.

These changes are moving quickly, and most organizations are still early in figuring out what they mean in practice. There’s no single right answer yet, which is part of what makes it valuable to hear from analysts with visibility across many organizations. A special thanks to Ruchi Goyal and Dr. Kate Holterhoff for sharing their perspective for this newsletter.

This week’s featured DevProd job openings. See more open roles here.

• American Express is hiring a Sr. Manager, Digital Product Management - DevProd | Hybrid - London UK

• CoreWeave is hiring a Sr. Software Engineer - Developer Experience | Livingston NJ; New York, NY

• DoorDash is hiring an Engineering Manager - Developer Experience | San Francisco, CA; Sunnyvale, CA; Seattle, WA; Los Angeles, CA; New York, New York

• Figma is hiring a Staff Software Engineer, Developer Experience | Remote; US

• Plaid is hiring a Software Engineer - Platform | New York, NY

• UserTesting is hiring a Software Engineer, Developer Experience (Platform) | Spain

That’s it for this week. Thanks for reading.

Share

Developer productivity is often treated as a tooling problem or a sentiment problem. In reality, it’s neither. It’s a socio-technical systems problem that spans engineering foundations, leadership alignment, organizational design, and culture.

In this episode, I’m joined by Uma Namasivayam, Senior Director, Engineering Productivity at Dropbox, to explore how Dropbox approaches developer experience at scale. We talk about why productivity needs to be framed as a business problem, how executive alignment creates the conditions for meaningful change, and what it takes to treat developer experience as a real product with developers as customers.

We also dig into Dropbox’s approach to AI adoption. Uma shares why strong foundations, such as build, test, and observability, are prerequisites for AI to actually accelerate work, how Dropbox encourages daily AI use without mandating a single tool, and where build-versus-buy decisions break down at scale.

We close with an honest look at what remains unsolved: how to connect gains in developer productivity and AI-driven capacity to real business outcomes, and where engineering leaders should focus next in 2026.

Some takeaways:

Developer productivity is a socio-technical problem

• Productivity cannot be solved through tooling alone; it spans engineering systems, leadership behavior, organizational structure, and people practices.

• Problems like build and test are engineering problems, while problems like focus time and interruptions are people problems, and both matter equally.

• Treating productivity as a system forces tradeoffs to be explicit, rather than hidden inside isolated tooling initiatives.

Executive alignment matters more than any single metric

• Top-down sponsorship creates permission to act, especially when productivity work cuts across org boundaries.

• A shared framework creates alignment, not answers; its value is giving leaders and engineers a common language.

• System metrics matter more than single metrics, because productivity improvements rarely move one dimension in isolation.

• Distributed accountability makes productivity a company problem, not a developer experience team problem.

Developer experience works best when treated as a product discipline

• Developers are customers, and their experience must be understood through both qualitative feedback and quantitative signals.

• Good system metrics do not guarantee good developer experience, which is why sentiment and perception matter.

• DX surveys surface where systems break differently for different teams, such as desktop, mobile, and web developers.

• Continuous feedback loops are essential, combining surveys, direct conversations, and usage data.

• Internal communication is part of the product, reinforcing to developers that their feedback leads to real change.

Prioritization requires structure, not intuition

• Finite capacity makes prioritization unavoidable, even in large, well-resourced engineering orgs.

• Segmenting developer populations clarifies tradeoffs, since different teams experience different bottlenecks.

• DX survey data provides a defensible starting point, but prioritization still requires judgment.

• Leadership-level stack ranking helps resolve conflicts, especially when multiple teams compete for attention.

• Frameworks make hard decisions easier to explain, even when they do not make them easy.

AI and developer experience must advance in parallel

• AI accelerates work, while developer experience reduces friction, and both are required for sustained gains.

• Foundational systems act as plumbing, enabling trust in speed, quality, and safety.

• Without strong CI, testing, and observability, faster code creation increases risk instead of value.

• Trust in guardrails enables confidence in AI-assisted development, especially at scale.

AI adoption succeeds through choice, not mandates

• Early organic adoption revealed real developer needs, rather than forcing a single tool.

• Different teams require different AI tools, particularly for mobile, desktop, and large-repo workflows.

• Supporting multiple tools increased adoption, rather than reducing it.

• Daily use depends on fitting AI into existing workflows, not adding extra steps.

• Habits matter more than access, which is why SDLC-level integration is critical.

Build vs. buy decisions change at scale

• Many AI tools fail when tested at large-company scale, despite working well in smaller contexts.

• Cost and performance become gating factors, not feature completeness.

• Internal platforms can abstract complexity, enabling teams to build AI workflows safely and consistently.

• Shared internal platforms unlock reuse, allowing teams to innovate without rebuilding infrastructure.

• Speed of iteration remains the primary differentiator, even when building in-house.

In this episode, we cover:

(00:00) Intro

(00:45) Dropbox’s engineering org

(01:59) Why developer productivity is a business problem

(04:08) The role of executive sponsorship in developer productivity

(06:02) How DX’s Core Four framework created a shared language

(08:13) Treating developer experience as a product

(11:30) How Dropbox prioritizes developer experience work

(14:20) The challenge of tying developer experience to business outcomes

(16:38) How AI and developer experience intersect at Dropbox

(18:35) The prerequisites for AI adoption to accelerate work

(20:26) How Dropbox encourages daily AI use

(23:12) AI use beyond code completion

(25:00) Managing AI tool demand at scale

(27:56) Early results from Dropbox’s AI efforts

(30:05) Progress on developer experience at Dropbox

(32:55) Advice for organizations investing in developer experience

(34:25) Capacity tradeoffs for developer experience

(35:59) The unanswered questions around AI and capacity in 2026

Referenced:

• DX Core 4 Productivity Framework

• Dropbox.com

Subscribe now

In 2025, we saw engineering leaders focus on rolling out AI coding assistants at scale across their organizations. As these tools became more widely used, it became clear that outcomes depended less on having access to AI and more on how teams were educated and enabled. In response, DX published the Guide to AI Assisted Engineering, outlining best practices and high-value prompting use cases to help engineering teams use AI effectively in their day-to-day work.

Now, as organizations move beyond pilots, the focus has shifted from adoption and enablement to operational improvements and more complex use cases. Successfully applying AI in these contexts requires more structured prompting practices than those used during early experimentation. To support that next step, we’ve created our first supplement to the original guide: Advanced Prompting Guide for AI Engineering.

This new guide follows the same format as the original, with clear Do and Don’t scenarios, full prompt examples, and code output examples. It is vendor-agnostic, with an emphasis on prompting structure, constraints, and context so the techniques can be applied across tools and architectures.

Inside, you’ll find prompt and code examples that focus on:

• Complexity management - For systems with cascading rules or conflicting requirements, the guide demonstrates graph-based prompting to reveal hidden dependencies, prioritize rules, and deal with changing state

• Governance and quality - Workflows that execute controlled validation loops, which result in higher accuracy, and can deal with more edge cases

• Risk mitigation - Dual-implementation strategies can yield more bulletproof outcomes, especially when dealing with critical transactions that require 100% accuracy

• Operational efficiency - Techniques like diff-only refactoring can reduce invasive changes to large, complex code repositories, as well as reduce tokens

These use cases are drawn from interviews, educational talks, and community interaction, and can be deployed across multiple scenarios. Whether you’re using coding assistants, building prompts for agents, or writing specs for spec-driven-development, you’ll find applicable methods in the guide.

One other important update: When the original guide was published, it was written primarily for developers. But 2025 was a pivotal year for elevating traditional non-builders. As highlighted in our Q4 AI Impact Report, engineering leaders are shipping more code, and designers and PMs have the ability to create deeper designs and prototypes. We still encourage engineering leaders to distribute this guide to their engineering teams, but this guide need not be exclusive to engineers. Whether engineer, designer, PM, or leader, if you are working on complex problems, this guide can provide useful perspectives.

Download the guide

This week’s featured DevProd job openings. See more open roles here.

• American Express is hiring a Sr. Manager, Digital Product Management - DevProd | Hybrid - London UK

• CoreWeave is hiring a Sr. Software Engineer - Developer Experience | Livingston NJ; New York, NY

• DoorDash is hiring an Engineering Manager - Developer Experience | San Francisco, CA; Sunnyvale, CA; Seattle, WA; Los Angeles, CA; New York, New York

• Experian is hiring a Software Engineering Manager - Security Platform | Remote

• Gusto is hiring a Sr. Platform Engineer | Denver, CO; San Francisco, CA; Atlanta, GA; Austin, TX; Chicago, IL; Miami, FL; Seattle, WA

• Notion is hiring a Senior Platform Engineer | Dubai, United Arab Emirates

• Plaid is hiring a Software Engineer - Platform | New York, NY

That’s it for this week. Thanks for reading.

Share

Subscribe now

🗓️ We recently announced DX Annual, our flagship conference for developer productivity leaders navigating the AI era. Go here to learn about the event and request an invite to attend.

One of the insights from the AI impact report released last quarter was that newer AI-native tools appeared to be outperforming others (see page 14). Even in side-by-side deployments, AI-native tools like agentic IDEs were associated with higher throughput compared to older or less specialized solutions.

Given how quickly the space is evolving, we decided to revisit this data to see whether those patterns still hold. In this latest analysis, we looked at PR throughput and median adoption rate across a slightly larger sample of companies (219, versus the 170 we examined last quarter). Here’s an updated look at what we’re seeing:

Comparing our latest Q1 2026 data to our Q4 2025 report we observe a performance jump consistently across all tools. This indicates that trends from last quarter’s data are continuing, and that stronger performance is being realized as tool maturity and adoption practice improves.

It’s notable that the throughput rankings themselves have not changed over the last three months, with newer, agentic tools still ranking highest in terms of velocity impact. Regardless of overall rank, we observed velocity increases across every tool, with a few major shifts since Q4 2025:

• Cursor has seen the largest increase in PR throughput. In Q4 2025, daily users merged a median of 2.8 PRs. Today, that number has jumped to 4.1, representing a 46% increase in throughput for frequent users.

• Claude leads for weekly and monthly users, with PR throughput exceeding 4.0. This is a significant move from the 2.6 (weekly) and 2.2 (monthly) reported just one quarter ago.

• GitHub Copilot daily users have also accelerated, moving from 2.5 PRs/week in Q4 to 3.61 today.

• Tabnine continues to show the lowest throughput at 1.83 for daily users. As noted in the previous report, this is likely because Tabnine is common in large enterprises where PR throughput is lower overall due to organizational complexity.

The last quarter of 2025 has been a landmark period for AI, with broader industry education and best practice converging with improved models, like Opus 4.5 and GPT-5.2, and better tool workflows such as those found in Cursor’s Agent Mode and Claude Code’s spec-driven focus.

Despite rapid advancement and innovation, many unsolved problems remain. Limitations such as memory and context constraints provide technical hurdles, while lack of organizational alignment on policies and practices continue to drive cultural challenges. With all this opportunity for improvement, I expect to see deliberate advancements continue, and productivity gains rise even further in the coming months.

While we were analyzing PR throughput data associated with different AI tools, we also looked at which tools are becoming daily drivers. (Note: adoption metrics don’t directly reveal productivity impact, as is mentioned in the DX AI Measurement Framework. But adoption metrics can give us a signal into which tools are being used for daily tasks versus those used only for specialized, less regular tasks.)

• GitHub Copilot remains the leader in overall stickiness, with the highest daily adoption rate at 9.76%. Its integration into the existing workflow may be helping it become a constant companion for developers. Additionally, for many businesses, Copilot is easy to procure since an existing purchasing agreement with Microsoft is likely to already be in place.

• Cursor is rapidly becoming a primary workspace, boasting an impressive 31.56% adoption rate among weekly users.

• Windsurf and Tabnine show a longer-tail adoption pattern. Windsurf, in particular, has the highest monthly adoption at 35.87%, suggesting it is being used as a powerful specialized tool for specific complex tasks rather than a total IDE replacement for most.

Summary

The data provides evidence for the belief that organizations will realize greater productivity gains as developers become more familiar with AI tools. Further, the significant jump we see in throughput shows how quickly the space is advancing. It’ll be fascinating to watch and see whether we see even higher throughput numbers next quarter, or gains will start to level over time.

I’ll conclude with a few reminders on how to use this data:

1. Correlate usage with throughput: Don’t just measure seat count. Ensure that your highest-cost tools are being used by your Daily or Weekly cohorts where the throughput gains are most visible.

2. Avoid vendor lock-in: Because the performance of these tools is evolving so quickly, we recommend a multi-vendor approach. Different tools excel at different frequencies (e.g., Copilot for daily assistance, Claude for weekly deep-dives). See our webinar on data-driven evaluations of AI tools for more information.

3. Measure to confirm ROI: Use these benchmarks to establish a baseline for your own organization. While AI metrics tell you what is happening, your core metrics, like PR throughput, confirm whether the investment is actually working.

This week’s featured DevProd job openings. See more open roles here.

• American Express is hiring a Sr. Manager, Digital Product Management - DevProd | Hybrid - London UK

• CoreWeave is hiring a Sr. Software Engineer - Developer Experience | Livingston NJ; New York, NY

• DoorDash is hiring an Engineering Manager - Developer Experience | San Francisco, CA; Sunnyvale, CA; Seattle, WA; Los Angeles, CA; New York, New York

• Experian is hiring a Software Engineering Manager - Security Platform | Remote

• Gusto is hiring a Sr. Platform Engineer | Denver, CO; San Francisco, CA; Atlanta, GA; Austin, TX; Chicago, IL; Miami, FL; Seattle, WA

• Notion is hiring a Senior Platform Engineer | Dubai, United Arab Emirates

• Plaid is hiring a Software Engineer - Platform | New York, NY

That’s it for this week. Thanks for reading.

Share

Subscribe now

🗓️ In case you missed it: we recently announced DX Annual, our flagship conference for developer productivity leaders navigating the AI era. Go here to learn more and request an invite to attend.

Business leaders understand the ability of centralized developer productivity and internal platform teams to promote better throughput and faster time to market for software features. But, how large should these teams be? What’s the best proportion of developer productivity specialists to other engineers within the company?

To find an answer, our team analyzed data from 39 companies with varying engineering team sizes across a number of industry verticals and looked for trends in organizational composition for teams explicitly dedicated to developer productivity and experience. We found that engineering leaders typically dedicate 2% to 6% of total headcount to centralized developer productivity. However, this ratio is not linear: as organizations scale beyond 1,000 engineers, the percentage of dedicated headcount steadily decreases due to tooling leverage and automation.

I’m excited to continue following these trends as AI efforts become more centralized and embedded in developer productivity teams, as we may see it lead to further hiring and expansion of responsibilities.

Defining the developer productivity umbrella

To ensure these benchmarks weren’t artificially inflated, we used a narrow definition of what counts as a team focused strictly on developer productivity. We only looked at teams that are unambiguously dedicated to internal platforms, developer productivity, and developer experience, and didn’t include teams focused on SRE or infrastructure, for example.

Specifically, we included the following teams in our analysis:

• Developer Experience and Productivity: Teams explicitly named “Developer Experience,” “DevEx,” “Developer Productivity,” “Developer Enablement,” or “Engineering Enablement”

• Engineering Effectiveness: Teams focused on “Engineering Productivity,” “Engineering Effectiveness,” or “Engineering Excellence”

• Internal Developer Platform Engineers: Groups building “Internal Developer Platforms”, “Internal Developer Portals,” “Developer Frameworks,” “Documentation Systems,” “Service Catalogs” or “Developer Tools”

• Build & Release Infrastructure: “CI Infrastructure,” “CI Platform,” “Build Infrastructure,” “Build Systems,” “Core Automation Platforms” focused on CI/CD, and “Release Engineering” teams

• Developer Education and Support: “Developer Education,” “Engineering Support Organization,” and developer onboarding teams

• Test Infrastructure: Teams dedicated to “Test Infrastructure” and “Test Frameworks” (not QA or product testing)

We excluded several categories that often get conflated with developer tooling, such as:

• General Cloud Infrastructure: Cloud operations, data center management, storage infrastructure, and network infrastructure teams

• Business Platform Teams: “Data Platform,” “ML Platform,” “Analytics Platform,” “Payment Platform,” “Content Platform,” and other product or customer-facing platforms

• Site Reliability Engineering (SRE): While SRE teams are critical, they focus on production reliability rather than developer tooling (except when explicitly named as developer-facing)

• Security Infrastructure: Security operations, compliance, and infrastructure security teams

• Product Infrastructure: Teams building infrastructure for customer-facing products rather than internal tools

• Lab Infrastructure: Physical and virtual infrastructure for product testing and validation

Most companies dedicate 4.7% of engineering headcount to developer productivity

In looking at our first sample of teams that have less than 1,000 engineers, we found that most teams dedicate between 2-6% of their overall engineering headcount to centralized developer productivity functions, with an average of 4.7% across the board. Some range past 8% on the high end, and others range below 2%.

For the average team then, per engineer, that means roughly one productivity engineer per seventeen developers at the high range, or one for every fifty engineers at the lower range. The distribution is shown in the chart below.

These numbers are not surprising, given the variance between company cultures, repository/build architectures, and general attitudes towards productivity. DORA’s State of DevOps reports suggest 10-20% investment between centralized teams and embedded productivity work – our study focuses on centralized, named roles. In general, we’ll find higher investment amongst engineering teams with more complicated deployments (microservices, polyglot stacks, monorepo), and lower investment amongst teams that focus mostly on CI/CD with more monolithic architectures.

When we looked at headcount ratios, we also analyzed team structure: even if two companies have 20 productivity engineers, they may organize them differently. Most organizations distribute their developer productivity team members across 2 to 6 distinct teams. However, some companies take a specialist model, spinning up as many as 15 specialized teams to tackle narrow problems like test optimization or internal portals, while others prefer a single, centralized developer experience or developer productivity unit that handles all enablement tasks under one roof.

Benchmarks by company size and industry vertical

While the 2–6% range is a benchmark, the data shows that investment doesn’t scale linearly. As engineering organizations grow beyond 1,000 people, the percentage of dedicated DevProd headcount starts to drop. This is likely due to diminishing returns in investment, as teams grow and scale, single developer productivity resources can take advantage of more automation, knowledge sharing, tooling, and teamwide collaboration to reduce the need for headcount.

We also saw clear differences in how different industries value DevProd investment:

• Technology (4.89%): These companies lead the pack, averaging about one productivity specialist for every 20 engineers. Because software is their primary profit center, shipping features faster directly impacts their bottom line.

• Fintech and Financial Services (4.36%): These organizations follow closely behind as they continue to modernize their infrastructure and prioritize engineering speed.

• Retail (3.8%) and Large Enterprise (3.32%): These sectors typically show lower ratios, though the latter (large enterprises) may have more to do with the diminishing returns noted in the trends of organizations with 1000+ engineers.

How to use these numbers

These numbers shouldn’t be used as a strict quota, but as a guide for your organizational strategy. The “right” team size is one that balances these industry averages with the specific bottlenecks your developers are facing.

If you are building a new “center of excellence,” use these ratios to set your initial headcount. To ensure you’re getting a real return on that investment, combine these benchmarks with a measurement framework like the Core 4 to track whether your new team is actually removing friction and improving the developer’s day.

This week’s featured DevProd job openings. See more open roles here.

• American Express is hiring a Sr. Manager, Digital Product Management - DevProd | Hybrid - London UK

• Capital One is hiring a Product Manager - Developer Experience | Plano TX; McLean VA; Richmond VA

• Gusto is hiring a Sr. Platform Engineer | Denver, CO; San Francisco, CA; Atlanta, GA; Austin, TX; Chicago, IL; Miami, FL; Seattle, WA

• Plaid is hiring a Software Engineer - Platform | New York, NY

• Reddit: Staff Software Engineer - Developer Experience | Remote - United States

That’s it for this week. Thanks for reading.

Share

As we close out 2025, I wanted to step back and take stock of what we have actually learned about AI adoption in engineering organizations. Not just where usage has increased, but where impact is real, where it is overstated, and what questions remain unanswered.

In this special year-end episode, I’m joined by Brian Houck from Microsoft, Collin Green and Ciera Jaspan from Google, and Eirini Kalliamvakou from GitHub. Together, we unpack the research each of them worked on this year and explore how leading organizations are thinking about AI measurement, developer experience, and long-term productivity. We talk candidly about why measuring AI’s impact is so difficult, why familiar metrics like lines of code keep resurfacing despite their flaws, and how multidimensional approaches like SPACE and DORA offer a more realistic lens.

We also look ahead to 2026. We discuss how AI is beginning to reshape the identity of the developer, how junior engineers’ skill sets may evolve, where agentic workflows are gaining traction, and why some of the most widely shared AI studies were misunderstood. This episode is an honest conversation about moving past hype and toward a more grounded, evidence-based approach to AI adoption in engineering teams.

Some takeaways:

Measuring AI impact requires multiple lenses

• There is no single metric that can capture AI’s impact. Developer productivity and experience are inherently multidimensional, requiring trade-offs to be evaluated across speed, quality, collaboration, and meaning.

• Frameworks like SPACE and DORA help avoid metric tunnel vision. They encourage teams to examine complementary signals rather than optimizing one dimension at the expense of others.

• Measurement must reflect systems, not tools. AI does not operate in isolation; its impact depends on organizational context, workflows, and existing engineering practices.

Why familiar metrics keep failing us

• Lines of code remains a deeply misleading metric. AI tends to generate verbose code, making raw output a poor proxy for productivity, quality, or long-term maintainability.

• More code does not equal better outcomes. Excess code can increase maintenance burden, technical debt, and cognitive load over time.

• Easy-to-measure metrics are often the most dangerous. Their simplicity makes them attractive during periods of uncertainty, even when they obscure what is actually changing.

The limits of tracking AI-generated code

• Measuring the percentage of AI-generated code oversimplifies reality. AI may write, delete, refactor, or reorganize code in ways that raw percentages fail to capture.
  AI-generated code does not inherently signal higher risk. In some contexts, AI output may be more consistent or higher quality than human-written code.

• These metrics are better used as supporting signals, not goals. They can inform budgeting, experimentation, or adoption patterns but should not drive performance targets.

How AI is reshaping the role of the developer

• Developers are shifting from implementers to orchestrators. Advanced AI users spend more time framing problems, setting context, and validating outcomes than writing raw code.

• AI fluency is becoming a core skill. Knowing how to guide, correct, and collaborate with agents is increasingly important.

• Adoption follows a progression. Developers tend to move from skepticism to exploration, collaboration, and eventually strategic use as expectations recalibrate.

What this means for junior engineers

• Skill development may accelerate rather than disappear. Junior engineers may practice delegation, planning, and system-level thinking earlier by working with AI agents.

• Technical fundamentals still matter. Understanding architecture, requirements, and failure modes remains essential for supervising AI-generated work.

• Interpersonal skills risk being deprioritized. Managing agents is not the same as managing people, raising concerns about how collaboration skills develop over time.

AI is not just a productivity tool

• Creativity and innovation benefit from friction. Research suggests that exposing decision points and seams can create space for new ideas rather than faster repetition.

• Automating everything is not always desirable. Removing all toil may reduce opportunities for learning, insight, and creative problem-solving.

• AI should augment thinking, not replace it. Tools that surface trade-offs and choices can support better outcomes than those that simply eliminate effort.

High-leverage AI use cases focus on toil

• Developers spend only about 14% of their time writing code. Optimizing coding alone rarely leads to large productivity gains.

• The biggest opportunities lie in removing friction. Documentation, compliance tasks, incident response, flaky tests, and knowledge discovery consistently rank as top pain points.

• AI excels at work developers dislike but must still do. Automating dull, repetitive tasks can improve satisfaction and free time for meaningful work.

Why leadership and change management matter

• AI adoption is a human problem before it is a technical one. Organizations that understand developer pain points deploy AI more effectively.
  Agentic workflows amplify organizational differences. Teams with strong experimentation cultures and feedback loops move faster and with less friction.

• Culture determines outcomes. How leaders communicate expectations, normalize experimentation, and support learning shapes whether AI adoption succeeds or stalls.

Looking ahead to 2026

• Task parallelization is an emerging frontier. Developers are beginning to use agents to explore multiple solution paths simultaneously.

• Collaboration with agents will redefine productivity. Teams, not just individuals, will increasingly work alongside AI systems.

• Research must evolve with the work itself. New workflows will require new metrics, new telemetry, and new ways of understanding impact.

Lessons from the METR paper

• Context matters more than headlines suggest. Results showing slower performance often reflected expert developers working in familiar codebases.

• AI may help most where familiarity is lowest. New domains, unfamiliar systems, and onboarding scenarios show different outcomes.

• Media oversimplification distorts understanding. Nuance is critical when interpreting AI research, especially as studies move into real-world environments.

In this episode, we cover:

(00:00) Intro

(02:35) Introducing the panel and the focus of the discussion

(04:43) Why measuring AI’s impact is such a hard problem

(05:30) How Microsoft approaches AI impact measurement

(06:40) How Google thinks about measuring AI impact

(07:28) GitHub’s perspective on measurement and insights from the DORA report

(10:35) Why lines of code is a misleading metric

(14:27) The limitations of measuring the percentage of code generated by AI

(18:24) GitHub’s research on how AI is shaping the identity of the developer

(21:39) How AI may change junior engineers’ skill sets

(24:42) Google’s research on using AI and creativity

(26:24) High-leverage AI use cases that improve developer experience

(32:38) Open research questions for AI and developer productivity in 2026

(35:33) How leading organizations approach change and agentic workflows

(38:02) Why the METR paper resonated and how it was misunderstood

Referenced:

• Measuring AI code assistants and agents

• Kiro

• Claude Code - AI coding agent for terminal & IDE

• SPACE framework: a quick primer

• DORA | State of AI-assisted Software Development 2025

• Martin Fowler - by Gergely Orosz - The Pragmatic Engineer

• Seamful AI for Creative Software Engineering: Use in Software Development Workflows | IEEE Journals & Magazine | IEEE Xplore

• AI Where It Matters: Where, Why, and How Developers Want AI Support in Daily Work - Microsoft Research

• Unpacking METR’s findings: Does AI slow developers down?

• DX Annual 2026

Subscribe now

🗓️ We recently announced DX Annual, our flagship conference for developer productivity leaders navigating the AI era. Go here to learn about the event and request an invite to attend.

This week, I’m summarizing “The GenAI Divide: State of AI in Business in 2025,” a report that MIT’s Project NANDA (Networked Agents and Decentralized AI) produced earlier this year. The report examines why, despite massive investment in AI, most organizations are seeing little to no business impact. Drawing on interviews, surveys, and analysis of hundreds of AI initiatives, the authors argue that enterprise AI outcomes have split into two camps: a small minority extracting real value, and the rest stuck in pilots. They call this gap the GenAI Divide.

For DevProd and Platform leaders, this report provides a useful lens for understanding why AI initiatives stall inside engineering organizations. It also counters the “AI will replace engineers” narrative.

My summary of the paper

Headlines often suggest that engineers will soon be replaced by AI, or that the software industry is on the brink of collapse. While AI tools are improving quickly and demos are impressive, Project NANDA set out to study how organizations are actually using these systems in practice. As the authors put it, the GenAI Divide reflects “high adoption but low transformation,” with 95% of organizations seeing zero measurable return.

The researchers conducted 53 structured interviews and collected survey responses from 153 senior leaders. The observation window covered roughly six months following initial AI pilots, and the authors are careful to describe the findings as a directionally accurate snapshot rather than a definitive market analysis.

Here are the key findings from the report.

The pilot-to-production gap

The most visible expression of the AI Divide is the steep drop-off from experimentation to real deployment. While many organizations evaluate AI tools, very few succeed in moving them into production. Roughly 60% evaluated task-specific or enterprise AI systems, about 20% reached pilot stage, and just 5% deployed systems that delivered sustained productivity or P&L impact.

Large enterprises struggled most with this transition. They led in pilot volume and investment, but had the lowest rates of successful scale-up. Mid-market organizations moved faster and more decisively, often reaching full implementation within 90 days. Across successful cases, the difference was ultimately about focus:

• Successful teams focused on narrow, workflow-specific use cases with clearly defined operational outcomes, rather than broad or generalized AI deployments.

• Implementation ownership sat with domain leaders and frontline managers, not centralized AI labs or exploratory teams, enabling faster decision-making and clearer accountability.

• AI systems were embedded directly into existing tools, processes, and data flows, reducing context switching and allowing them to operate within day-to-day work rather than alongside it.

Buy beats build

Organizations that crossed the AI Divide approached implementation differently. Rather than building AI systems entirely in-house, they were far more likely to partner with external vendors. Externally sourced tools reached deployment at roughly twice the rate of internal builds, which frequently failed due to brittleness and poor workflow fit.

Additionally, successful buyers treated AI vendors less like SaaS providers and more like service partners. They demanded deep customization, evaluated tools based on measurable outcomes, and expected systems to integrate into existing workflows with minimal disruption.

Individual productivity gains are not the same as organizational transformation

The report makes it clear that while many teams report that AI helps engineers move faster on specific tasks, those gains rarely translate into systemic impact unless teams design the workflows around them. Two critical reasons highlight this point:

1. 70% of users in the study prefer AI for quick tasks like generating simple unit tests or performing basic refactoring, but for complex projects involving multi-week work and stakeholder management, 90% of users still prefer human colleagues.

2. 66% of executives noted that they want systems in their core workflows that learn from feedback and have more robust memory, while many agents repeat mistakes and fail at complex tasks repeatedly.

As one CIO interviewed noted: “Once we’ve invested time in training a system to understand our workflows, the switching costs become prohibitive.” For now, AI will remain a powerful feature of the engineer’s overall toolkit, but not a replacement for the engineer.

Ultimately, the dividing line between human and agent isn’t just “intelligence,” it’s context. Engineers don’t just generate code; they maintain a mental model of the codebase, the business requirements, and the team’s historical preferences, culture, and style. Current AI solutions lack the contextual adaptability and true persistent memory necessary to perform complex human tasks.

Humans remain in the loop

Across the successful deployments examined, a consistent pattern emerged: AI systems were designed to augment human judgment, not replace it. Engineers remained responsible for architecture, correctness, and evolution, while AI handled acceleration, suggestion, and pattern recognition within defined constraints.

Analyzing the findings shows that the future isn’t one in which engineers disappear, but rather one in which their roles shift. There’s less time spent on toilsome rote work and more on the difficult problems of system design, quality, integration, and decision-making.

Final thoughts

The most revealing takeaway from this report is where AI adoption breaks down. The sharp drop from pilots to production suggests that real constraints only surface once AI is embedded in long-lived, complex workflows. That same pattern makes claims about AI replacing engineers feel premature.

If AI were truly ready to replace engineers, we would already be seeing faster delivery at scale, shrinking teams, and systems that reliably design, build, and integrate software with minimal human oversight. Instead, the report shows that most AI efforts fail at the point where real-world context, dependencies, and coordination matter most.

This represents more responsibility for engineers, not less. Software engineering has never been just about producing code. The companies getting real value from AI aren’t the ones trying to eliminate engineers. They’re the ones who understand AI’s capabilities to significantly increase throughput and innovative capacity. These teams invest in adoption, integration, and measurement.

This is the final issue of the Engineering Enablement newsletter in 2025. Enjoy the holidays, and we’ll see you in January.

Subscribe now

Subscribe now

🗓️We recently announced DX Annual, our flagship conference for developer productivity leaders navigating the AI era. Go here to learn about the event and request an invite to attend.

In 2025, AI-assisted engineering moved from an experiment to a core business expectation.

At the start of the year, adoption rates varied widely across organizations, and a path toward widespread use was just beginning to come into focus. Now, at the end of 2025, that picture looks very different: roughly 90% of developers across the industry are using AI tools at least once a month to get work done, with more than 40% relying on them every day.

As adoption has increased, so have our questions about its impact. Looking back at 2025, we’ve learned a lot as an industry about how AI is changing the way software gets made. To close out the year, I hosted a research roundtable with prominent voices in the AI and developer productivity research space. I invited them to reflect on what we’ve learned so far, and to share the questions they’re carrying into 2026.

A few clear themes emerged from that conversation. Below, I expand on those themes and add my own perspective on what they mean for the year ahead.

Watch the full discussion with Brian Houck (Microsoft), Ciera Jaspan (Google), Collin Green (Google), and Eirini Kalliamvakou (GitHub) here.

Lines of code is still a bad metric. Don’t let uncertainty and change make you reach for it

Building on themes from the last decade of developer productivity research, 2025’s research into measuring AI impact landed in a familiar place: there’s no single number that tells you whether AI is actually making a difference. Across the industry, organizations are using a broad range of metrics to measure impact (like the AI Measurement Framework, or check out this research piece on how Google, Microsoft, GitHub, and others are actually measuring productivity).

But even if there are a lot of good patterns out there, there’s also one bad pattern that this group of researchers called out: using lines of code (LOC) as a measurement for AI impact. This raw output metric is easy to measure, and in the absence of a clear alternative (especially in times of change), it can be tempting to reach for numbers that seem predictable and objective. Collectively, this group warned not to mistake output for impact. AI lends itself to writing a lot of lines of code, but that doesn’t necessarily mean a positive impact on your teams, organizations, or business.

Talking to AI and not talking to your colleagues might not be great for teams longer-term

Brian Houck, Sr. Principal Applied Scientist at Microsoft and co-author of the SPACE Framework of Developer Productivity, shared insights from his paper The SPACE of AI: Real-World Lessons on AI’s Impact on Developers, which shows how the impact of AI tools can vary widely across five key dimensions of developer productivity: Satisfaction, Performance, Activity, Collaboration, and Efficiency. While 90% of developers report that AI makes them more productive, fewer than half agree that it makes them more collaborative and communicative with their teammates. As teams use AI for longer periods of time, it will be interesting to see how this pattern plays out when it comes to knowledge sharing or even the long-term maintenance of codebases.

AI changes what skills developers need, but also how they perceive themselves

Eirini Kalliamvakou, Research Advisor at GitHub, shared some details from her recent research The new identity of a developer: What changes and what doesn’t in the AI era. As developers become more fluent with AI, their identity is shifting from traditional “code producer” toward a role focused on directing, delegating, and validating AI-assisted workflows, with creative judgment and strategic orchestration becoming central to their craft.

Interestingly, many of today’s heavy AI users started out as skeptics. Hands-on experience with the tools often changed both their sentiment and their expectations.

This identity shift has real implications for organizations, particularly around career progression, hiring, and upskilling. Both companies and developers need to place more value on AI fluency, systems thinking, and judgment, rather than raw coding output alone. And keeping pace with the ecosystem will require broader AI enablement, not just tool-specific training programs.

Is AI the death of the junior developer, or an accelerant to help them level up faster?

Lots of folks, from new grads to seasoned developers, are concerned about how AI will impact the talent pipeline. The usual narrative is that AI puts junior developers at risk of extinction, because why hire a junior when a senior developer can just delegate tasks to an AI agent instead? This could be a short-sighted optimization that leaves us with no developing talent a few years from now.

Ciera Jaspan from Google offered a compelling alternative perspective: what if the skills that devs need to level up in seniority—strong problem-solving skills, managing work, delegating work, and clearly defining outcomes—are now learned earlier by junior engineers because of the ways they need to interact with agents? Previously, these skills would be delayed because a tech lead or senior team member would take on the brunt of the project and professional management for these juniors. But when juniors are in the role of team lead for a handful of agents, do they actually level up faster because they get more practice solving problems end-to-end, even if their time spent actually typing code is reduced? Of course, in order for this to happen, companies still need to hire juniors, which isn’t always the case anymore. Find out how you might observe this pattern within your own teams.

Collin Green, another Google researcher, connected this back to earlier concerns about communication. Faster leveling through AI interaction doesn’t automatically translate to stronger collaboration skills. If juniors primarily work with AI rather than people, what does that mean for their professional development? And if seniors spend less time mentoring, what downstream effects might that have as well?

Is automation and code generation the right focus for the future of AI tools?

Collin advised on an AI paper that had a slightly different focus: creativity in software engineering. If creativity is the goal, and not productivity or automation, tools might better help us reimagine how work gets done and lead to more impactful outcomes, rather than just getting to the same outcomes faster. A shift in focus from creativity to productivity changes how tools are built and how we use them. The current emphasis on automation covers only a small surface area, with many developers coding only about 1 day a week on average. Everything else—scoping, experimenting, and validating ideas—exists in a very creative space, which AI is well-suited to help with, but many tools started with an emphasis on productivity and efficiency.

At the same time, there are so many tasks classified as “toil” that would serve developers well to be automated. These types of tasks make developers look at their to-do list and say, “ugh, today is not going to be great.” But Collin shared, we’ve had really solid research and technology for automating tasks for the last 40+ years. We should be automating things that are “dull, dirty, and dangerous.” But the most consistent problems that add friction or toil to developers’ days—like tech debt, lack of documentation, compliance tasks, even expense reports—still haven’t been solved with automation. Can AI change that, or are they just too complex to be automated? Is AI the right tool to automate it anyway?

Headlines will oversimplify AI research to the point of being incorrect, even if the research itself is full of nuance

Earlier this year, METR published a study that showed how, in some contexts, developers actually slow down when using AI, even if their own perception is that they were moving faster. This discrepancy between self-perception and actual result definitely made a big splash in our industry, and not a week goes by that I don’t hear someone cite the study as evidence that we’re all careening off an AI-generated cliff.

Importantly, the study itself had a lot more nuance and depth than the one-line headlines captured. And since one-line headlines are mostly what gets shared on LinkedIn and other platforms, it didn’t take long for the (well-done) METR study to be oversimplified and distilled down to the simple point that AI makes developers slower, which wasn’t really the point of the paper at all.

But the headline that AI isn’t actually as helpful as promised definitely resonated with people, and the study was widely shared. Many people felt drawn to the results that more accurately reflected their own personal experience trying to get started with a tool that was unreliable. For others, it was a good antidote to all the hype. One thing that the METR study did show was that AI research was now taking place in situ, meaning with real developers solving real problems, and specifically their problems.

An important thing to remember as 2026 will surely bring even more headlines that are greatly oversimplified summaries: stay curious about what’s behind the number. For the METR study, which was “AI makes devs 19% slower.” But not all devs, and not all context. Those questions are missing from the headlines, so you need to ask them yourself.

Looking to 2026

Closing out 2025, I can confidently say we haven’t seen the full spectrum of AI impact yet. We’ve made a lot of progress on understanding the impact of AI on teams, but we still need to keep digging to assess whether AI is achieving what we want it to, and to fully understand the impact on not just the whole software delivery lifecycle, but all levels of organizations.

One thing is clear to me though: the companies who will see the biggest wins with AI in 2026 are the ones who deeply understand their existing bottlenecks. The real acceleration from AI doesn’t come from using the newest models and testing every new tool; it comes from pointing AI at the real problems that slow developers down.

This week’s featured DevProd job openings. See more open roles here.

• American Express is hiring a Sr. Manager, Digital Product Management - DevProd | Hybrid - London UK

• Capital One is hiring a Product Manager - Developer Experience | Plano TX; McLean VA; Richmond VA

• Gusto is hiring a Sr. Platform Engineer | Denver, CO; San Francisco, CA; Atlanta, GA; Austin, TX; Chicago, IL; Miami, FL; Seattle, WA

• Plaid is hiring a Software Engineer - Platform | New York, NY

• Reddit: Staff Software Engineer - Developer Experience | Remote - United States

• Whatnot is hiring a Software Engineer - Platform | San Francisco, Los Angeles, Seattle, NYC

That’s it for this week. Thanks for reading.

Share

AI engineering tools are evolving fast. Every month brings new coding assistants, debugging agents, and automation capabilities. I want to help engineering leaders take advantage of that innovation while avoiding costly experiments that distract from real product work.

In this episode, Abi Noda and I share a practical, data-driven approach to evaluating AI tools. I walk through how to shortlist tools by use case, design structured trials that reflect real work, select representative participants, and measure impact using baselines and proven frameworks. My goal is to give you a way to test and adopt AI tools with confidence and a clear return on investment.

Some takeaways:

Data-driven evaluations are essential

• Structured, measurable trials prevent bias. Without them, decisions are driven by novelty hype or a few loud voices.

• Define a clear business outcome first (reduce toil, improve delivery speed, or raise code quality).

• Evaluations must inform real decisions, not just check a procurement box.

Choose the right set of tools to evaluate

• Group tools by use case and interaction mode (chat, agentic IDEs, code review assistants, etc.) to ensure fair comparisons.

• Match shortlist size to org capacity to support multiple cohorts and reliable results.

• Multi-vendor strategies reduce lock-in in a rapidly shifting market.

Re-evaluations are essential, not optional

• Incumbent tools must be retested as capabilities evolve and new challengers emerge.

• Triggers for re-evaluation include major feature launches, organic developer adoption of a new tool, and upcoming renewal cycles.

• Every challenger tool evaluation requires a baseline of the incumbent, so you can compare like-for-like.

• A cadence of every 8–14 months ensures decisions reflect the current reality, not the past purchase.

Design trials around research questions

• Start with a hypothesis. It keeps experiments aligned to actual goals.

• Developer sentiment is necessary but insufficient without measurable outcomes.

• Success criteria must be defined in advance to avoid subjective decision-making.

Select representative participants

• Diverse cohorts reveal real impact across languages, teams, and seniority levels.

• Include skeptical and late adopters to uncover onboarding and enablement needs.

• Volunteer-only trials distort results and won’t scale to full org rollout.

Run evaluations long enough to capture true behavior

• Eight to twelve weeks is the minimum to get past the novelty phase and into sustained usage.

• Align evaluation windows to procurement cycles so insights guide buying decisions.

• Short trials lead to false signals and either inflate enthusiasm or create false negativity.

Use self-reported time savings carefully

• Self-reporting is a strong early indicator of perceived usefulness.

• Humans misremember time, often benchmarking against recent AI use.

• Treat CSAT and time savings as directional, not the final truth.

• Objective metrics validate real ROI, including throughput, quality, and innovation time.

Expect variation rather than a single winner

• Different tools shine in different contexts, so multiple standards are often the best path.

• Continuous re-evaluation is required as capabilities evolve every quarter.

• The right goal isn’t the “best tool”, but the best tool for each problem space.

In this episode, we cover:

(00:00) Intro: Running a data-driven evaluation of AI tools

(02:36) Challenges in evaluating AI tools

(06:11) How often to reevaluate AI tools

(07:02) Incumbent tools vs challenger tools

(07:40) Why organizations need disciplined evaluations before rolling out tools

(09:28) How to size your tool shortlist based on developer population

(12:44) Why tools must be grouped by use case and interaction mode

(13:30) How to structure trials around a clear research question

(16:45) Best practices for selecting trial participants

(19:22) Why support and enablement are essential for success

(21:10) How to choose the right duration for evaluations

(22:52) How to measure impact using baselines and the AI Measurement Framework

(25:28) Key considerations for an AI tool evaluation

(28:52) Q&A: How reliable is self-reported time savings from AI tools?

(32:22) Q&A: Why not adopt multiple tools instead of choosing just one?

(33:27) Q&A: Tool performance differences and avoiding vendor lock-in

Where to find Laura Tacho:

• LinkedIn: https://www.linkedin.com/in/lauratacho/

• X: https://x.com/rhein_wein

• Website: https://lauratacho.com/

• Laura’s course (Measuring Engineering Performance and AI Impact): https://lauratacho.com/developer-productivity-metrics-course

Where to find Abi Noda:

• LinkedIn: https://www.linkedin.com/in/abinoda

• Substack: ​​https://substack.com/@abinoda

Referenced:

• Measuring AI code assistants and agents

• QCon conferences

• DX Core 4 engineering metrics

• DORA’s 2025 research on the impact of AI

• Unpacking METR’s findings: Does AI slow developers down?

• METR’s study on how AI affects developer productivity

• Claude Code

• Cursor

• Windsurf

• Do newer AI-native IDEs outperform other AI coding assistants?

Since DX’s founding, customers have asked us for a venue where they can learn from each other and study what the best companies are doing. With AI transforming the SDLC at an unprecedented rate, the need for a gathering like this is more important than ever.

The inaugural DX Annual will be held on April 16th in San Francisco, bringing together a curated group of ~400 senior engineering leaders from companies like Pinterest, Nationwide, Dropbox, Netflix, and Dell. The program will include keynotes, fireside chats, and roundtables focused on the top questions leaders are facing: how to apply AI across the SDLC, scale best practices, and rethink the role of DevProd teams.

There will be a strong focus on facilitating meaningful and relevant connections among peers.

🔗 Request an invite here.

If you have questions about the event or want to explore whether it’s a fit for you or someone on your team, reply to this email. I’d love to hear from you.

-Abi

Subscribe now

Announcements:

🗓 Laura is hosting a year-in-review roundtable next week with developer productivity researchers from Microsoft, Google, and GitHub, covering this year’s biggest insights and what engineering leaders should expect in 2026. Sign up here.

📚 Abi just announced his book, Frictionless, with Nicole Forsgren, on developer experience. Get a copy here.

Across engineering organizations, AI is often framed as a cost-cutting tool: automate work, reduce headcount, and run leaner teams. But as more companies move beyond pilots, a different pattern is emerging. When AI is applied to the right problems, it doesn’t shrink engineering teams. It increases the return on every engineer you hire.

Zapier is a clear example of this shift. I recently had the chance to interview Andrew Kordampalos, who leads Zapier’s AI Agents transformation. Zapier has built an internal ecosystem of lightweight AI agents that automate thousands of operational and administrative tasks across engineering, product, and internal operations. Their experience points to a simple conclusion: as AI increases the value of each engineer, it becomes more rational to hire more engineers, not fewer.

This article looks at how Zapier reached that conclusion and how they built a culture that treats AI as a force multiplier for hiring and value creation, rather than as a headcount-reduction strategy.

Using AI to remove friction around engineering work

Zapier’s starting point was not “How do we replace developers?” but “Where does work slow developers down?” When Kordampalos joined, his team had grown out of a startup (acquired by Zapier) that originally built real-time meeting transcript agents. Their team quickly turned towards a bigger question: Would it be more effective to automate the choreography around engineering rather than the core engineering itself?

With this line of thinking, they formed a hypothesis that engineers would become far more productive if the friction around them dissolved through agentic automation. The goal was not to change what engineers do, but to change how much of their time is spent on work only they can do.

Zapier’s AI Agents team introduced a network of lightweight internal agents that automate the coordination and overhead surrounding development work:

• Agents that collect async standup updates and summarize them, allowing teams to replace five daily standups with two weekly sessions.

• Agents that run onboarding steps, including generating email signatures and triggering access to tools, reduce onboarding time to roughly two weeks—significantly less than the industry standard 30–90 days.

• Agents that manage Slack workflows, approvals, and operational routines, cutting down on context switching and interruptions.

The underlying mindset is simple: “When you find you’re doing the same thing again and again, you build the automation.” Zapier already had a culture of “build the robot.” AI agents gave that culture a larger surface area.

How AI agents changed Zapier’s hiring math

As Zapier deployed more internal agents, they saw a compounding effect. New hires reached productivity faster. Existing employees increased their throughput without extending their working hours. The organization started to experience AI not as a way to substitute for engineers, but as infrastructure that made every engineer more effective.

This is where the economics shift. If AI removes 10–15% of the administrative and coordination work from an engineer’s week, their effective output goes up. The cost of hiring remains the same, but the value per hire increases. In that scenario, reducing headcount undercuts your own leverage.

Zapier leaned into this logic. As Kordampalos puts it, “We are doubling down and hiring even more people because we want to boost our productivity by using AI.” AI didn’t replace engineers; it replaced the parts of engineering work that prevent engineers from doing their best engineering.

The move from summarizing meetings to a full agent ecosystem

Zapier’s path to a broad agent ecosystem began with a narrower use case. Through the acquisition of Vowel, Kordampalos had already been working with early AI models that transcribed and summarized meetings in real time. That work raised a broader ambition: to make automation conversational, accessible, and embedded in daily work, rather than something only specialists configured. “We realized that the future of creating automations wouldn’t be drag-and-drop interfaces,” he says. “It would be natural language, that’s the universal interface.”

Instead of relying solely on traditional workflows, teams at Zapier began building small agents using their own platform to handle repetitive internal tasks: managing approvals, posting updates, summarizing meetings, and even generating email signatures for new hires. Over time, “There are more bots than humans at Zapier” moved from a joke to a fair description of reality.

For most engineers, the biggest drag on productivity isn’t code, it’s meetings, as I further explore in this article about the true opportunities for productivity improvement with AI. When Kordampalos’s team grew, he noticed they were spending too much time on daily standups. So they built a pair of agents: one that pinged each team member for their async updates, and another that summarized them for everyone. Almost overnight, the daily standup became a twice-a-week sync. “It’s not always one magic automation that replaces your work,” he explained. “It’s the orchestration of smaller agents and bots that you have.”

The insight that multiple small agents can work together like a team (referred to at Zapier as a “pod”) has guided Zapier’s approach. They use Slack as the coordination layer. Agents communicate there, respond to emoji reactions, post updates, and even hand tasks to one another. This ecosystem evolved organically, with employees encouraged to test new agents in sandbox channels before rolling them into production.

On average, Kordampalos says, a new idea takes days, not weeks, to become a working agent.

What Platform and DevEx leaders can learn from Zapier’s model

Zapier’s experience offers a playbook for any platform or DevEx team looking to introduce AI-driven automation inside their organization:

1. Start with the real bottlenecks. Automate daily stand-ups, status reporting, onboarding steps, and the tasks that steal cognitive energy from engineers.

2. Keep the feedback loops tight. Give teams space to experiment safely. Launch agents in controlled channels, observe the signal-to-noise ratio, and scale what works.

3. Build observability and governance early. Create a single dashboard or inventory of agents to manage ownership, access, and performance.

4. Tie automation to outcomes, not novelty. Focus on measurable gains: reduced meeting hours, faster onboarding, or improved throughput per engineer.

Conclusion: replace or augment?

Zapier effectively tested both paths. They could have treated AI as a way to justify a smaller engineering organization. Instead, by focusing on friction reduction, they turned AI into infrastructure that increased the value of every engineer and made additional hiring more attractive.

Their lesson: AI shouldn’t replace engineers. It should replace the friction around engineers, freeing organizations to hire confidently and get more value from each individual contributor. In a world where AI is reshaping how software gets built, the companies that win will be the ones that use AI to augment their teams and accelerate headcount investment, not the ones that try to do the same work with fewer people.

Who’s hiring right now

This week’s featured DevProd job openings. See more open roles here.

• American Express is hiring a Sr. Manager, Digital Product Management - DevProd | Hybrid - London UK

• Capital One is hiring a Product Manager - Developer Experience | Plano TX; McLean VA; Richmond VA

• Plaid is hiring a Software Engineer - Platform | New York, NY

• Reddit: Staff Software Engineer - Developer Experience | Remote - United States

• Tesco: Senior Product Manager - Infrastructure | Hybrid - Tesco UK, Welwyn Garden City

• Whatnot is hiring a Software Engineer - Platform | San Francisco, Los Angeles, Seattle, NYC

That’s it for this week. Thanks for reading.

Share

Subscribe now

On December 11th, Laura is hosting a live year-in-review roundtable with developer productivity researchers from Microsoft, Google, and GitHub. Register to join here.

I also just announced my book, Frictionless, with Nicole Forsgren, on developer experience. Get a copy here.

This week, I’m summarizing a recent paper by Microsoft researchers: AI Where It Matters: Where, Why, and How Developers Want AI Support in Daily Work. This study examines where developers actually want AI support in their daily workflows. For leaders evaluating or rolling out AI tooling, it offers insight into where AI can deliver real value and where gaps still exist.

My summary of the paper

Leaders want to integrate AI where developers need it, and this has been an ongoing area of research. However, while recent studies have explored which tasks developers want automated, the authors of this paper saw an opportunity to dig deeper into questions of where in their workflow developers actually turn to AI, why they want help there, and how they decide when to use or avoid it.

To answer these questions, the researchers conducted a mixed-methods study with 860 Microsoft developers. They started by developing a list of day-to-day tasks to ask developers about, as well as a list of AI safety and control features. They then surveyed developers and asked several questions about each task they selected, including:

• How they view each task (how important it is, whether it feels tied to their identity or job pride, whether they would be held accountable if it went wrong, and how difficult or tiring it is)

• How open are they to AI help for that task

• How often do they use AI for that task

• Where they want AI support and where they don’t want it

• Which five Responsible AI principles are most important for AI tools in that task area

Here are the key findings from the study.

Developers’ perceptions of a task strongly shape their willingness to use AI

The researchers wanted to understand whether the way developers feel about different tasks affects how open they are to using AI, and how often they actually use it. Here’s what they found:

• For tasks developers see as important, high-stakes, or difficult, they are more open to using AI and tend to use it more often. They use AI as a way to reduce effort, double-check their thinking, or avoid mistakes, but they still want to stay in control.

• For tasks that feel central to their identity (like core coding or design decisions), developers are less open to letting AI take over, but they do use AI to help them improve or speed things up.

• For “people work” like mentoring teammates or building AI features themselves, developers were much more likely to limit AI involvement. They see these tasks as deeply human, involving skills like judgment, experience, relationship-building, and personal growth. Developers said mentoring needs trust and empathy, and building AI tools requires craftsmanship. In these cases, they prefer to do the work themselves, using AI only at the margins.

• Personal traits matter. Junior developers and those with more AI experience use AI more. People who are more risk-tolerant or enthusiastic about AI tend to rely on AI more often, especially for high-stakes or demanding work. More cautious developers remain careful with AI when the work feels risky.

• Finally, for operational and coordination work, like maintaining systems, setting up environments, updating documentation, and managing logistics, developers strongly want AI help to cut down on repetitive or boring tasks. But there’s a catch: they only want AI to help if it is safe, reliable, and easy to supervise. They don’t want AI making risky changes on its own or replacing the human judgment needed for strategic decisions or stakeholder communication.

In short, AI is welcome for tedious tasks, cautiously accepted for high-stakes technical work, and firmly limited for interpersonal or identity-defining work.

To make these patterns easier to act on, the researchers mapped every task onto a simple chart showing how much developers want AI support (need) versus how much they actually use it today (use). This creates four quadrants that leaders can use as a decision tool:

• Tasks with high need but low use (“Build”) are prime opportunities for new AI support.

• Tasks with high need and high use (“Improve”) benefit from hardening and reliability work; tasks with low need and high use (“Sustain”) should be maintained but not over-invested in.

• Tasks with low need and low use (“De-prioritize”) should remain mostly human-led.

This map offers a practical way to prioritize where AI can help and where automation is less likely to add value.

Developers prioritize AI that is safe, private, transparent, and easy to control, especially when the work is high-stakes

The researchers asked developers to pick the five qualities they think are most important for AI tools to have when helping with their work. Across the board, developers overwhelmingly prioritized the basics: the AI must be reliable and safe, protect sensitive information, explain what it’s doing, stay aligned with the developer’s goals, and be easy to steer or override. These qualities mattered most for technical, high-stakes tasks like coding, testing, and operations, where AI mistakes can create real risk or wasted effort; for more human-facing or creative tasks, developers placed greater emphasis on fairness and inclusiveness.

Individual differences shaped priorities as well: more experienced and AI-savvy developers placed even higher value on transparency and control, wanting AI they can inspect and correct.

Final thoughts

This study offers a grounded way to align AI investments with what developers actually want and need. The quadrant map should be an especially helpful starting point for distinguishing high-leverage opportunities for applying AI from areas where automation is unlikely to pay off.

Who’s hiring right now

This week’s featured DevProd job openings. See more open roles here.

• American Express is hiring a Sr. Manager, Digital Product Management - DevProd | Hybrid - London UK

• Capital One is hiring a Product Manager - Developer Experience | Plano TX; McLean VA; Richmond VA

• Plaid is hiring a Software Engineer - Platform | New York, NY

• Reddit: Staff Software Engineer - Developer Experience | Remote - United States

• Tesco: Senior Product Manager - Infrastructure | Hybrid - Tesco UK, Welwyn Garden City

• Whatnot is hiring a Software Engineer - Platform | San Francisco, Los Angeles, Seattle, NYC

That’s it for this week. Thanks for reading.

-Abi

Share