<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd" xmlns:googleplay="http://www.google.com/schemas/play-podcasts/1.0"><channel><title><![CDATA[Engineering Enablement: Engineering Enablement Podcast]]></title><description><![CDATA[A weekly podcast covering how top companies measure and improve developer productivity]]></description><link>https://newsletter.getdx.com/s/podcast</link><image><url>https://substackcdn.com/image/fetch/$s_!upog!,w_256,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fabb0ddc7-a50e-4130-b210-b90b7896ba15_110x106.png</url><title>Engineering Enablement: Engineering Enablement Podcast</title><link>https://newsletter.getdx.com/s/podcast</link></image><generator>Substack</generator><lastBuildDate>Mon, 16 Nov 2026 05:54:09 GMT</lastBuildDate><atom:link href="https://newsletter.getdx.com/feed" rel="self" type="application/rss+xml"/><copyright><![CDATA[Abi Noda]]></copyright><language><![CDATA[en]]></language><webMaster><![CDATA[abinoda@substack.com]]></webMaster><itunes:owner><itunes:email><![CDATA[abinoda@substack.com]]></itunes:email><itunes:name><![CDATA[Abi Noda]]></itunes:name></itunes:owner><itunes:author><![CDATA[Abi Noda]]></itunes:author><googleplay:owner><![CDATA[abinoda@substack.com]]></googleplay:owner><googleplay:email><![CDATA[abinoda@substack.com]]></googleplay:email><googleplay:author><![CDATA[Abi Noda]]></googleplay:author><itunes:block><![CDATA[Yes]]></itunes:block><item><title><![CDATA[Inside the DevEx Metrics Compass with André Meyer]]></title><description><![CDATA[Researcher Andr&#233; Meyer shares how teams can measure AI&#8217;s impact beyond adoption, choose meaningful productivity metrics, and understand what makes developers&#8217; work rewarding.]]></description><link>https://newsletter.getdx.com/p/inside-the-devex-metrics-compass</link><guid isPermaLink="false">https://newsletter.getdx.com/p/inside-the-devex-metrics-compass</guid><dc:creator><![CDATA[Brian Houck]]></dc:creator><pubDate>Thu, 12 Nov 2026 16:49:03 GMT</pubDate><enclosure url="https://api.substack.com/feed/podcast/224203641/b547bf69079195a3b8c33f8d3c7b7c87.mp3" length="0" type="audio/mpeg"/><content:encoded><![CDATA[<p>Listen and watch now on <strong><a href="https://youtu.be/vt-ZB-i3d34">YouTube</a>, <a href="https://podcasts.apple.com/us/podcast/engineering-enablement-by-abi-noda/id1619140476">Apple</a>, and <a href="https://open.spotify.com/show/3NxjyIsuxeDMQtisDqBy7D">Spotify</a></strong>.</p><p>Are we measuring the impact of AI on engineering, or just how much it&#8217;s being used?</p><p>In this episode of <em>Engineering Enablement</em>, I talk with researcher Andr&#233; Meyer about his DevEx Metrics Compass, why traditional productivity metrics often miss what matters, and how to measure the value AI actually delivers.</p><p>We also revisit Andr&#233;&#8217;s <em>Today Was a Good Day</em> study to explore what makes developers&#8217; work rewarding, how AI is changing that experience, and practical ways to balance focused work, collaboration, and growing code review demands.</p><div id="youtube2-vt-ZB-i3d34" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;vt-ZB-i3d34&quot;,&quot;startTime&quot;:null,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/vt-ZB-i3d34?rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><h2><strong>Some takeaways: </strong></h2><h3><strong>Why traditional metrics fall short</strong></h3><ul><li><p><strong>AI adoption and output metrics don&#8217;t demonstrate business value.</strong> Metrics like AI usage, acceptance rates, and pull request counts reveal activity, not outcomes.</p></li><li><p><strong>Collaboration is largely overlooked in developer productivity measurement.</strong> Developers spend a significant portion of their time communicating and collaborating, yet few metrics capture its impact.</p></li><li><p><strong>Measuring customer and business value remains a major challenge.</strong> Connecting engineering data to customer satisfaction, feature adoption, and business outcomes requires looking beyond traditional development metrics.</p></li></ul><h3><strong>Choosing the right DevEx metrics</strong></h3><ul><li><p><strong>The DevEx Metrics Compass maps more than 120 metrics from research and industry.</strong> It helps teams navigate existing measurement frameworks and identify relevant metrics.</p></li><li><p><strong>There&#8217;s no universal set of DevEx metrics.</strong> Organizations should select measures based on their specific goals, challenges, and level of measurement maturity.</p></li><li><p><strong>More metrics don&#8217;t necessarily produce better insights.</strong> Focus on a manageable set of priorities, with more detailed metrics available to investigate specific problems.</p></li></ul><h3><strong>Building a more effective measurement system</strong></h3><ul><li><p><strong>Telemetry tells you what changed, but not necessarily why.</strong> Pair quantitative data with developer surveys and qualitative feedback to understand the full picture.</p></li><li><p><strong>Metrics should be evaluated and retired as needs evolve.</strong> Regularly assess whether they remain actionable, inform decisions, or show signs of being gamed.</p></li><li><p><strong>Measure AI&#8217;s impact across multiple dimensions.</strong> Look beyond activity to developer satisfaction, quality, efficiency, and business outcomes, tracking changes over time.</p></li></ul><h3><strong>What makes a good day for developers</strong></h3><ul><li><p><strong>Developers find satisfaction in making meaningful progress. </strong>They report better workdays when they can create something valuable, encounter fewer blockers, and have time to focus.</p></li><li><p><strong>AI doesn&#8217;t necessarily diminish the satisfaction developers get from their work.</strong> Developers can still enjoy creating software when they retain control over what gets built and how decisions are made.</p></li><li><p><strong>Not all context switching is harmful.</strong> Observational research revealed that the cost depends on the nature of the task switch, something automated telemetry alone can&#8217;t capture.</p></li></ul><h3><strong>Protecting focus time and improving collaboration</strong></h3><ul><li><p><strong>The timing of meetings matters as much as their frequency.</strong> Scattered meetings fragment the workday and make sustained focus difficult.</p></li><li><p><strong>Teams should coordinate focused work and collaboration time.</strong> Shared schedules can reduce interruptions while preserving opportunities to communicate and solve problems together.</p></li><li><p><strong>AI-generated code is increasing the burden of peer review.</strong> Clear PR expectations, author accountability, and risk-based review practices can help reduce cognitive load and protect code quality.</p></li></ul><h2><strong>In this episode, we cover:</strong></h2><p>(<a href="https://www.youtube.com/watch?v=vt-ZB-i3d34">00:00</a>) Intro</p><p>(<a href="https://www.youtube.com/watch?v=vt-ZB-i3d34&amp;t=162s">02:42</a>) Why AI dashboards fail to capture actual value</p><p>(<a href="https://www.youtube.com/watch?v=vt-ZB-i3d34&amp;t=308s">05:08</a>) What the DevEx Metrics Compass is and why it was created</p><p>(<a href="https://www.youtube.com/watch?v=vt-ZB-i3d34&amp;t=418s">06:58</a>) Why measuring collaboration matters</p><p>(<a href="https://www.youtube.com/watch?v=vt-ZB-i3d34&amp;t=587s">09:47</a>) Determining the right metrics for your organization</p><p>(<a href="https://www.youtube.com/watch?v=vt-ZB-i3d34&amp;t=723s&amp;pp=0gcJCWMAwfN6Pr3D">12:03</a>) Distinguishing measurement gaps from gaps in public reporting</p><p>(<a href="https://www.youtube.com/watch?v=vt-ZB-i3d34&amp;t=875s">14:35</a>) Why some metrics are harder to act on at scale</p><p>(<a href="https://www.youtube.com/watch?v=vt-ZB-i3d34&amp;t=1091s">18:11</a>) Why customer and business value are difficult to measure</p><p>(<a href="https://www.youtube.com/watch?v=vt-ZB-i3d34&amp;t=1203s">20:03</a>) How to know when it&#8217;s time to retire a metric</p><p>(<a href="https://www.youtube.com/watch?v=vt-ZB-i3d34&amp;t=1460s">24:20</a>) Hierarchies of metrics</p><p>(<a href="https://www.youtube.com/watch?v=vt-ZB-i3d34&amp;t=1651s">27:31</a>) How to measure AI&#8217;s impact beyond activity metrics</p><p>(<a href="https://www.youtube.com/watch?v=vt-ZB-i3d34&amp;t=1820s&amp;pp=0gcJCWMAwfN6Pr3D">30:20</a>) Andr&#233;&#8217;s Good Day study: What makes a good workday for developers</p><p>(<a href="https://www.youtube.com/watch?v=vt-ZB-i3d34&amp;t=1916s">31:56</a>) What observing developers revealed about context switching</p><p>(<a href="https://www.youtube.com/watch?v=vt-ZB-i3d34&amp;t=2068s">34:28</a>) What makes developers&#8217; work rewarding in the age of AI</p><p>(<a href="https://www.youtube.com/watch?v=vt-ZB-i3d34&amp;t=2187s">36:27</a>) The tradeoffs of delegating coding to AI</p><p>(<a href="https://www.youtube.com/watch?v=vt-ZB-i3d34&amp;t=2281s">38:01</a>) Balancing collaboration with focused work</p><p>(<a href="https://www.youtube.com/watch?v=vt-ZB-i3d34&amp;t=2531s">42:11</a>) Two ways to improve developers&#8217; workdays</p><p><strong>Where to find Andre&#769; Meyer-Baron:</strong></p><p>&#8226; LinkedIn: <a href="https://www.linkedin.com/in/andrenmeyer">https://www.linkedin.com/in/andrenmeyer</a></p><p>&#8226; Website: <a href="https://andre-meyer.ch">https://andre-meyer.ch</a> and <a href="http://www.devexcompass.com">www.devexcompass.com</a></p><p><strong><span>Where to find Brian Houck:</span></strong></p><p><span>&#8226; LinkedIn: </span><a href="https://www.linkedin.com/in/brianhouck"><span>https://www.linkedin.com/in/brianhouck</span></a></p><h2><strong>Referenced:</strong></h2><p>&#8226; <a href="https://getdx.com/whitepaper/the-cafes-framework/">CAFE(S): Your Agent Is Only as Good as Its Context</a></p><p>&#8226; <a href="https://queue.acm.org/doi/fullHtml/10.1145/3831360">Too Many DevEx Metrics, Too Little Guidance</a></p><p>&#8226; <a href="https://www.microsoft.com/en-us/research/wp-content/uploads/2019/04/devtime-preprint-TSE19.pdf">Today was a Good Day: The Daily Life of Software Developers</a></p><p>&#8226; <a href="https://devexcompass.com/">DevEx Metrics Compass</a></p><p>&#8226; <a href="https://www.linkedin.com/in/gailcmurphy/">Gail Murphy | LinkedIn</a></p><p>&#8226; <a href="https://www.linkedin.com/in/thomas-fritz-aa53a8/">Thomas Fritz | LinkedIn</a></p><p>&#8226; <a href="https://www.linkedin.com/in/patrickomeyer">Patrick Meyer | LinkedIn</a></p><p>&#8226; <a href="https://space-framework.com/">The SPACE Framework</a></p><p>&#8226; <a href="https://arxiv.org/abs/2603.22106">From Technical Debt to Cognitive and Intent Debt: Rethinking Software Health in the Age of AI</a></p>]]></content:encoded></item><item><title><![CDATA[Roblox’s approach to tripling AI code review acceptance rates]]></title><description><![CDATA[Andrew Swerdlow shares how Roblox is building trust in AI agents to autonomously review, test, and deploy code and what that means for engineers.]]></description><link>https://newsletter.getdx.com/p/robloxs-approach-to-tripling-ai-code</link><guid isPermaLink="false">https://newsletter.getdx.com/p/robloxs-approach-to-tripling-ai-code</guid><dc:creator><![CDATA[Brian Houck]]></dc:creator><pubDate>Fri, 30 Oct 2026 14:33:04 GMT</pubDate><enclosure url="https://api.substack.com/feed/podcast/221552896/5859fe03bf7134c8a5d98e1f86a49b9d.mp3" length="0" type="audio/mpeg"/><content:encoded><![CDATA[<p>Listen and watch now on <strong><a href="https://youtu.be/j3YuAfNCZGk">YouTube</a>, <a href="https://podcasts.apple.com/us/podcast/engineering-enablement-by-abi-noda/id1619140476">Apple</a>, and <a href="https://open.spotify.com/show/3NxjyIsuxeDMQtisDqBy7D">Spotify</a></strong>.</p><p>In this episode of <em>Engineering Enablement</em>, I talk with Andrew Swerdlow, VP of Engineering at Roblox, about how the company is building trust in AI agents to review, test, and deploy code autonomously. Andrew explains how Roblox turns past code review feedback into rules for agents and why engineering is shifting toward specifications and intent. We explore what that means for the skills engineers need, how Roblox measures AI&#8217;s impact on productivity, and how it manages costs. We also discuss the guardrails needed to let agents safely ship code with minimal human intervention.</p><div id="youtube2-j3YuAfNCZGk" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;j3YuAfNCZGk&quot;,&quot;startTime&quot;:null,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/j3YuAfNCZGk?rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><h2><strong>Some takeaways: </strong></h2><h3><strong>Building trust in autonomous development</strong></h3><ul><li><p><strong>Generating code is no longer the biggest bottleneck.</strong> Code review, testing, and deployment still slow delivery. Realizing AI&#8217;s full potential requires automating more of the software development lifecycle.</p></li><li><p><strong>Autonomy has to be earned through trust.</strong> Strong testing infrastructure, production canaries, and automated rollback capabilities are essential for allowing agents to operate independently.</p></li></ul><h3><strong>Organization-specific rules lifted AI review acceptance to nearly 70%</strong></h3><ul><li><p><strong>Generic AI models lack organization-specific knowledge.</strong> Companies need ways to capture their internal engineering practices and make that information available to agents.</p></li><li><p><strong>Code review comments are a valuable source of institutional knowledge.</strong> Roblox mines historical feedback to create rules called exemplars that guide AI-generated code and code reviews.</p></li><li><p><strong>Organization-specific guidance makes AI code review more effective.</strong> Roblox&#8217;s AI-generated review comments now have an acceptance rate of nearly 70%, compared with 50&#8211;55% for human-written comments.</p></li></ul><h3><strong>Humans approve the rules, and about 5% of PRs auto-merge</strong></h3><ul><li><p><strong>Human judgment is shifting from reviewing individual changes to defining the rules.</strong> Engineers can establish and approve guardrails that allow AI to review code without requiring constant human intervention.</p></li><li><p><strong>Roblox is already allowing some code to reach production autonomously.</strong> About 5% of PRs are automatically merged after passing AI code review, programmatic safety checks, and risk assessments.</p></li></ul><h3><strong>Intent, specs, and judgment are becoming the core engineering skills</strong></h3><ul><li><p><strong>Engineering work is shifting from code toward intent, architecture, and specifications.</strong> As agents handle more implementation details, engineers will increasingly collaborate on higher-level designs.</p></li><li><p><strong>Experience and judgment remain essential.</strong> Senior engineers are often better equipped to challenge AI-generated solutions and anticipate problems with scalability, maintainability, and system design.</p></li><li><p><strong>Engineering interviews may need to evaluate process rather than just results.</strong> How efficiently candidates use AI, develop specifications, and arrive at solutions could become more important than traditional coding exercises.</p></li></ul><h3><strong>Roblox tracks accepted, rolled-back, and unattended code</strong></h3><ul><li><p><strong>The amount of AI-generated code is becoming a less meaningful metric.</strong> What matters is whether that code survives in production, delivers value, and requires less human intervention.</p></li><li><p><strong>Roblox is experimenting with new measures of productivity.</strong> Agent autonomy, feature velocity, and developer sentiment help reveal whether AI tools are genuinely improving engineering outcomes.</p></li></ul><h3><strong>Session tracing and caching are the main levers on AI cost</strong></h3><ul><li><p><strong>Observability is essential for understanding AI costs and waste.</strong> Roblox tracks agent sessions, token usage, and code outcomes to identify inefficient workflows and opportunities for improvement.</p></li><li><p><strong>The right infrastructure can dramatically reduce costs.</strong> Roblox reduced the cost of individual code review rule evaluations by 90% through caching and shared context.</p></li><li><p><strong>Flexibility matters in a rapidly changing AI landscape.</strong> Rather than building every capability internally or committing to one vendor, Roblox supports multiple models and tools so engineers can adopt the most effective solutions.</p></li></ul><h2><strong>In this episode, we cover:</strong></h2><p>(<a href="https://www.youtube.com/watch?v=j3YuAfNCZGk">00:00</a>) Intro</p><p>(<a href="https://www.youtube.com/watch?v=j3YuAfNCZGk&amp;t=140s">02:20</a>) Building trust in AI across the software development lifecycle</p><p>(<a href="https://www.youtube.com/watch?v=j3YuAfNCZGk&amp;t=284s">04:44</a>) When human oversight helps and when it gets in the way</p><p>(<a href="https://www.youtube.com/watch?v=j3YuAfNCZGk&amp;t=410s">06:50</a>) What AI agents need to know about your organization</p><p>(<a href="https://www.youtube.com/watch?v=j3YuAfNCZGk&amp;t=525s">08:45</a>) How Roblox turns engineering knowledge into rules for AI agents</p><p>(<a href="https://www.youtube.com/watch?v=j3YuAfNCZGk&amp;t=760s">12:40</a>) How AI code review compares to human code review</p><p>(<a href="https://www.youtube.com/watch?v=j3YuAfNCZGk&amp;t=899s">14:59</a>) Capturing engineering knowledge beyond documentation</p><p>(<a href="https://www.youtube.com/watch?v=j3YuAfNCZGk&amp;t=1094s">18:14</a>) How AI changes what engineers need to understand</p><p>(<a href="https://www.youtube.com/watch?v=j3YuAfNCZGk&amp;t=1217s">20:17</a>) The shift to spec-driven development</p><p>(<a href="https://www.youtube.com/watch?v=j3YuAfNCZGk&amp;t=1404s">23:24</a>) How AI changes how we evaluate engineers</p><p>(<a href="https://www.youtube.com/watch?v=j3YuAfNCZGk&amp;t=1593s">26:33</a>) Rethinking engineering interviews in the age of AI</p><p>(<a href="https://www.youtube.com/watch?v=j3YuAfNCZGk&amp;t=1703s">28:23</a>) How Roblox tracks AI-generated code</p><p>(<a href="https://www.youtube.com/watch?v=j3YuAfNCZGk&amp;t=2156s">35:56</a>) Model routing and choosing the right AI tools</p><p>(<a href="https://www.youtube.com/watch?v=j3YuAfNCZGk&amp;t=2390s">39:50</a>) Measuring AI&#8217;s impact on engineering productivity</p><p>(<a href="https://www.youtube.com/watch?v=j3YuAfNCZGk&amp;t=2684s">44:44</a>) Optimizing AI costs at scale</p><p>(<a href="https://www.youtube.com/watch?v=j3YuAfNCZGk&amp;t=2892s">48:12</a>) Builders at Roblox and the future of software engineering</p><p>(<a href="https://www.youtube.com/watch?v=j3YuAfNCZGk&amp;t=3019s">50:19</a>) Building trust in autonomous code review</p><p>(<a href="https://www.youtube.com/watch?v=j3YuAfNCZGk&amp;t=3191s">53:11</a>) The cultural shift toward autonomous engineering</p><p><strong>Where to find Andrew Swerdlow:</strong></p><p>&#8226; LinkedIn: <a href="https://www.linkedin.com/in/andrewswerdlow/">https://www.linkedin.com/in/andrewswerdlow/</a></p><p>&#8226; X:  <a href="https://x.com/andrewswerdlow">https://x.com/andrewswerdlow</a></p><p>&#8226; Website: <a href="https://andrewswerdlow.net">https://andrewswerdlow.net</a></p><p><strong><span>Where to find Brian Houck:</span></strong></p><p><span>&#8226; LinkedIn: </span><a href="https://www.linkedin.com/in/brianhouck"><span>https://www.linkedin.com/in/brianhouck</span></a></p><h2><strong>Referenced:</strong></h2><p>&#8226; <a href="https://getdx.com/whitepaper/the-cafes-framework/">CAFE(S): Your Agent Is Only as Good as Its Context</a></p><p>&#8226; <a href="https://www.roblox.com/">Roblox</a></p><p>&#8226; <a href="https://www.amazon.com/dp/B0CK45BHLX">Tech Leadership by Andrew Swerdlow</a></p><p>&#8226; <a href="https://margaretstorey.com/">Margaret-Anne Storey</a></p><p>&#8226; <a href="https://www.linkedin.com/in/malteubl/">Malte Ubl - Vercel | LinkedIn</a></p><p>&#8226; <a href="https://grafana.com/">Grafana Labs</a></p><p>&#8226; <a href="https://www.linkedin.com/in/kushan02/">Kushan Mehta | LinkedIn</a></p><p>&#8226; <a href="https://engineering.fb.com/2025/08/06/developer-tools/diff-risk-score-drs-ai-risk-aware-software-development-meta/">Diff Risk Score: AI-driven risk-aware software development - Engineering at Meta</a></p>]]></content:encoded></item><item><title><![CDATA[The CAFE(S) framework: Improving AI agent effectiveness]]></title><description><![CDATA[Brian Houck and his co-authors explore how the CAFE(S) framework helps engineering teams improve context quality, reduce costly AI failures, and make agents more effective.]]></description><link>https://newsletter.getdx.com/p/the-cafes-framework-improving-ai</link><guid isPermaLink="false">https://newsletter.getdx.com/p/the-cafes-framework-improving-ai</guid><dc:creator><![CDATA[Brian Houck]]></dc:creator><pubDate>Fri, 16 Oct 2026 14:21:30 GMT</pubDate><enclosure url="https://api.substack.com/feed/podcast/220404919/9d708e59ab8c191cdf46753b74c8f7d9.mp3" length="0" type="audio/mpeg"/><content:encoded><![CDATA[<p>Listen and watch now on <strong><a href="https://youtu.be/V32mImG3Lg8">YouTube</a>, <a href="https://podcasts.apple.com/us/podcast/engineering-enablement-by-abi-noda/id1619140476">Apple</a>, and <a href="https://open.spotify.com/show/3NxjyIsuxeDMQtisDqBy7D">Spotify</a></strong>.</p><p>As AI agents become more capable, context quality is increasingly the limiting factor in their performance. In this episode of <em>Engineering Enablement</em>, I&#8217;m joined by my CAFE(S) co-authors Max Kanat-Alexander (Capital One), Eirini Kalliamvakou (DX), and Margaret-Anne Storey (University of Victoria) to introduce our framework for evaluating whether agents have the context they need to do their jobs well.</p><p>We introduce the five dimensions of CAFE(S): Clarity, Actionability, Fidelity, Efficiency, and Security. We walk through common context failures, how poor context accrues technical and cognitive debt, what it costs in developer productivity and operating spend, how AI is changing the skills engineers need, and why context review could become as important as code review.</p><div id="youtube2-V32mImG3Lg8" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;V32mImG3Lg8&quot;,&quot;startTime&quot;:null,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/V32mImG3Lg8?rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><h2><strong>Some takeaways: </strong></h2><h3><strong>Why context quality matters for AI agents</strong></h3><ul><li><p><strong>Improving models and harnesses isn&#8217;t enough to maximize AI agent effectiveness.</strong> The quality of the context agents receive is just as important, yet organizations rarely evaluate it systematically.</p></li><li><p><strong>Poor context creates compounding costs and risks.</strong> Ambiguous instructions, irrelevant information, and insecure context can lead to runaway token usage, incorrect outputs, security breaches, and additional developer toil.</p></li></ul><h3><strong>The five dimensions of the CAFE(S) framework</strong></h3><ul><li><p><strong>CAFE(S) defines five properties of high-quality context: clarity, actionability, fidelity, efficiency, and security.</strong> Each addresses a distinct failure mode and helps identify where context needs improvement.</p></li><li><p><strong>Clarity may be the most important dimension because ambiguity is often invisible to the person providing instructions.</strong> Agents can&#8217;t reliably infer unstated assumptions, and an incorrect interpretation can compound throughout a task.</p></li></ul><h3><strong>Why model failures are often context failures</strong></h3><ul><li><p><strong>Agents frequently make mistakes because critical knowledge exists only in developers&#8217; heads.</strong> Without documented requirements, constraints, and decisions, agents may build unwanted features or take actions that conflict with the team&#8217;s intent.</p></li><li><p><strong>Security failures can stem from giving agents access to information they shouldn&#8217;t have.</strong> One real-world example involved an agent finding an unrelated API credential and deleting a production database.</p></li></ul><h3><strong>How poor context contributes to technical, cognitive, and intent debt</strong></h3><ul><li><p><strong>Undocumented goals, constraints, and decisions create intent debt that can become more costly with AI agents.</strong> Agents need access to the reasoning behind decisions, including what a system should not do.</p></li><li><p><strong>Better context can help prevent debt, but successful agent workflows can also encourage complacency.</strong> Teams may stop documenting important decisions when agents appear to be working well.</p></li></ul><h3><strong>Creating high-quality context is becoming a core engineering skill</strong></h3><ul><li><p><strong>AI fluency involves learning to anticipate what agents need rather than correcting mistakes after they happen.</strong> More experienced developers proactively define goals, constraints, and success criteria based on lessons from previous failures.</p></li><li><p><strong>Skills traditionally associated with senior engineers may become essential at every level. </strong>Clearly communicating intent and requirements has long been part of senior engineering roles, but AI agents are making those skills increasingly important for all developers.</p></li></ul><h3><strong>Why context review could become as important as code review</strong></h3><ul><li><p><strong>Shared context deserves more scrutiny because its effects multiply across agent interactions.</strong> A mistake in a widely used instructions file can be repeated in every session, while an improvement can make every agent more effective.</p></li><li><p><strong>Context review should be proportional to the importance and reach of the information.</strong> Critical shared instructions may need formal review, while lightweight documentation updates should remain easy to make.</p></li><li><p><strong>Reviewing context can also build shared understanding across engineering teams.</strong> Like code review, the process can help developers align on requirements, decisions, and the intended behavior of their systems.</p></li></ul><h2><strong>In this episode, we cover:</strong></h2><p>(<a href="https://www.youtube.com/watch?v=V32mImG3Lg8">00:00</a>) Intro</p><p>(<a href="https://www.youtube.com/watch?v=V32mImG3Lg8&amp;t=78s">01:18</a>) Meet the CAFE(S) authors</p><p>(<a href="https://www.youtube.com/watch?v=V32mImG3Lg8&amp;t=193s">03:13</a>) Introducing CAFE(S): Why context quality matters</p><p>(<a href="https://www.youtube.com/watch?v=V32mImG3Lg8&amp;t=392s">06:32</a>) Common context failure modes</p><p>(<a href="https://www.youtube.com/watch?v=V32mImG3Lg8&amp;t=510s">08:30</a>) The five dimensions of high-quality context</p><p>(<a href="https://www.youtube.com/watch?v=V32mImG3Lg8&amp;t=753s">12:33</a>) When model failures are actually context failures</p><p>(<a href="https://www.youtube.com/watch?v=V32mImG3Lg8&amp;t=1104s">18:24</a>) Why the panel prioritizes clarity in CAFE(S)</p><p>(<a href="https://www.youtube.com/watch?v=V32mImG3Lg8&amp;t=1354s">22:34</a>) How context quality relates to technical, cognitive, and intent debt</p><p>(<a href="https://www.youtube.com/watch?v=V32mImG3Lg8&amp;t=1535s">25:35</a>) How human interventions reveal context failures</p><p>(<a href="https://www.youtube.com/watch?v=V32mImG3Lg8&amp;t=1691s">28:11</a>) How context quality relates to AI fluency</p><p>(<a href="https://www.youtube.com/watch?v=V32mImG3Lg8&amp;t=1828s">30:28</a>) How AI is changing what makes a good engineer</p><p>(<a href="https://www.youtube.com/watch?v=V32mImG3Lg8&amp;t=2028s">33:48</a>) The case for context review as a discipline</p><p>(<a href="https://www.youtube.com/watch?v=V32mImG3Lg8&amp;t=2324s">38:44</a>) Wrap-up</p><p><strong>Where to find Margaret-Anne Storey:</strong></p><p>&#8226; LinkedIn: <a href="https://www.linkedin.com/in/margaret-anne-storey">https://www.linkedin.com/in/margaret-anne-storey</a></p><p>&#8226; X:  <a href="https://x.com/margaretstorey">https://x.com/margaretstorey</a></p><p>&#8226; Website: <a href="https://margaretstorey.com">https://margaretstorey.com</a></p><p><strong>Where to find Max Kanat-Alexander:</strong></p><p>&#8226; LinkedIn: <a href="https://www.linkedin.com/in/mkanat">https://www.linkedin.com/in/mkanat</a></p><p>&#8226; X: <a href="https://x.com/mkanat">https://x.com/mkanat</a></p><p>&#8226; Website: <a href="https://www.codesimplicity.com">https://www.codesimplicity.com</a></p><p><strong>Where to find Eirini Kalliamvakou:</strong></p><p>&#8226; LinkedIn: <a href="https://www.linkedin.com/in/eirini-kalliamvakou-1016865">https://www.linkedin.com/in/eirini-kalliamvakou-1016865</a></p><p>&#8226; X: <a href="https://x.com/irina_kAl">https://x.com/irina_kAl</a></p><p><strong>Where to find Brian Houck:</strong></p><p>&#8226; LinkedIn: <a href="https://www.linkedin.com/in/brianhouck">https://www.linkedin.com/in/brianhouck</a></p><h2><strong>Referenced:</strong></h2><p>&#8226; <a href="https://getdx.com/whitepaper/the-cafes-framework/">CAFE(S): Your Agent Is Only as Good as Its Context</a></p><p>&#8226; <a href="https://nicolefv.com/">Nicole Forsgren</a></p><p>&#8226; <a href="https://annievella.com/posts/the-productivity-experience-paradox/">The Productivity-Experience Paradox - Annie Vella</a></p><p>&#8226; <a href="https://startup-studio.ai/">Startup Studio</a></p><p>&#8226; <a href="https://arxiv.org/abs/2603.22106">From Technical Debt to Cognitive and Intent Debt: Rethinking Software Health in the Age of AI</a></p><p>&#8226; <a href="https://getdx.com/blog/the-ai-native-developer/">The AI-native developer</a></p><p>&#8226; <a href="https://codescene.com/">CodeScene</a></p>]]></content:encoded></item><item><title><![CDATA[How Meta reduced diff authoring time by 40%]]></title><description><![CDATA[Meta researcher Moritz Beller shares how Meta measures developer productivity and how AI is changing engineers&#8217; work, metrics, and testing.]]></description><link>https://newsletter.getdx.com/p/how-meta-reduced-diff-authoring-time</link><guid isPermaLink="false">https://newsletter.getdx.com/p/how-meta-reduced-diff-authoring-time</guid><dc:creator><![CDATA[Brian Houck]]></dc:creator><pubDate>Fri, 02 Oct 2026 14:40:54 GMT</pubDate><enclosure url="https://api.substack.com/feed/podcast/218108782/32da7f4f8b157f1e5c5f5d76f7454ce3.mp3" length="0" type="audio/mpeg"/><content:encoded><![CDATA[<p>Listen and watch now on <strong><a href="https://youtu.be/d1gFQ_6OWAQ">YouTube</a>, <a href="https://podcasts.apple.com/us/podcast/engineering-enablement-by-abi-noda/id1619140476">Apple</a>, and <a href="https://open.spotify.com/show/3NxjyIsuxeDMQtisDqBy7D">Spotify</a></strong>.</p><p>In this episode of Engineering Enablement, I talk with Meta researcher Moritz Beller about how Meta measures developer productivity. Moritz explains diff authoring time and how the company uses it to evaluate tools and guide engineering decisions. We explore how AI is changing engineers&#8217; work and revealing gaps in traditional metrics. We also revisit his highly influential &#8220;Mind the Gap&#8221; study and discuss the promise and risks of AI agents for testing.</p><div id="youtube2-d1gFQ_6OWAQ" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;d1gFQ_6OWAQ&quot;,&quot;startTime&quot;:null,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/d1gFQ_6OWAQ?rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><h2><strong>Some takeaways: </strong></h2><h3><strong>Diff authoring time enables more precise productivity measurement</strong></h3><ul><li><p><strong>Diff authoring time measures the active work involved in creating, testing, and reviewing a specific code change.</strong> This gives Meta more precision than metrics averaged across an entire day.</p></li><li><p><strong>No single diff metric provides a complete picture of productivity.</strong> Meta considers authoring time alongside throughput, diff size, and quality to make changes in the data easier to interpret.</p></li><li><p><strong>Meta does not use diff authoring time to evaluate individual developers.</strong> It is used at the team, organizational, and company levels to identify trends, regressions, and opportunities for improvement.</p></li></ul><h3><strong>A/B experiments help Meta prioritize engineering investments</strong></h3><ul><li><p><strong>Diff-level measurement allows Meta to quantify whether changes to developer tools and frameworks save engineers time.</strong> This helps leaders decide which improvements deserve further investment.</p></li><li><p><strong>Automatic memoization in the React compiler reduced diff authoring time by roughly 30% compared with implementing caching manually.</strong> Results of that magnitude suggest that foundational improvements can be more valuable than small interface optimizations.</p></li></ul><h3><strong>AI is changing how engineers spend their time</strong></h3><ul><li><p><strong>Meta&#8217;s year-over-year diff authoring time has fallen by more than 40%, while developers are producing more and larger diffs.</strong> Moritz views the company-wide pattern as a strong signal of AI&#8217;s impact.</p></li><li><p><strong>It is not yet clear where all the saved authoring time goes.</strong> Moritz believes engineers are spending more time gathering context and performing work adjacent to coding, while relatively little time goes into writing prompts.</p></li><li><p><strong>As code becomes cheaper to generate, intent becomes more valuable.</strong> The important question is increasingly whether the finished code accurately reflects what the developer intended to build.</p></li></ul><h3><strong>Traditional telemetry misses important engineering work</strong></h3><ul><li><p><strong>Activities like whiteboarding, brainstorming, and architectural alignment are difficult to connect to code changes through conventional telemetry.</strong> Meeting transcripts and other AI-generated records could help make that work more visible.</p></li><li><p><strong>Cheaper code generation does not eliminate the need for architecture.</strong> Skipping a design document can simply transfer the cost to reviewers, who must reconstruct the intended design from the implementation.</p></li></ul><h3><strong>Developer productivity is both measured and perceived</strong></h3><ul><li><p><strong>Moritz&#8217;s &#8220;Mind the Gap&#8221; study connected automatically measured activity with developers&#8217; perceptions of their own productivity.</strong> Time spent coding was an important predictor, but sleep, interruptions, and on-call responsibilities also mattered.</p></li><li><p><strong>A modern version of the study would need to account for agentic work.</strong> Moritz would examine how developers interact with agents, how much they trust them, and whether managing parallel work creates cognitive overload.</p></li><li><p><strong>Higher output does not necessarily improve developer experience.</strong> Brian points to research showing that output can rise while flow, cognitive load, and overall experience remain flat or worsen.</p></li></ul><h3><strong>AI makes testing easier but not necessarily safer</strong></h3><ul><li><p><strong>Agents make it inexpensive to generate unit and end-to-end tests.</strong> This could fill gaps left by developers who previously did little testing.</p></li><li><p><strong>Passing tests can create false confidence when both the implementation and tests reflect the same misunderstanding.</strong> The agent may satisfy its interpretation of an underspecified request without delivering what the developer actually intended.</p></li><li><p><strong>Specification-driven development could become a new form of test-driven development.</strong> Defining correctness before implementation may matter more as agents take on more of the coding.</p></li></ul><h3><strong>Collaboration remains one of the hardest parts of productivity to measure</strong></h3><ul><li><p><strong>Interpersonal dynamics, alignment, and knowledge sharing are central to engineering productivity but poorly captured by existing metrics.</strong> Faster implementation also makes it easier for teams to unknowingly duplicate one another&#8217;s work.</p></li><li><p><strong>Reducing low-quality meetings can produce major throughput gains, but eliminating collaboration creates different problems.</strong> Teams still need enough interaction to generate ideas, share context, and stay aligned.</p></li><li><p><strong>Small human interactions can have measurable value.</strong> Brian&#8217;s research found that informal conversation before and after meetings predicted self-reported productivity better than internet quality.</p></li></ul><h2><strong>In this episode, we cover:</strong></h2><p>(<a href="https://www.youtube.com/watch?v=d1gFQ_6OWAQ">00:00</a>) Intro</p><p>(<a href="https://www.youtube.com/watch?v=d1gFQ_6OWAQ&amp;t=130s">02:10</a>) Moritz&#8217;s role at Meta</p><p>(<a href="https://www.youtube.com/watch?v=d1gFQ_6OWAQ&amp;t=241s">04:01</a>) Measuring diff authoring time</p><p>(<a href="https://www.youtube.com/watch?v=d1gFQ_6OWAQ&amp;t=500s">08:20</a>) Measuring A/B experiments</p><p>(<a href="https://www.youtube.com/watch?v=d1gFQ_6OWAQ&amp;t=731s&amp;pp=0gcJCWMAwfN6Pr3D">12:11</a>) What diff authoring time reveals</p><p>(<a href="https://www.youtube.com/watch?v=d1gFQ_6OWAQ&amp;t=885s">14:45</a>) Planning and leadership reporting</p><p>(<a href="https://www.youtube.com/watch?v=d1gFQ_6OWAQ&amp;t=1116s">18:36</a>) Why Meta measures teams, not individuals</p><p>(<a href="https://www.youtube.com/watch?v=d1gFQ_6OWAQ&amp;t=1353s">22:33</a>) Where developer time goes with AI</p><p>(<a href="https://www.youtube.com/watch?v=d1gFQ_6OWAQ&amp;t=1566s">26:06</a>) AI&#8217;s impact on junior and senior developers</p><p>(<a href="https://www.youtube.com/watch?v=d1gFQ_6OWAQ&amp;t=1638s">27:18</a>) Capturing invisible work with AI</p><p>(<a href="https://www.youtube.com/watch?v=d1gFQ_6OWAQ&amp;t=1772s">29:32</a>) The &#8220;Mind the Gap&#8221; study</p><p>(<a href="https://www.youtube.com/watch?v=d1gFQ_6OWAQ&amp;t=2020s">33:40</a>) Revisiting &#8220;Mind the Gap&#8221; in 2026</p><p>(<a href="https://www.youtube.com/watch?v=d1gFQ_6OWAQ&amp;t=2312s">38:32</a>) AI agents for software testing</p><p>(<a href="https://www.youtube.com/watch?v=d1gFQ_6OWAQ&amp;t=2509s">41:49</a>) What remains hard to measure</p><p><strong>Where to find Moritz Beller:</strong></p><p>&#8226; LinkedIn: <a href="https://www.linkedin.com/in/inventitech">https://www.linkedin.com/in/inventitech</a></p><p>&#8226; X: <a href="https://x.com/Inventitech">https://x.com/Inventitech</a></p><p>&#8226; GitHub: <a href="https://github.com/Inventitech">https://github.com/Inventitech</a></p><p>&#8226; Website: <a href="https://inventitech.com">https://inventitech.com</a></p><p><strong><span>Where to find Brian Houck:</span></strong></p><p><span>&#8226; LinkedIn: </span><a href="https://www.linkedin.com/in/brianhouck"><span>https://www.linkedin.com/in/brianhouck</span></a></p><h2><strong>Referenced:</strong></h2><p>&#8226; <a href="https://getdx.com/report/State-of-AI-Impact-in-Engineering-Q2-Report">State of AI Impact in Engineering Q2 Report 2026</a></p><p>&#8226; <a href="https://arxiv.org/abs/2606.00438">GitHub Copilot and Developer Productivity: An Observational Dose-Response Analysis</a></p><p>&#8226; <a href="https://qb4.acm.org/detail.cfm?id=3807966">From Technical Debt to Cognitive and Intent Debt</a></p><p>&#8226; <a href="https://arxiv.org/abs/2012.07428">Mind the Gap: On the Relationship Between Automatically Measured and Self-Reported Productivity</a></p><p>&#8226; <a href="https://arxiv.org/abs/2605.23135">The Impact of AI Coding Assistants on Software Engineering: A Longitudinal Study</a></p><p>&#8226; <a href="https://dl.acm.org/doi/10.1145/2786805.2786843">When, how, and why developers (do not) test in their IDEs | Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering</a></p><p>&#8226; <a href="https://getdx.com/guide/dora-space-devex/">DORA, SPACE, and DevEx: Which framework should you use</a></p><p>&#8226; <a href="https://arxiv.org/abs/2008.11147">A Tale of Two Cities: Software Developers Working from Home During the COVID-19 Pandemic</a></p>]]></content:encoded></item><item><title><![CDATA[How Okta sets guardrails and context for AI agents]]></title><description><![CDATA[Okta Chief Architect Robert Lucero explains how organizations can securely deploy AI agents while improving engineering productivity, testing, and software delivery.]]></description><link>https://newsletter.getdx.com/p/how-okta-governs-ai-agents-at-enterprise</link><guid isPermaLink="false">https://newsletter.getdx.com/p/how-okta-governs-ai-agents-at-enterprise</guid><dc:creator><![CDATA[Brian Houck]]></dc:creator><pubDate>Fri, 11 Sep 2026 15:26:03 GMT</pubDate><enclosure url="https://api.substack.com/feed/podcast/214945154/730bcabfb8f2312341627e1d34c47850.mp3" length="0" type="audio/mpeg"/><content:encoded><![CDATA[<p>Listen and watch now on <strong><a href="https://youtu.be/KgA5wy4BINk">YouTube</a>, <a href="https://podcasts.apple.com/us/podcast/engineering-enablement-by-abi-noda/id1619140476">Apple</a>, and <a href="https://open.spotify.com/show/3NxjyIsuxeDMQtisDqBy7D">Spotify</a></strong>.</p><p><span>In this episode of Engineering Enablement, I sit down with Robert Lucero, Chief Architect at Okta, to discuss how agentic AI is changing identity and access management. We explore how organizations should define agent identities, why authentication and authorization remain foundational, and how sandboxing, fine-grained permissions, and just-in-time access let agents operate autonomously without creating unacceptable risk.</span></p><p><span>Robert shares what Okta has learned from driving AI adoption among security-minded engineers, what makes a repository ready for AI agents, and why strong testing, CI, and review processes matter even more as AI generates more code. We also discuss AI&#8217;s role in software validation, the need for human judgment, and whether AI will ultimately give the advantage to security teams or attackers.</span></p><div id="youtube2-KgA5wy4BINk" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;KgA5wy4BINk&quot;,&quot;startTime&quot;:null,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/KgA5wy4BINk?rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><h2><strong>Some takeaways: </strong></h2><h3><strong><span>Identity becomes critical when AI agents move beyond content generation</span></strong></h3><ul><li><p><strong><span>AI agents need access to documents, source code, CI systems, ticketing tools, and other resources to perform useful work.</span></strong><span> Identity systems determine which resources they can access and what actions they can take.</span></p></li><li><p><strong><span>Non-determinism makes tightly scoped access especially important.</span></strong><span> Organizations need dynamic policies, fine-grained permissions, and visibility when agents attempt to operate beyond their boundaries.</span></p></li></ul><h3><strong><span>The definition of an agent identity is still evolving</span></strong></h3><ul><li><p><strong><span>Organizations have not yet settled on whether every agent or ephemeral workload needs its own identity.</span></strong><span> An agent might instead operate under an orchestrator, a service account, or the identity of a human owner.</span></p></li><li><p><strong><span>The fundamentals of identity remain the same.</span></strong><span> Agents still need to be authenticated before organizations can apply authorization policies, governance, and access controls.</span></p></li></ul><h3><strong><span>Treat AI agents like new hires</span></strong></h3><ul><li><p><strong><span>Start agents in a sandbox with tightly scoped access and clearly defined tasks.</span></strong><span> Like new employees, agents may seek out tools or information when their environment is missing something they need.</span></p></li><li><p><strong><span>Rely on technical controls rather than assuming an agent has earned trust.</span></strong><span> Sandboxing, guardrails, and just-in-time access can limit the blast radius of unpredictable behavior.</span></p></li></ul><h3><strong><span>Identity systems are a control plane, not an orchestration layer</span></strong></h3><ul><li><p><strong><span>The identity layer governs what agents can access, trigger, and call.</span></strong><span> The workflows connecting coding, review, testing, and deployment agents will likely be orchestrated elsewhere.</span></p></li><li><p><strong><span>Identity systems can provide essential visibility into agent relationships.</span></strong><span> A relational record of agent-to-agent and agent-to-resource access makes it possible to investigate why an agent was able to take a particular action.</span></p></li></ul><h3><strong><span>Successful AI adoption depends on useful outcomes</span></strong></h3><ul><li><p><strong><span>Okta&#8217;s security-minded engineers were initially skeptical because early tools did not reliably help them do their jobs.</span></strong><span> Adoption increased as coding models improved and became more effective in real engineering workflows.</span></p></li><li><p><strong><span>The value of AI extends far beyond writing code.</span></strong><span> Engineers can use it to summarize documentation, review product specs, investigate production issues, and respond to customer requests more efficiently.</span></p></li></ul><h3><strong><span>AI readiness starts with engineering fundamentals</span></strong></h3><ul><li><p><strong><span>Repositories need clear instructions, discoverable dependencies, compatible harnesses, and defined development practices before agents can work effectively.</span></strong></p></li><li><p><strong><span>Strong testing, CI, linting, and code review provide the safety nets AI-generated changes require.</span></strong><span> Improving a repository for AI agents often makes it easier for human developers to work in as well.</span></p></li></ul><h3><strong><span>Passing tests does not prove that AI-generated code is correct</span></strong></h3><ul><li><p><strong><span>An agent can write tests and produce code that passes them without understanding the intended business outcome.</span></strong><span> Reliable validation depends on giving agents access to the right requirements and context.</span></p></li><li><p><strong><span>Human judgment remains essential for determining whether software actually solves the problem.</span></strong><span> Developers, designers, test engineers, and product owners must still evaluate whether the result meets user and business needs.</span></p></li></ul><h2><strong>In this episode, we cover:</strong></h2><p>(<a href="https://www.youtube.com/watch?v=KgA5wy4BINk">00:00</a>) Intro</p><p>(<a href="https://www.youtube.com/watch?v=KgA5wy4BINk&amp;t=136s">02:16</a>) Meet Robert Lucero</p><p>(<a href="https://www.youtube.com/watch?v=KgA5wy4BINk&amp;t=176s">02:56</a>) Why identity becomes critical as AI agents access more systems</p><p>(<a href="https://www.youtube.com/watch?v=KgA5wy4BINk&amp;t=367s">06:07</a>) Governing automated access for non-deterministic agents</p><p>(<a href="https://www.youtube.com/watch?v=KgA5wy4BINk&amp;t=481s">08:01</a>) Defining and managing agent identities</p><p>(<a href="https://www.youtube.com/watch?v=KgA5wy4BINk&amp;t=632s">10:32</a>) Authentication and authorization for AI agents</p><p>(<a href="https://www.youtube.com/watch?v=KgA5wy4BINk&amp;t=813s">13:33</a>) Determining how much autonomy to give AI agents</p><p>(<a href="https://www.youtube.com/watch?v=KgA5wy4BINk&amp;t=1184s">19:44</a>) Identity systems as the control plane for AI agents</p><p>(<a href="https://www.youtube.com/watch?v=KgA5wy4BINk&amp;t=1360s">22:40</a>) Driving AI adoption among security-minded engineers</p><p>(<a href="https://www.youtube.com/watch?v=KgA5wy4BINk&amp;t=1548s">25:48</a>) Why AI&#8217;s impact on engineering extends beyond coding</p><p>(<a href="https://www.youtube.com/watch?v=KgA5wy4BINk&amp;t=1724s">28:44</a>) The three components of AI readiness for repositories</p><p>(<a href="https://www.youtube.com/watch?v=KgA5wy4BINk&amp;t=1855s">30:55</a>) AI&#8217;s role in software testing and validation</p><p>(<a href="https://www.youtube.com/watch?v=KgA5wy4BINk&amp;t=2066s">34:26</a>) Whether AI gives defenders or attackers the advantage</p><p>(<a href="https://www.youtube.com/watch?v=KgA5wy4BINk&amp;t=2161s">36:01</a>) Where to find the best food in the country</p><p><strong><span>Where to find Robert Lucero:</span></strong></p><p><span>&#8226; LinkedIn: </span><a href="https://www.linkedin.com/in/rlucero"><span>https://www.linkedin.com/in/rlucero</span></a></p><p><strong><span>Where to find Brian Houck:</span></strong></p><p><span>&#8226; LinkedIn: </span><a href="https://www.linkedin.com/in/brianhouck"><span>https://www.linkedin.com/in/brianhouck</span></a></p><h2><strong>Referenced:</strong></h2><p><span>&#8226; </span><a href="https://getdx.com/report/State-of-AI-Impact-in-Engineering-Q2-Report"><span>State of AI Impact in Engineering Q2 Report 2026</span></a></p><p><span>&#8226; </span><a href="https://www.okta.com/"><span>Okta</span></a></p><p><span>&#8226; </span><a href="https://openai.com/index/hugging-face-incident-and-the-road-ahead/"><span>The Hugging Face incident and the road ahead | OpenAI</span></a></p><p><span>&#8226; </span><a href="https://github.com/features/copilot"><span>GitHub Copilot &#183; Your AI pair programmer</span></a></p><p><span>&#8226; </span><a href="https://claude.com/product/claude-code"><span>Claude Code by Anthropic | AI Coding Agent, Terminal, IDE</span></a></p><p><span>&#8226; </span><a href="https://queue.acm.org/doi/10.1145/3807963"><span>Eight Myths on Software Engineering and GenAI | Queue</span></a></p><p><span>&#8226; </span><a href="https://www.linkedin.com/in/adriana-corona/"><span>Adriana Corona</span></a></p>]]></content:encoded></item><item><title><![CDATA[Code Quality & AI Readiness at Capital One]]></title><description><![CDATA[Max Kanat-Alexander, Executive Distinguished Engineer at Capital One, discusses how AI is reshaping software development and why strong engineering fundamentals matter more than ever.]]></description><link>https://newsletter.getdx.com/p/how-capital-one-assesses-ai-readiness</link><guid isPermaLink="false">https://newsletter.getdx.com/p/how-capital-one-assesses-ai-readiness</guid><dc:creator><![CDATA[Brian Houck]]></dc:creator><pubDate>Wed, 26 Aug 2026 11:31:48 GMT</pubDate><enclosure url="https://api.substack.com/feed/podcast/212730058/1af29867e3619ccf46328c74fd14afa7.mp3" length="0" type="audio/mpeg"/><content:encoded><![CDATA[<p>Listen and watch now on <strong><a href="https://youtu.be/gEuD4bP17uc">YouTube</a>, <a href="https://podcasts.apple.com/us/podcast/engineering-enablement-by-abi-noda/id1619140476">Apple</a>, and <a href="https://open.spotify.com/show/3NxjyIsuxeDMQtisDqBy7D">Spotify</a></strong>.</p><p><span>In this episode of Engineering Enablement, I sit down with Max Kanat-Alexander, Executive Distinguished Engineer at Capital One, to discuss how AI is changing software development and why strong engineering fundamentals matter more than ever.</span></p><p><span>We explore how the skills engineers need are expanding, why AI amplifies both the strengths and weaknesses of the development lifecycle, and how teams should think differently about code review, quality, and testing. Max also shares how leaders can assess whether their organizations are ready for more advanced AI workflows and why we need more research into how junior engineers become senior engineers.</span></p><div id="youtube2-gEuD4bP17uc" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;gEuD4bP17uc&quot;,&quot;startTime&quot;:null,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/gEuD4bP17uc?rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><h2><strong>Some takeaways: </strong></h2><h3><strong><span>AI amplifies the engineering system you already have</span></strong></h3><ul><li><p><strong><span>Strong engineering fundamentals matter even more with AI.</span></strong><span> AI amplifies everything that is good or bad about a software development lifecycle, so teams with strong tooling, CI, testing, and workflows are positioned to see the greatest gains.</span></p></li><li><p><strong><span>Faster coding doesn&#8217;t solve problems elsewhere in the development lifecycle.</span></strong><span> If engineers spend significant time navigating broken processes, waiting for reviews, or dealing with poor tooling, accelerating code generation only addresses a small part of their work.</span></p></li></ul><h3><strong><span>The skills engineers need are changing</span></strong></h3><ul><li><p><strong><span>Clearly expressing requirements and providing feedback are becoming increasingly important engineering skills.</span></strong><span> These are capabilities senior engineers have traditionally developed through years of experience, but they become essential much earlier when working effectively with coding agents.</span></p></li><li><p><strong><span>Deep technical knowledge still helps engineers recognize when something is wrong.</span></strong><span> Experience with languages, systems, and software design develops the pattern recognition needed to spot a flawed approach without reviewing hundreds of lines of AI-generated code.</span></p></li><li><p><strong><span>Domain understanding and problem definition are becoming more valuable.</span></strong><span> As AI gets better at reproducing knowledge found in documentation, engineers can differentiate themselves through understanding what needs to be built and why.</span></p></li></ul><h3><strong><span>Code review is fundamentally about maintainability</span></strong></h3><ul><li><p><strong><span>Human code review shouldn&#8217;t be the primary way teams catch bugs.</span></strong><span> Tests, static analysis, and other automation are more effective and scalable ways to detect defects.</span></p></li><li><p><strong><span>Human judgment remains important for evaluating maintainability and design.</span></strong><span> Reviewers can assess whether code is understandable, structured appropriately, and aligned with the longer-term intent of a system.</span></p></li><li><p><strong><span>The right quality bar depends on the codebase.</span></strong><span> A disposable script and enterprise software expected to operate for a decade shouldn&#8217;t be subjected to identical standards because the consequences and costs of getting them wrong are very different.</span></p></li></ul><h3><strong><span>Not every change needs the same review process</span></strong></h3><ul><li><p><strong><span>Code review is an important tool for developing less experienced engineers.</span></strong><span> Max says reviewing someone&#8217;s code is the most effective way he has found to improve their core software engineering skills.</span></p></li><li><p><strong><span>Trusted engineers may not need every change reviewed on codebases they deeply understand.</span></strong><span> That trust should be specific to the combination of the person and the codebase, rather than becoming a blanket exemption from review.</span></p></li><li><p><strong><span>The risk of the individual change can also determine whether human review is necessary.</span></strong><span> Teams can evaluate confidence and potential consequences rather than requiring the same review process for every PR.</span></p></li></ul><h3><strong><span>AI can create a vicious cycle of declining code quality</span></strong></h3><ul><li><p><strong><span>AI struggles when it has to build on poorly structured, poorly tested code.</span></strong><span> When weak review allows its bad output back into the codebase, the code becomes even harder for AI to work with, creating a cycle that can eventually leave teams with systems they can&#8217;t understand or maintain.</span></p></li><li><p><strong><span>AI can also help teams reverse that cycle.</span></strong><span> Agents can write tests, improve testability, and are particularly effective at refactoring when engineers deliberately direct them toward improving the codebase.</span></p></li><li><p><strong><span>Refactoring should support forward progress on product goals.</span></strong><span> Rather than attempting to clean up an entire codebase, teams should improve the parts they need to touch for upcoming work and make the necessary developer experience improvements along the way.</span></p></li></ul><h3><strong><span>AI readiness depends on how far left rigor extends</span></strong></h3><ul><li><p><strong><span>Fixing an existing software development lifecycle is much harder than building a good one from the beginning.</span></strong><span> Organizations with weak engineering practices often have to start with production problems and work backward through incident management, observability, deployment, testing, and earlier stages of development.</span></p></li><li><p><strong><span>How far left rigor extends is a useful signal of AI readiness.</span></strong><span> Organizations without strong controls around production, code review, and testing may introduce significant risk by giving engineers tools that allow them to move much faster.</span></p></li><li><p><strong><span>Leaders should evaluate the organization they actually have rather than following the AI hype cycle.</span></strong><span> Max is bullish on AI adoption, but argues that organizations need the engineering foundations to support the additional velocity it creates.</span></p></li></ul><h3><strong><span>The industry needs a better way to develop senior engineers</span></strong></h3><ul><li><p><strong><span>We still know surprisingly little about what reliably turns junior engineers into senior engineers.</span></strong><span> Mentorship and apprenticeship are widely considered important, but the practices that actually produce experienced engineers have not been deeply studied.</span></p></li><li><p><strong><span>AI makes understanding that development path more urgent.</span></strong><span> Senior engineers rely on experience to recognize failure patterns, which raises questions about how newer engineers will acquire that judgment as AI takes on more of the implementation work.</span></p></li><li><p><strong><span>AI may ultimately increase the need for engineering expertise rather than reduce it.</span></strong><span> As hundreds of millions more people gain the ability to build software and the rate of change accelerates, Max believes the world will need more experienced engineers to manage that complexity.</span></p></li></ul><h2><strong>In this episode, we cover:</strong></h2><p>(<a href="https://www.youtube.com/watch?v=gEuD4bP17uc">00:00</a>) Intro</p><p>(<a href="https://www.youtube.com/watch?v=gEuD4bP17uc&amp;t=114s">01:54</a>) Max&#8217;s role at Capital One</p><p>(<a href="https://www.youtube.com/watch?v=gEuD4bP17uc&amp;t=172s">02:52</a>) Where to invest in engineering organizations</p><p>(<a href="https://www.youtube.com/watch?v=gEuD4bP17uc&amp;t=396s">06:36</a>) The new entry-level engineering skills to pay attention to</p><p>(<a href="https://www.youtube.com/watch?v=gEuD4bP17uc&amp;t=639s">10:39</a>) Why deepening your understanding still matters</p><p>(<a href="https://www.youtube.com/watch?v=gEuD4bP17uc&amp;t=749s">12:29</a>) The bottlenecks around code review</p><p>(<a href="https://www.youtube.com/watch?v=gEuD4bP17uc&amp;t=1180s">19:40</a>) Why human code reviews still have value</p><p>(<a href="https://www.youtube.com/watch?v=gEuD4bP17uc&amp;t=1520s">25:20</a>) Why not all PRs need human review</p><p>(<a href="https://www.youtube.com/watch?v=gEuD4bP17uc&amp;t=1571s">26:11</a>) The vicious cycle of AI-driven development</p><p>(<a href="https://www.youtube.com/watch?v=gEuD4bP17uc&amp;t=1813s">30:13</a>) Using LLMs for refactoring</p><p>(<a href="https://www.youtube.com/watch?v=gEuD4bP17uc&amp;t=2038s">33:58</a>) AI readiness and why fixing engineering systems is so hard</p><p>(<a href="https://www.youtube.com/watch?v=gEuD4bP17uc&amp;t=2322s">38:42</a>) Why research is needed on creating good senior engineers</p><p>(<a href="https://www.youtube.com/watch?v=gEuD4bP17uc&amp;t=2460s">41:00</a>) Why AI will increase the need for engineers</p><p><strong><span>Where to find Max Kanat-Alexander:</span></strong></p><p><span>&#8226; LinkedIn: </span><a href="https://www.linkedin.com/in/mkanat"><span>https://www.linkedin.com/in/mkanat</span></a></p><p><span>&#8226; X: </span><a href="https://x.com/mkanat"><span>https://x.com/mkanat</span></a></p><p><span>&#8226; Blog: </span><a href="https://www.codesimplicity.com">https://www.codesimplicity.com</a></p><p><strong><span>Where to find Brian Houck:</span></strong></p><p><span>&#8226; LinkedIn: </span><a href="https://www.linkedin.com/in/brianhouck"><span>https://www.linkedin.com/in/brianhouck</span></a></p><h2><strong>Referenced:</strong></h2><p><span>&#8226; </span><a href="https://getdx.com/corefour"><span>DX Core 4 Productivity Framework</span></a></p><p><span>&#8226; </span><a href="https://www.capitalone.com/"><span>Capital One</span></a></p><p><span>&#8226; </span><a href="https://google.github.io/eng-practices/review/"><span>Code Review Guidelines at Google</span></a></p><p><span>&#8226; </span><a href="https://www.amazon.com/dp/1449313892?lv=shuf&amp;channelId=500&amp;plpRedirect=mhFallback"><span>Code Simplicity by Max Kanat-Alexander</span></a></p><p><span>&#8226; </span><a href="https://getdx.com/podcast/how-microsoft-sees-engineering-bottlenecks-changing-with-ai/"><span>How Microsoft sees engineering bottlenecks changing with AI</span></a></p>]]></content:encoded></item><item><title><![CDATA[AI in engineering: Q2 2026 benchmarks & research readout]]></title><description><![CDATA[Brian Houck and Justin Reock unpack DX&#8217;s latest AI Impact Report, exploring where AI is improving engineering velocity and where those gains are failing to translate into better outcomes.]]></description><link>https://newsletter.getdx.com/p/ai-in-engineering-q2-2026-benchmarks</link><guid isPermaLink="false">https://newsletter.getdx.com/p/ai-in-engineering-q2-2026-benchmarks</guid><dc:creator><![CDATA[Brian Houck]]></dc:creator><pubDate>Fri, 14 Aug 2026 16:00:48 GMT</pubDate><enclosure url="https://api.substack.com/feed/podcast/210943019/2e9703b85b75b80db162b7168468cc74.mp3" length="0" type="audio/mpeg"/><content:encoded><![CDATA[<p>Listen and watch now on <strong><a href="https://youtu.be/Bf44cPeNPjg">YouTube</a>, <a href="https://podcasts.apple.com/us/podcast/engineering-enablement-by-abi-noda/id1619140476">Apple</a>, and <a href="https://open.spotify.com/show/3NxjyIsuxeDMQtisDqBy7D">Spotify</a></strong>.</p><p><span>AI adoption among software developers is approaching 100%, AI-authored code now makes up more than half of merged code, and developers report saving more time with AI every quarter. But those gains aren&#8217;t translating evenly into better outcomes.</span></p><p><span>In this episode of Engineering Enablement, I sit down with Justin Reock, Deputy CTO at DX, to unpack findings from our latest AI Impact Report. We explore where AI is improving engineering velocity and developer experience, where concerns are emerging around PR size, change confidence, and failure rates, and why rising AI spend has yet to produce a comparable increase in innovation.</span></p><p><span>We also discuss how AI is changing the meaning of code maintainability and where developers&#8217; AI-driven time savings may actually be going.</span></p><div id="youtube2-Bf44cPeNPjg" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;Bf44cPeNPjg&quot;,&quot;startTime&quot;:null,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/Bf44cPeNPjg?rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><h2><strong>Some takeaways: </strong></h2><h3><strong><span>AI adoption is no longer the interesting question</span></strong></h3><ul><li><p><strong><span>AI adoption is approaching 100%, making adoption itself a less useful measure of success.</span></strong><span> DX sees 95% adoption through telemetry, and even developers outside that group are merging AI-generated code into production.</span></p></li><li><p><strong><span>The more important question is how effectively developers are using AI.</span></strong><span> As adoption becomes nearly universal, organizations need to look beyond usage and understand how sophisticated their AI usage is and whether or not it is producing meaningful outcomes.</span></p></li></ul><h3><strong><span>AI is increasing engineering velocity, but the gains need context</span></strong></h3><ul><li><p><strong><span>Developers report saving more time with AI each quarter, while PR throughput and deployment frequency are also increasing.</span></strong><span> Recent causal research gives additional evidence that AI is contributing to the increase in throughput.</span></p></li><li><p><strong><span>More than half of merged code is now AI-authored.</span></strong><span> But metrics like time saved, code generated, and PR throughput don&#8217;t tell us on their own whether teams are creating more customer value.</span></p></li></ul><h3><strong><span>Larger PRs create risks that extend beyond code review</span></strong></h3><ul><li><p><strong><span>Average PR size has increased from 42 to 72 lines of code, raising concerns about quality and code understanding.</span></strong><span> Larger PRs are harder to review and revert, more likely to be rubber-stamped, and can move more slowly through the development system.</span></p></li><li><p><strong><span>Code bloat can also create a vicious cycle of rising AI costs.</span></strong><span> As agents generate larger codebases, future agents have more code to read into context, consuming additional tokens repeatedly rather than creating only a one-time cost.</span></p></li></ul><h3><strong><span>Developers can change code more easily while trusting it less</span></strong></h3><ul><li><p><strong><span>Code maintainability is improving while change confidence is declining, even though those measures have historically been closely correlated.</span></strong><span> AI may make code easier to understand and modify while leaving developers less confident that their changes won&#8217;t break something in production.</span></p></li><li><p><strong><span>AI may be changing what &#8220;maintainable&#8221; code actually means.</span></strong><span> If developers increasingly rely on agents to understand and modify code, traditional measures of maintainability and ownership may no longer capture the same things they once did.</span></p></li></ul><h3><strong><span>AI is amplifying differences between engineering organizations</span></strong></h3><ul><li><p><strong><span>Change failure rates are becoming more volatile rather than moving uniformly in one direction.</span></strong><span> Some organizations continue to perform well while others are seeing higher failure rates, suggesting that AI can amplify existing strengths and weaknesses.</span></p></li><li><p><strong><span>Developer experience is also showing concerning signals despite clear improvements in areas like documentation.</span></strong><span> DX&#8217;s Developer Experience Index has fallen about 2% over two quarters, with declines in drivers such as incremental delivery outweighing some of AI&#8217;s benefits.</span></p></li></ul><h3><strong><span>AI spending is growing much faster than measurable innovation</span></strong></h3><ul><li><p><strong><span>AI spend has risen dramatically, reaching roughly 28 times year-ago levels among the largest companies in the dataset.</span></strong><span> At the same time, developer ramp-up has improved, but the innovation ratio has increased by only about one percentage point.</span></p></li><li><p><strong><span>Time saved by AI can easily be absorbed by existing organizational friction.</span></strong><span> Meeting-heavy days, interruptions, build and test wait times, developer environment toil, and review delays remain major sources of lost time. In one real-world case study, reducing unnecessary meetings produced roughly twice the PR-throughput gains achieved through AI.</span></p></li><li><p><strong><span>Ultimately, AI efficiency only matters if it translates into customer value.</span></strong><span> Removing friction and increasing throughput are useful, but the larger question is whether those gains allow organizations to ship more valuable software.</span></p></li></ul><h2><strong>In this episode, we cover:</strong></h2><p>(<a href="https://www.youtube.com/watch?v=Bf44cPeNPjg">00:00</a>) Intro</p><p>(<a href="https://www.youtube.com/watch?v=Bf44cPeNPjg&amp;t=105s">01:45</a>) How the current AI impact report is tied to Core 4</p><p>(<a href="https://www.youtube.com/watch?v=Bf44cPeNPjg&amp;t=204s">03:24</a>) The state of AI adoption</p><p>(<a href="https://www.youtube.com/watch?v=Bf44cPeNPjg&amp;t=312s">05:12</a>) How much time AI is saving developers and percentage of AI-authored code</p><p>(<a href="https://www.youtube.com/watch?v=Bf44cPeNPjg&amp;t=467s">07:47</a>) AI&#8217;s impact on PR throughput and deployment frequency</p><p>(<a href="https://www.youtube.com/watch?v=Bf44cPeNPjg&amp;t=669s">11:09</a>) How EMs are shipping more code</p><p>(<a href="https://www.youtube.com/watch?v=Bf44cPeNPjg&amp;t=782s">13:02</a>) Why larger PRs may be problematic</p><p>(<a href="https://www.youtube.com/watch?v=Bf44cPeNPjg&amp;t=1101s">18:21</a>) The growing gap between code maintainability and change confidence</p><p>(<a href="https://www.youtube.com/watch?v=Bf44cPeNPjg&amp;t=1308s">21:48</a>) How perceived code quality varies by organization size</p><p>(<a href="https://www.youtube.com/watch?v=Bf44cPeNPjg&amp;t=1429s">23:49</a>) The growing volatility in change failure rates</p><p>(<a href="https://www.youtube.com/watch?v=Bf44cPeNPjg&amp;t=1687s">28:07</a>) What the Developer Experience Index reveals</p><p>(<a href="https://www.youtube.com/watch?v=Bf44cPeNPjg&amp;t=1940s">32:20</a>) Cost, dev ramp-up, and innovation ratio</p><p>(<a href="https://www.youtube.com/watch?v=Bf44cPeNPjg&amp;t=2138s">35:38</a>) Where AI time savings are getting lost</p><p>(<a href="https://www.youtube.com/watch?v=Bf44cPeNPjg&amp;t=2231s">37:11</a>) Questions and wrap-up</p><p><strong><span>Where to find Justin Reock:</span></strong></p><p><span>&#8226; LinkedIn: </span><a href="https://www.linkedin.com/in/justinreock"><span>https://www.linkedin.com/in/justinreock</span></a></p><p><strong><span>Where to find Brian Houck:</span></strong></p><p><span>&#8226; LinkedIn: </span><a href="https://www.linkedin.com/in/brianhouck"><span>https://www.linkedin.com/in/brianhouck</span></a></p><h2><strong>Referenced:</strong></h2><p><span>&#8226; </span><a href="https://getdx.com/corefour"><span>DX Core 4 Productivity Framework</span></a></p><p><span>&#8226; </span><a href="https://getdx.com/report/State-of-AI-Impact-in-Engineering-Q2-Report/"><span>AI Impact report</span></a></p><p><span>&#8226; </span><a href="https://newsletter.getdx.com/p/the-ai-native-developer"><span>The AI-native developer - by Brian Houck</span></a></p><p><span>&#8226; </span><a href="https://arxiv.org/pdf/2606.00438"><span>GitHub Copilot and Developer Productivity: An Observational Dose-Response Analysis</span></a></p><p><span>&#8226; </span><a href="https://www.nber.org/papers/w35275"><span>Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools | NBER</span></a></p><p><span>&#8226; </span><a href="https://annievella.com/posts/the-productivity-experience-paradox/"><span>The Productivity-Experience Paradox - Annie Vella</span></a></p><p><span>&#8226; </span><a href="https://arxiv.org/abs/2605.04259"><span>EngThrive: Make It Fast and Easy to Do Great Work</span></a></p><p><span>&#8226; </span><a href="https://newsletter.getdx.com/p/the-ai-efficiency-plateau"><span>The AI efficiency plateau - by Brian Houck</span></a></p><p><span>&#8226; </span><a href="https://martinfowler.com/bliki/TradableQualityHypothesis.html"><span>Tradable Quality Hypothesis</span></a></p>]]></content:encoded></item><item><title><![CDATA[How Microsoft sees engineering bottlenecks changing with AI]]></title><description><![CDATA[Tim Bozarth, Microsoft CoreAI CVP, explains how AI is changing engineering productivity, why outcomes matter more than output, and what engineering leaders should measure instead.]]></description><link>https://newsletter.getdx.com/p/how-microsoft-sees-engineering-bottlenecks</link><guid isPermaLink="false">https://newsletter.getdx.com/p/how-microsoft-sees-engineering-bottlenecks</guid><dc:creator><![CDATA[Brian Houck]]></dc:creator><pubDate>Fri, 07 Aug 2026 15:47:36 GMT</pubDate><enclosure url="https://api.substack.com/feed/podcast/209296274/1135b2a44ce3e5af0cd78de10f17d202.mp3" length="0" type="audio/mpeg"/><content:encoded><![CDATA[<p>Listen and watch now on <strong><a href="https://youtu.be/fdofQkhpgDE">YouTube</a>, <a href="https://podcasts.apple.com/us/podcast/engineering-enablement-by-abi-noda/id1619140476">Apple</a>, and <a href="https://open.spotify.com/show/3NxjyIsuxeDMQtisDqBy7D">Spotify</a></strong>.</p><p><span>In this episode of Engineering Enablement, I sit down with Tim Bozarth, Corporate Vice President in Microsoft CoreAI and leader of Microsoft&#8217;s Engineering Thrive initiative. We discuss Engineering Thrive, Microsoft&#8217;s framework for measuring and improving engineering productivity, and why AI makes outcome-based metrics more important than ever.</span></p><p><span>Tim shares how AI is reshaping the software development lifecycle, where new bottlenecks are emerging, and why verification and confidence may become more valuable than code generation itself. We also explore why the purpose of engineering remains the same despite rising levels of abstraction, the skills that remain durable in an AI-driven world, and why engineering leaders should focus on outcomes rather than activity metrics.</span></p><div id="youtube2-fdofQkhpgDE" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;fdofQkhpgDE&quot;,&quot;startTime&quot;:null,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/fdofQkhpgDE?rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><h2><strong>Some takeaways: </strong></h2><h3><strong><span>Engineering Thrive gives Microsoft a common language for improving productivity</span></strong></h3><ul><li><p><strong><span>Engineering Thrive defines productivity through speed, ease, and quality.</span></strong><span> Its goal is to make it fast and easy to do great work by identifying friction across the developer experience rather than optimizing isolated activities.</span></p></li><li><p><strong><span>Speed, ease, and quality should not be treated as opposing goals.</span></strong><span> Engineering Thrive applies a &#8220;do no harm&#8221; principle: an improvement in one dimension should not be celebrated if it makes another materially worse.</span></p></li><li><p><strong><span>The framework helps Microsoft identify bottlenecks and invest where they will have the greatest impact.</span></strong><span> Tim argues that developer time is one of the company&#8217;s most valuable resources, making productivity improvements a strategic investment.</span></p></li></ul><h3><strong><span>AI makes outcome-based metrics more important, not obsolete</span></strong></h3><ul><li><p><strong><span>The industry is returning to activity metrics it rejected years ago.</span></strong><span> Tim sees renewed attention to lines of code, pull request counts, and similar measures as a search for easy answers to how AI is affecting productivity.</span></p></li><li><p><strong><span>More engineering activity does not necessarily mean more value.</span></strong><span> Engineering Thrive instead measures outcomes such as product quality, end-to-end speed, and the amount of time engineers can devote to innovation.</span></p></li><li><p><strong><span>A web of outcome metrics is harder to game than any individual measure.</span></strong><span> Looking across speed, ease, quality, and innovation time creates a more durable picture of whether an organization is actually improving.</span></p></li></ul><h3><strong><span>Planning and validation are becoming the new bottlenecks</span></strong></h3><ul><li><p><strong><span>Before AI, most engineering time was spent creating and operating software.</span></strong><span> Tim estimates that those phases historically accounted for more than 90% of engineering time, with operations alone consuming roughly 70% to 80%.</span></p></li><li><p><strong><span>On frontier teams, the bottlenecks have already shifted to planning and validation.</span></strong><span> Code creation is taking dramatically less time, while deciding what to build and determining whether the result is trustworthy are consuming a greater share of the work.</span></p></li><li><p><strong><span>The create phase may continue to shrink as models and agent harnesses improve.</span></strong><span> Tim expects planning and validation to remain durable constraints over the next several years, even as other parts of the software development lifecycle become increasingly automated.</span></p></li></ul><h3><strong><span>The code review bottleneck is really a confidence bottleneck</span></strong></h3><ul><li><p><strong><span>Higher PR throughput has increased the amount of change humans must evaluate.</span></strong><span> Enterprise software still requires a level of trust that teams cannot achieve by simply accepting AI-generated code without review.</span></p></li><li><p><strong><span>Code review is only one way to establish confidence.</span></strong><span> Testing, continuous rollouts, feature flags, canaries, and deployment practices all contribute signals about whether a change is reliable.</span></p></li><li><p><strong><span>The next generation of verification should help humans ask higher-level questions.</span></strong><span> Rather than inspecting every branch or class, engineers should be able to assess the scope, complexity, impact, and fundamental purpose of a change.</span></p></li></ul><h3><strong><span>More abstraction does not change the purpose of engineering</span></strong></h3><ul><li><p><strong><span>AI will reduce the time engineers spend on low-level implementation details.</span></strong><span> Tim sees that as another step in the long history of abstractions that allow engineers to spend more time describing system behavior and intended outcomes.</span></p></li><li><p><strong><span>Great engineers have never been defined by their ability to write a line of code.</span></strong><span> Their value comes from systems thinking, understanding objectives, and expressing functional and nonfunctional requirements coherently.</span></p></li><li><p><strong><span>Producing more software increases the need for strong engineering judgment.</span></strong><span> The faster organizations can build, the more important it becomes to ensure that their systems remain coherent, valid, and reliable.</span></p></li></ul><h3><strong><span>A maker&#8217;s mindset and ability to experiment effectively remain durable advantages</span></strong></h3><ul><li><p><strong><span>The maker&#8217;s mindset is relentlessly oriented toward producing a valuable final outcome.</span></strong><span> It combines a clear vision of what should be built with the ability to understand the needs of the customer using it.</span></p></li><li><p><strong><span>Engineers will need to continuously experiment and evaluate changes in outcomes.</span></strong><span> Tim argues that this discipline is no longer limited to model trainers; everyone using AI tools must determine whether a new approach actually improves the result.</span></p></li><li><p><strong><span>Using AI is not itself the goal.</span></strong><span> The goal is to accomplish more, and sometimes the best way to do that will be not to use AI at all.</span></p></li></ul><h3><strong><span>Engineering leaders should measure idea-to-value, not PR velocity</span></strong></h3><ul><li><p><strong><span>PR velocity says little about the success of an engineering organization.</span></strong><span> Tim urges leaders to stop evaluating teams through activity measures the industry already recognized as inadequate years ago.</span></p></li><li><p><strong><span>Idea-to-value connects engineering work to meaningful outcomes.</span></strong><span> Leaders should also examine how much time engineers spend innovating compared with keeping the lights on or handling corporate overhead.</span></p></li><li><p><strong><span>Product quality and business results should remain the ultimate measures of success.</span></strong><span> Focusing on outcomes can improve developer happiness, productivity, margins, and the value delivered by the organization.</span></p></li></ul><h2><strong>In this episode, we cover:</strong></h2><p>(<a href="https://www.youtube.com/watch?v=fdofQkhpgDE">00:00</a>) Intro</p><p>(<a href="https://www.youtube.com/watch?v=fdofQkhpgDE&amp;t=133s">02:13</a>) What Engineering Thrive is and the problem it solves</p><p>(<a href="https://www.youtube.com/watch?v=fdofQkhpgDE&amp;t=286s">04:46</a>) Why Engineering Thrive isn&#8217;t specific to Microsoft</p><p>(<a href="https://www.youtube.com/watch?v=fdofQkhpgDE&amp;t=550s">09:10</a>) The impact of Engineering Thrive at Microsoft</p><p>(<a href="https://www.youtube.com/watch?v=fdofQkhpgDE&amp;t=871s">14:31</a>) Why AI makes outcome-based productivity metrics more important</p><p>(<a href="https://www.youtube.com/watch?v=fdofQkhpgDE&amp;t=1102s">18:22</a>) Where AI is creating new bottlenecks in the SDLC</p><p>(<a href="https://www.youtube.com/watch?v=fdofQkhpgDE&amp;t=1477s">24:37</a>) Why more abstraction doesn&#8217;t change the purpose of engineering</p><p>(<a href="https://www.youtube.com/watch?v=fdofQkhpgDE&amp;t=1645s">27:25</a>) The durable skills of good engineers</p><p>(<a href="https://www.youtube.com/watch?v=fdofQkhpgDE&amp;t=1983s">33:03</a>) The changing economics of software development</p><p>(<a href="https://www.youtube.com/watch?v=fdofQkhpgDE&amp;t=2216s">36:56</a>) Advice for leaders: measure outcomes, not activity</p><p><strong><span>Where to find Tim Bozarth:</span></strong></p><p><span>&#8226; LinkedIn: </span><a href="http://linkedin.com/in/tbozarth"><span>linkedin.com/in/tbozarth</span></a></p><p><strong><span>Where to find Brian Houck:</span></strong></p><p><span>&#8226; LinkedIn: </span><a href="https://www.linkedin.com/in/brianhouck"><span>https://www.linkedin.com/in/brianhouck</span></a></p><h2><strong>Referenced:</strong></h2><p><span>&#8226; </span><a href="https://getdx.com/corefour"><span>DX Core 4 Productivity Framework</span></a></p><p><span>&#8226; </span><a href="https://spawn-queue.acm.org/doi/10.1145/3819080"><span>EngThrive: Make It Fast and Easy to Do Great Work: Building a durable model for outcome-oriented engineering measurement</span></a></p><p><span>&#8226; </span><a href="https://www.goodreads.com/quotes/10278659-tell-me-how-you-measure-me-and-i-ll-tell-you"><span>Quote by Eliyahu M. Goldratt: &#8220;Tell me how you measure me and...&#8221;</span></a></p><p><span>&#8226; </span><a href="https://en.wikipedia.org/wiki/Jevons_paradox"><span>Jevons paradox - Wikipedia</span></a></p>]]></content:encoded></item><item><title><![CDATA[Research briefing with Brian Houck: Measuring AI agents and revisiting the Core 4]]></title><description><![CDATA[Justin Reock and Brian Houck explore how AI coding agents are reshaping engineering metrics and what leaders need to measure in the age of AI.]]></description><link>https://newsletter.getdx.com/p/research-briefing-with-brian-houck</link><guid isPermaLink="false">https://newsletter.getdx.com/p/research-briefing-with-brian-houck</guid><dc:creator><![CDATA[Justin Reock]]></dc:creator><pubDate>Fri, 24 Jul 2026 13:50:33 GMT</pubDate><enclosure url="https://api.substack.com/feed/podcast/207581348/14162e401718c8fcc01f420bcb2cbc98.mp3" length="0" type="audio/mpeg"/><content:encoded><![CDATA[<p>Listen and watch now on <strong><a href="https://youtu.be/U3p-rqOrAts">YouTube</a>, <a href="https://podcasts.apple.com/us/podcast/engineering-enablement-by-abi-noda/id1619140476">Apple</a>, and <a href="https://open.spotify.com/show/3NxjyIsuxeDMQtisDqBy7D">Spotify</a></strong>.</p><p><span>AI coding agents are changing how software gets built, but they&#8217;re also forcing us to rethink how we measure engineering effectiveness. Traditional developer experience metrics were designed for humans, not AI agents, so how should engineering leaders adapt?</span></p><p><span>In this webinar, I&#8217;m joined by Brian Houck, Distinguished Scientist at DX and co-author of the SPACE framework, to explore the emerging field of agent experience and how it builds on developer experience rather than replacing it. We discuss how organizations can prepare for AI-assisted software development, how the DX Core 4 applies in the age of AI, why metrics like token usage and PR throughput don&#8217;t tell the whole story, and the growing importance of documentation. We also examine the impact AI-driven pressure is having on burnout and cognitive overload.</span></p><p><span>Throughout the conversation, we share practical guidance for building engineering organizations where both developers and AI agents can do their best work.</span></p><div id="youtube2-U3p-rqOrAts" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;U3p-rqOrAts&quot;,&quot;startTime&quot;:null,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/U3p-rqOrAts?rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><h2><strong>Some takeaways: </strong></h2><h2><strong><span>Agent experience builds on developer experience</span></strong></h2><ul><li><p><strong><span>Agent experience focuses on creating the conditions for AI agents to succeed.</span></strong><span> Brian defines agent experience as the environment surrounding AI agents, including the quality of context, documentation, validation, and feedback they receive. As agents become part of software teams, improving those conditions becomes increasingly important.</span></p></li><li><p><strong><span>Model quality is only one part of successful AI adoption.</span></strong><span> Organizations often focus on choosing the best model, but Brian argues that context, clear intent, and effective workflows often have a greater impact on outcomes than incremental improvements in model capability.</span></p></li><li><p><strong><span>The same systems that help developers often help AI agents.</span></strong><span> Investments in documentation, development workflows, and engineering platforms create a stronger foundation for both humans and AI to produce high-quality work.</span></p></li></ul><h2><strong><span>Developer experience and agent experience don&#8217;t always align</span></strong></h2><ul><li><p><strong><span>Many improvements benefit both developers and AI agents.</span></strong><span> Better documentation, clearer context, and stronger engineering practices improve outcomes across the board, making existing developer experience investments even more valuable.</span></p></li><li><p><strong><span>Optimizing for one doesn&#8217;t automatically optimize for the other.</span></strong><span> Brian explains that organizations will increasingly encounter situations where workflows that help AI agents introduce friction for developers, or vice versa.</span></p></li><li><p><strong><span>Organizations should measure both independently.</span></strong><span> Rather than assuming every AI optimization improves the developer experience, engineering leaders should evaluate where the two reinforce each other and where they diverge.</span></p></li></ul><h2><strong><span>Preparing for AI requires organizational change</span></strong></h2><ul><li><p><strong><span>Successful AI adoption requires more than coding tools.</span></strong><span> Justin and Brian describe AI readiness as a combination of developer tooling, engineering platforms, and organizational practices rather than a single technology decision.</span></p></li><li><p><strong><span>The DX Core 4 still provides a useful foundation.</span></strong><span> Instead of abandoning existing engineering metrics, organizations should reinterpret them for AI-assisted development while continuing to focus on business outcomes rather than activity.</span></p></li><li><p><strong><span>Validation becomes more important as generation becomes easier.</span></strong><span> As AI produces more code, engineering organizations need stronger review, testing, and verification processes to ensure quality keeps pace with productivity.</span></p></li></ul><h2><strong><span>Documentation becomes infrastructure for AI agents</span></strong></h2><ul><li><p><strong><span>Documentation is no longer just for people.</span></strong><span> AI agents rely on high-quality documentation to understand systems, follow conventions, and complete work accurately, making documentation a core engineering asset rather than an afterthought.</span></p></li><li><p><strong><span>Not all documentation delivers equal value.</span></strong><span> Brian highlights that the biggest returns come from documenting information that helps agents understand systems, architecture, and engineering intent rather than simply producing more documentation.</span></p></li><li><p><strong><span>Capturing organizational knowledge improves both human and AI performance.</span></strong><span> Teams that make important context explicit reduce repeated questions, improve onboarding, and enable AI agents to work more effectively.</span></p></li></ul><h2><strong><span>Engineering metrics need to evolve with AI</span></strong></h2><ul><li><p><strong><span>Token usage is a cost metric, not a productivity metric.</span></strong><span> Brian cautions against treating token consumption as a measure of engineering effectiveness because it reflects AI usage rather than business value or software quality.</span></p></li><li><p><strong><span>PR throughput tells only part of the story.</span></strong><span> Larger pull requests and faster code generation may indicate increased AI adoption, but they can also increase review complexity and cognitive load if organizations measure throughput in isolation.</span></p></li><li><p><strong><span>Outcome metrics matter more than activity metrics.</span></strong><span> Justin emphasizes measuring whether engineering teams deliver value, improve quality, and create better developer experiences instead of rewarding raw AI utilization.</span></p></li></ul><h2><strong><span>AI changes how engineering work feels&#8212;not just how it&#8217;s done</span></strong></h2><ul><li><p><strong><span>AI pressure is contributing to burnout and cognitive overload.</span></strong><span> Brian describes growing pressure to move faster with AI while simultaneously reviewing larger code changes and maintaining confidence in increasingly AI-generated systems.</span></p></li><li><p><strong><span>Software engineering is much more than writing code.</span></strong><span> Even as AI accelerates code generation, engineers remain responsible for judgment, communication, system design, validation, and building trust in what gets shipped.</span></p></li><li><p><strong><span>The long-term challenge is balancing speed with confidence.</span></strong><span> Organizations that move faster than their ability to verify AI-generated work risk increasing technical debt, developer stress, and uncertainty rather than creating sustainable productivity gains.</span></p></li></ul><h2><strong>In this episode, we cover:</strong></h2><p>(<a href="https://www.youtube.com/watch?v=U3p-rqOrAts">00:00</a>) Intro</p><p>(<a href="https://www.youtube.com/watch?v=U3p-rqOrAts&amp;t=86s">01:26</a>) Justin&#8217;s new role at DX</p><p>(<a href="https://www.youtube.com/watch?v=U3p-rqOrAts&amp;t=233s">03:53</a>) What agent experience is and why engineering leaders should care</p><p>(<a href="https://www.youtube.com/watch?v=U3p-rqOrAts&amp;t=535s">08:55</a>) How to improve agent experience at the platform level</p><p>(<a href="https://www.youtube.com/watch?v=U3p-rqOrAts&amp;t=685s">11:25</a>) How agent experience and developer experience influence each other</p><p>(<a href="https://www.youtube.com/watch?v=U3p-rqOrAts&amp;t=881s">14:41</a>) Preparing engineering teams for agentic work</p><p>(<a href="https://www.youtube.com/watch?v=U3p-rqOrAts&amp;t=1298s">21:38</a>) Why the DX Core 4 still matters in the age of AI</p><p>(<a href="https://www.youtube.com/watch?v=U3p-rqOrAts&amp;t=1643s">27:23</a>) What PR throughput actually measures</p><p>(<a href="https://www.youtube.com/watch?v=U3p-rqOrAts&amp;t=1967s">32:47</a>) The limits of token metrics</p><p>(<a href="https://www.youtube.com/watch?v=U3p-rqOrAts&amp;t=2230s">37:10</a>) What the data shows about documentation and developer experience</p><p>(<a href="https://www.youtube.com/watch?v=U3p-rqOrAts&amp;t=2372s">39:32</a>) Improving documentation for AI agents</p><p>(<a href="https://www.youtube.com/watch?v=U3p-rqOrAts&amp;t=2443s">40:43</a>) AI-washing, burnout, and cognitive overload</p><p>(<a href="https://www.youtube.com/watch?v=U3p-rqOrAts&amp;t=2735s">45:35</a>) Brian&#8217;s upcoming research on agent experience</p><p><strong><span>Where to find Brian Houck:</span></strong></p><p><span>&#8226; LinkedIn: </span><a href="https://www.linkedin.com/in/brianhouck"><span>https://www.linkedin.com/in/brianhouck</span></a></p><p><strong><span>Where to find Justin Reock:</span></strong></p><p><span>&#8226; LinkedIn: </span><a href="https://www.linkedin.com/in/justinreock"><span>https://www.linkedin.com/in/justinreock</span></a></p><h2><strong>Referenced:</strong></h2><p><span>&#8226; </span><a href="https://getdx.com/corefour"><span>DX Core 4 Productivity Framework</span></a></p><p><span>&#8226; </span><a href="https://spawn-queue.acm.org/doi/10.1145/3807964"><span>The SPACE of AI | Queue</span></a></p><p><span>&#8226; </span><a href="https://www.linkedin.com/in/sarachizari/"><span>Sara Chizari on LinkedIn</span></a></p><p><span>&#8226; </span><a href="https://annievella.com/posts/the-middle-loop/"><span>The Middle Loop - Annie Vella</span></a></p><p><span>&#8226; </span><a href="https://github.com/gastownhall/gastown"><span>gastownhall/gastown: Gas Town - multi-agent workspace manager &#183; GitHub</span></a></p><p><span>&#8226; </span><a href="https://getdx.com/guide/dora-space-devex/"><span>DORA, SPACE, and DevEx: Which framework should you use?</span></a></p><p><span>&#8226; </span><a href="https://getdx.com/blog/ai-impact-report-q1-2026/"><span>AI Impact Report: Q1 2026</span></a></p><p><span>&#8226; </span><a href="https://newsletter.getdx.com/p/ai-authored-code-has-nearly-doubled"><span>AI-authored code has nearly doubled, but so has PR size</span></a></p><p><span>&#8226; </span><a href="https://spawn-queue.acm.org/doi/10.1145/3819080"><span>EngThrive: Make It Fast and Easy to Do Great Work: Building a durable model for outcome-oriented engineering measurement</span></a></p><p><span>&#8226; </span><a href="https://psychsafety.com/googles-project-aristotle/"><span>Google&#8217;s Project Aristotle - Psychological Safety</span></a></p><p><span>&#8226; </span><a href="https://zapier.com"><span>Zapier</span></a></p>]]></content:encoded></item><item><title><![CDATA[Adopting the product operating model at Priceline]]></title><description><![CDATA[How Priceline used developer experience metrics, organizational change, and a product operating model to improve engineering effectiveness and prepare for AI-driven software development.]]></description><link>https://newsletter.getdx.com/p/adopting-the-product-operating-model</link><guid isPermaLink="false">https://newsletter.getdx.com/p/adopting-the-product-operating-model</guid><dc:creator><![CDATA[Justin Reock]]></dc:creator><pubDate>Fri, 10 Jul 2026 15:38:54 GMT</pubDate><enclosure url="https://api.substack.com/feed/podcast/205653265/2adc0bae43ca43be00e7327ced84f67e.mp3" length="0" type="audio/mpeg"/><content:encoded><![CDATA[<p>Listen and watch now on <strong><a href="https://youtu.be/c-O1wrEjx6w">YouTube</a>, <a href="https://podcasts.apple.com/us/podcast/engineering-enablement-by-abi-noda/id1619140476">Apple</a>, and <a href="https://open.spotify.com/show/3NxjyIsuxeDMQtisDqBy7D">Spotify</a></strong>.</p><p><span>In this episode of the Engineering Enablement podcast, I sit down with Sejal Amin, Chief Technology Officer at Priceline, and Pedro Gutierrez, Senior Director of Software Engineering, to discuss how Priceline adopted a product operating model and the role developer experience played in making that transformation successful.</span></p><p><span>We explore why the company moved away from a project-based approach, how DX metrics and developer feedback helped uncover organizational bottlenecks, and why a phased rollout, clear communication, and empowered engineering managers were critical to building trust and improving developer experience. We also discuss creating a dedicated developer experience team, lessons learned throughout the transformation, and how Priceline&#8217;s product operating model has helped the organization adapt to AI-driven software development.</span></p><div id="youtube2-c-O1wrEjx6w" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;c-O1wrEjx6w&quot;,&quot;startTime&quot;:null,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/c-O1wrEjx6w?rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><h2><strong>Some takeaways: </strong></h2><p><strong><span>Developer experience as a driver of organizational change</span></strong></p><ul><li><p><strong><span>Developer experience data can reveal organizational problems that traditional engineering metrics miss.</span></strong><span> At Priceline, DX signals uncovered organizational bottlenecks&#8212;including handoffs, dependencies, and team friction&#8212;that ultimately led the company to adopt a product operating model.</span></p></li><li><p><strong><span>Developer experience should be treated as a strategic capability, not just an engineering metric.</span></strong><span> Rather than measuring developer satisfaction in isolation, Priceline used DX insights to guide structural changes that improved autonomy, delivery, and engineering culture.</span></p></li></ul><p><strong><span>Adopting a product operating model</span></strong></p><ul><li><p><strong><span>Reducing dependencies gives teams greater ownership.</span></strong><span> Priceline shifted from a project-based organization to cross-functional product teams, reducing handoffs and giving teams the people and capabilities needed to own outcomes end to end.</span></p></li><li><p><strong><span>Autonomy requires visibility into team health.</span></strong><span> DX metrics gave engineering managers a clear view of the obstacles affecting their teams, allowing them to improve local workflows while staying aligned with broader organizational goals.</span></p></li></ul><p><strong><span>Turning developer feedback into action</span></strong></p><ul><li><p><strong><span>Developer experience surveys should lead to action&#8212;not just measurement.</span></strong><span> Managers reviewed survey results, completed a triage process, created quarterly action plans, and measured whether those improvements had an impact in the next survey cycle.</span></p></li><li><p><strong><span>Small workflow improvements can have an outsized impact.</span></strong><span> DX data helped teams reclaim focus time, identify tooling regressions after migrations, surface cross-team dependencies, and address day-to-day friction before it became systemic.</span></p></li></ul><p><strong><span>Building trust in developer experience metrics</span></strong></p><ul><li><p><strong><span>Clear communication is essential for adoption.</span></strong><span> Leaders consistently reinforced that DX metrics existed to improve teams rather than evaluate individuals, helping build confidence in the process from the outset.</span></p></li><li><p><strong><span>Trust grows when developers see meaningful change.</span></strong><span> Acting on feedback quarter after quarter encouraged greater participation, strengthened psychological safety, and made developer experience part of the organization&#8217;s culture.</span></p></li></ul><p><strong><span>The evolving role of engineering managers</span></strong></p><ul><li><p><strong><span>Engineering managers became owners of developer experience.</span></strong><span> Managers were expected to understand DX data, improve their team&#8217;s DXI each quarter, and make developer experience part of their regular operating rhythm.</span></p></li><li><p><strong><span>Developer experience became part of everyday engineering leadership.</span></strong><span> DX metrics were discussed openly in all-hands meetings and other forums, making developer experience a visible measure of organizational health rather than a one-time initiative.</span></p></li></ul><p><strong><span>Preparing engineering organizations for AI</span></strong></p><ul><li><p><strong><span>AI changes where bottlenecks occur&#8212;not whether they exist.</span></strong><span> As AI accelerated code generation, Priceline used its product operating model and developer experience data to identify where constraints had shifted and respond accordingly.</span></p></li><li><p><strong><span>A strong operating model helps organizations adapt to AI.</span></strong><span> Autonomous teams, continuous measurement, and visibility into developer workflows allowed Priceline to embrace AI while continuing to improve flow across the software development lifecycle.</span></p></li></ul><h2><strong>In this episode, we cover:</strong></h2><p>(<a href="https://www.youtube.com/watch?v=c-O1wrEjx6w">00:00</a>) Intro</p><p>(<a href="https://www.youtube.com/watch?v=c-O1wrEjx6w&amp;t=67s">01:07</a>) Meet Sejal Amin and Pedro Gutierrez</p><p>(<a href="https://www.youtube.com/watch?v=c-O1wrEjx6w&amp;t=107s">01:47</a>) How Priceline&#8217;s developer experience journey began</p><p>(<a href="https://www.youtube.com/watch?v=c-O1wrEjx6w&amp;t=295s">04:55</a>) Lessons from Priceline&#8217;s first developer experience surveys</p><p>(<a href="https://www.youtube.com/watch?v=c-O1wrEjx6w&amp;t=415s">06:55</a>) How DX improved Priceline&#8217;s developer experience surveys</p><p>(<a href="https://www.youtube.com/watch?v=c-O1wrEjx6w&amp;t=587s">09:47</a>) Identifying the causes of organizational slowness</p><p>(<a href="https://www.youtube.com/watch?v=c-O1wrEjx6w&amp;t=753s">12:33</a>) How the product operating model changed the way Priceline works</p><p>(<a href="https://www.youtube.com/watch?v=c-O1wrEjx6w&amp;t=850s">14:10</a>) Priceline&#8217;s phased rollout with DX</p><p>(<a href="https://www.youtube.com/watch?v=c-O1wrEjx6w&amp;t=1094s">18:14</a>) How DX insights drove organizational changes</p><p>(<a href="https://www.youtube.com/watch?v=c-O1wrEjx6w&amp;t=1173s">19:33</a>) Why Priceline improved developer experience before org change was complete</p><p>(<a href="https://www.youtube.com/watch?v=c-O1wrEjx6w&amp;t=1338s">22:18</a>) How clear communication builds trust</p><p>(<a href="https://www.youtube.com/watch?v=c-O1wrEjx6w&amp;t=1465s">24:25</a>) Early results from Priceline&#8217;s Core Four</p><p>(<a href="https://www.youtube.com/watch?v=c-O1wrEjx6w&amp;t=1538s">25:38</a>) Creating a culture of continuous feedback to build trust</p><p>(<a href="https://www.youtube.com/watch?v=c-O1wrEjx6w&amp;t=1660s">27:40</a>) What has changed in the engineering manager role</p><p>(<a href="https://www.youtube.com/watch?v=c-O1wrEjx6w&amp;t=1810s">30:10</a>) Resources for learning about the product operating model</p><p>(<a href="https://www.youtube.com/watch?v=c-O1wrEjx6w&amp;t=1960s">32:40</a>) What Pedro learned from implementing DX</p><p>(<a href="https://www.youtube.com/watch?v=c-O1wrEjx6w&amp;t=2091s">34:51</a>) The developer experience team</p><p>(<a href="https://www.youtube.com/watch?v=c-O1wrEjx6w&amp;t=2159s">35:59</a>) How AI tools have impacted Priceline&#8217;s teams</p><p>(<a href="https://www.youtube.com/watch?v=c-O1wrEjx6w&amp;t=2240s">37:20</a>) How the product operating model supports AI-driven development</p><p>(<a href="https://www.youtube.com/watch?v=c-O1wrEjx6w&amp;t=2353s">39:13</a>) Final advice for engineering leaders</p><p><strong><span>Where to find Sejal Amin:</span></strong></p><p><span>&#8226; LinkedIn: </span><a href="https://www.linkedin.com/in/sejal-amin"><span>https://www.linkedin.com/in/sejal-amin</span></a></p><p><strong><span>Where to find Pedro Gutierrez:</span></strong></p><p><span>&#8226; LinkedIn: </span><a href="https://www.linkedin.com/in/pedro-gutierrez-b6605422"><span>https://www.linkedin.com/in/pedro-gutierrez-b6605422</span></a></p><p><strong><span>Where to find Justin Reock:</span></strong></p><p><span>&#8226; LinkedIn: </span><a href="https://www.linkedin.com/in/justinreock"><span>https://www.linkedin.com/in/justinreock</span></a></p><h2><strong>Referenced:</strong></h2><p><span>&#8226; </span><a href="https://getdx.com/research/measuring-developer-productivity-with-the-dx-core-4/"><span>Measuring developer productivity with the DX Core 4</span></a></p><p><span>&#8226; </span><a href="https://www.amazon.com/dp/1119697336?lv=shuf&amp;channelId=500&amp;plpRedirect=mhFallback"><span>Transformed: Moving to the Product Operating Model (Silicon Valley Product Group)</span></a></p><p><span>&#8226; </span><a href="https://teamtopologies.com/"><span>Team Topologies</span></a></p><p><span>&#8226; </span><a href="https://flowframework.org/"><span>Flow Framework</span></a></p><p><span>&#8226; </span><a href="https://www.amazon.com/dp/1942788398?lv=shuf&amp;channelId=500&amp;plpRedirect=mhFallback"><span>Project to Product: How to Survive and Thrive in the Age of Digital Disruption with the Flow Framework</span></a></p>]]></content:encoded></item><item><title><![CDATA[AI and engineering productivity: Debating the headlines]]></title><description><![CDATA[Listen now | Leaders from Etsy, Twilio, GitHub, Google, and Microsoft debate how AI is changing engineering productivity, technical debt, developer roles, and the future of software teams.]]></description><link>https://newsletter.getdx.com/p/ai-and-engineering-productivity-debating</link><guid isPermaLink="false">https://newsletter.getdx.com/p/ai-and-engineering-productivity-debating</guid><dc:creator><![CDATA[Justin Reock]]></dc:creator><pubDate>Mon, 29 Jun 2026 14:07:18 GMT</pubDate><enclosure url="https://api.substack.com/feed/podcast/203305239/a5a8cc5da73a044fc35877683971ba8c.mp3" length="0" type="audio/mpeg"/><content:encoded><![CDATA[<p>Listen and watch now on <strong><a href="https://youtu.be/BcnqmcgScgM">YouTube</a>, <a href="https://podcasts.apple.com/us/podcast/engineering-enablement-by-abi-noda/id1619140476">Apple</a>, and <a href="https://open.spotify.com/show/3NxjyIsuxeDMQtisDqBy7D">Spotify</a></strong>.</p><p><span>In this closing panel from DX Annual, Rafe Colburn, Chief Product and Technology Officer at Etsy; Jesse Adametz, Senior Director of Engineering, Platform Engineering at Twilio; Eirini Kalliamvakou, Research Advisor at GitHub; Collin Green, Senior Staff UX Researcher at Google; and Brian Houck, Senior Principal Applied Scientist at Microsoft debate some of the biggest questions surrounding AI and engineering productivity.</span></p><p><span>They discuss whether AI will reduce the need for engineers, how AI is affecting technical debt, the future role of software engineers in an agentic world, and whether organizations should mandate AI adoption. They also explore how bottlenecks are shifting across the software development lifecycle, the challenges facing junior engineers, and why learning, culture, and change management may ultimately matter more than the tools themselves.</span></p><div id="youtube2-BcnqmcgScgM" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;BcnqmcgScgM&quot;,&quot;startTime&quot;:null,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/BcnqmcgScgM?rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><h2><strong>Some takeaways: </strong></h2><p><strong><span>AI is changing software engineering, but not eliminating the need for engineers</span></strong></p><ul><li><p><strong><span>The panel largely rejected the idea that an AI-first SDLC means dramatically fewer engineers.</span></strong><span> As the cost of building software decreases, demand for software is likely to increase, creating new opportunities rather than eliminating the need for technical talent.</span></p></li><li><p><strong><span>Several panelists argued that the role of software engineers will evolve rather than disappear.</span></strong><span> The tasks that make up the job may change, but organizations will continue to need people who can solve problems, make decisions, and build systems.</span></p></li></ul><p><strong><span>Technical debt remains a tradeoff, not just an AI problem</span></strong></p><ul><li><p><strong><span>Panelists disagreed on whether AI is creating technical debt faster than it can remove it.</span></strong><span> Some argued that AI is accelerating both code generation and technical debt, while others believed the underlying business pressures that create technical debt remain largely unchanged.</span></p></li><li><p><strong><span>The discussion also introduced the idea of cognitive debt.</span></strong><span> As engineers rely more heavily on AI-generated code, understanding and maintaining systems may become more difficult even if development velocity increases.</span></p></li></ul><p><strong><span>The future engineer may work at a higher level of abstraction</span></strong></p><ul><li><p><strong><span>Several panelists predicted that engineers will spend less time writing code directly and more time defining intent, setting constraints, providing context, and validating results.</span></strong><span> Rather than replacing engineering work, AI may shift it to a different level of abstraction.</span></p></li><li><p><strong><span>The panel also pushed back on the idea that engineers will simply become managers of agents.</span></strong><span> Effective AI use still requires technical judgment, communication skills, and careful oversight.</span></p></li></ul><p><strong><span>Mandates rarely create meaningful AI adoption</span></strong></p><ul><li><p><strong><span>Most panelists opposed the idea that organizations should mandate AI usage.</span></strong><span> Instead, they emphasized enablement, reducing friction, and helping developers discover value through their own workflows.</span></p></li><li><p><strong><span>Usage metrics can easily become the wrong goal.</span></strong><span> The group cautioned against treating AI usage itself as a performance metric, arguing that outcomes matter more than activity.</span></p></li></ul><p><strong><span>Junior engineers remain essential to the future of the profession</span></strong></p><ul><li><p><strong><span>The panel strongly rejected the idea that organizations will no longer need junior engineers.</span></strong><span> Today&#8217;s junior engineers become tomorrow&#8217;s senior engineers, making talent development critical to the long-term health of the industry.</span></p></li><li><p><strong><span>Several speakers also noted that newer engineers may bring valuable AI-native perspectives.</span></strong><span> Just as previous technology shifts rewarded developers who grew up with new tools, the next generation may help shape how AI is used in practice.</span></p></li></ul><p><strong><span>The biggest AI adoption challenges are human, not technical</span></strong></p><ul><li><p><strong><span>While tooling matters, the panel repeatedly returned to learning, culture, incentives, and change management as the biggest barriers to successful AI adoption.</span></strong><span> Engineers are navigating rapid technological change, shifting workflows, and new expectations about their role.</span></p></li><li><p><strong><span>Organizations that create space for learning appear to see stronger results.</span></strong><span> The panel highlighted examples where teams learned together, experimented together, and achieved better adoption outcomes than individuals working in isolation.</span></p></li></ul><h2><strong>In this episode, we cover:</strong></h2><p>(<a href="https://www.youtube.com/watch?v=BcnqmcgScgM">00:00</a>) Intro</p><p>(<a href="https://www.youtube.com/watch?v=BcnqmcgScgM&amp;t=76s">01:16</a>) Why an AI-first SDLC doesn&#8217;t mean fewer engineers</p><p>(<a href="https://www.youtube.com/watch?v=BcnqmcgScgM&amp;t=189s">03:09</a>) The debate over AI and technical debt</p><p>(<a href="https://www.youtube.com/watch?v=BcnqmcgScgM&amp;t=460s">07:40</a>) AI-generated code and the future role of engineers</p><p>(<a href="https://www.youtube.com/watch?v=BcnqmcgScgM&amp;t=856s">14:16</a>) Why mandating AI use doesn&#8217;t necessarily lead to better outcomes</p><p>(<a href="https://www.youtube.com/watch?v=BcnqmcgScgM&amp;t=1243s">20:43</a>) Predictions for the future of junior engineers</p><p>(<a href="https://www.youtube.com/watch?v=BcnqmcgScgM&amp;t=1402s">23:22</a>) Where the bottlenecks are in the SDLC now</p><p>(<a href="https://www.youtube.com/watch?v=BcnqmcgScgM&amp;t=1705s">28:25</a>) How risk influences AI use</p><p>(<a href="https://www.youtube.com/watch?v=BcnqmcgScgM&amp;t=1958s">32:38</a>) Why the human side is the biggest AI adoption challenge</p><h2><strong>Referenced:</strong></h2><p><span>&#8226; </span><a href="https://www.etsy.com/"><span>Etsy</span></a></p><p><span>&#8226; </span><a href="https://github.com/"><span>GitHub</span></a></p><p><span>&#8226; </span><a href="https://www.microsoft.com/en-us"><span>Microsoft</span></a></p><p><span>&#8226; </span><a href="https://www.twilio.com/"><span>Twilio</span></a></p><p><span>&#8226; </span><a href="https://www.google.com/"><span>Google</span></a></p><p><span>&#8226; </span><a href="http://linkedin.com/in/stewartreichling"><span>Stewart Reichling</span></a></p><p><span>&#8226; </span><a href="https://getdx.com/blog/space-metrics/"><span>What is the SPACE framework and when should you use it?</span></a></p>]]></content:encoded></item><item><title><![CDATA[2x the power users: How structured AI training scaled developer productivity]]></title><description><![CDATA[How Indeed drove AI coding tool adoption from 25% to 97% across 2,000 engineers, and what it learned about training, enablement, and preparing for the next phase of AI-assisted development.]]></description><link>https://newsletter.getdx.com/p/2x-the-power-users-how-structured</link><guid isPermaLink="false">https://newsletter.getdx.com/p/2x-the-power-users-how-structured</guid><pubDate>Mon, 29 Jun 2026 14:04:35 GMT</pubDate><enclosure url="https://api.substack.com/feed/podcast/203305672/3f998975f9d9868650338b9a7537c14f.mp3" length="0" type="audio/mpeg"/><content:encoded><![CDATA[<p>Listen and watch now on <strong><a href="https://youtu.be/iomiGESWxMg">YouTube</a>, <a href="https://podcasts.apple.com/us/podcast/engineering-enablement-by-abi-noda/id1619140476">Apple</a>, and <a href="https://open.spotify.com/show/3NxjyIsuxeDMQtisDqBy7D">Spotify</a></strong>.</p><p><span>Indeed increased AI coding tool adoption from roughly 25% to 97% across its engineering organization, but getting engineers to use the tools was only part of the challenge.</span></p><p><span>In this session from DX Annual, Michael Redding, Principal Product Manager, and Jeff Davis, VP of Core Infrastructure at Indeed, explain how the company used structured training, leadership support, and ongoing community engagement to help more than 2,000 engineers build practical AI skills. They share why an early train-the-trainer model fell short, how they redesigned their approach around hands-on learning, and what they learned about balancing adoption, measurement, and psychological safety.</span></p><p><span>They also discuss the impact of the program on coding time, the role of continuous enablement after formal training ended, and how Indeed is preparing for the next phase of AI adoption, including agentic workflows and AI-powered coaching.</span></p><div id="youtube2-iomiGESWxMg" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;iomiGESWxMg&quot;,&quot;startTime&quot;:null,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/iomiGESWxMg?rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><h2><strong>Some takeaways: </strong></h2><p><strong><span>Indeed started with a productivity problem, not an AI problem</span></strong></p><ul><li><p><strong><span>At the beginning of 2025, Indeed&#8217;s DX survey showed that only about half of developer time was being spent on new features and innovation.</span></strong><span> The remaining 48% was consumed by maintenance, upgrades, incident response, and other forms of engineering overhead.</span></p></li><li><p><strong><span>The company&#8217;s AI strategy focused on two goals: reducing overhead work and increasing output during coding time.</span></strong><span> The long-term objective was to double engineering productivity by shrinking non-value-added work while helping engineers produce more during the time they spend building.</span></p></li></ul><p><strong><span>AI Coding Essentials succeeded where AI Coding Ambassadors fell short</span></strong></p><ul><li><p><strong><span>Indeed&#8217;s first enablement effort, AI Coding Ambassadors, used a train-the-trainer model built around roughly 60 AI champions across the organization.</span></strong><span> While ambassadors maintained high levels of engagement, adoption among their teammates declined after the program ended.</span></p></li><li><p><strong><span>The company responded by launching AI Coding Essentials (AICE), a structured training program designed for all engineers.</span></strong><span> The experience convinced the team that direct, hands-on learning was far more effective than relying on knowledge to spread organically through teams.</span></p></li></ul><p><strong><span>Indeed treated AI upskilling as a company-wide investment</span></strong></p><ul><li><p><strong><span>Training more than 2,000 engineers required significant organizational commitment and leadership support.</span></strong><span> Michael estimated the investment at roughly $3&#8211;4 million in engineering time across the company.</span></p></li><li><p><strong><span>Rather than mandating AI usage, Indeed strongly encouraged completion of the training itself.</span></strong><span> Managers were given visibility into participation, while engineers retained flexibility in how and whether they ultimately incorporated AI into their workflows.</span></p></li></ul><p><strong><span>AI adoption increased from 25% to 97%</span></strong></p><ul><li><p><strong><span>Despite offering AI tools, training resources, and executive support, weekly AI usage remained stuck around 25% at the start of 2025.</span></strong><span> The challenge was not tool access but helping engineers develop practical skills and confidence.</span></p></li><li><p><strong><span>By the time of the presentation, weekly AI tool usage had reached approximately 97%.</span></strong><span> The company also successfully navigated multiple tool transitions, moving from Cody and Copilot to newer agentic tools such as Claude Code, Cursor, Windsurf, and Amp.</span></p></li></ul><p><strong><span>Structured training produced measurable results</span></strong></p><ul><li><p><strong><span>Engineers who completed AI Coding Essentials reduced coding time by roughly 35&#8211;36%, while engineers who did not complete the training saw little change.</span></strong><span> Across the broader organization, coding time decreased by roughly 20%.</span></p></li><li><p><strong><span>Indeed measured coding time as the period between a developer picking up a Jira ticket and opening a diff in GitLab.</span></strong><span> The company continued to see benefits months after training ended, especially as newer frontier models became available.</span></p></li></ul><p><strong><span>Community and continuous enablement kept momentum going</span></strong></p><ul><li><p><strong><span>Indeed reinforced learning through coding forums, office hours, hackathons, Slack communities, and its AI Showcase recognition program.</span></strong><span> More than 100 unique community posts were being shared monthly in the company&#8217;s primary AI channel.</span></p></li><li><p><strong><span>The goal was to make AI learning continuous rather than event-based.</span></strong><span> Engineers had multiple ways to share discoveries, get help, and learn from peers long after formal training concluded.</span></p></li></ul><p><strong><span>The next challenge is moving beyond coding</span></strong></p><ul><li><p><strong><span>Indeed is now focused on agentic workflows, AI coaching, and expanding enablement beyond software engineering.</span></strong><span> Product managers, designers, researchers, and other R&amp;D functions are becoming part of the company&#8217;s AI adoption strategy.</span></p></li><li><p><strong><span>As coding becomes faster, bottlenecks are beginning to shift elsewhere in the development lifecycle.</span></strong><span> The team is already monitoring signs that code review and other downstream activities may become the next constraints on engineering throughput.</span></p></li></ul><h2><strong>In this episode, we cover:</strong></h2><p>(<a href="https://www.youtube.com/watch?v=iomiGESWxMg">00:00</a>) Intro</p><p>(<a href="https://www.youtube.com/watch?v=iomiGESWxMg&amp;t=65s">01:05</a>) Indeed&#8217;s DX survey from January 2025</p><p>(<a href="https://www.youtube.com/watch?v=iomiGESWxMg&amp;t=150s">02:30</a>) The two-part strategy to double engineering productivity</p><p>(<a href="https://www.youtube.com/watch?v=iomiGESWxMg&amp;t=261s">04:21</a>) How Indeed increased AI adoption from 25% to 97%</p><p>(<a href="https://www.youtube.com/watch?v=iomiGESWxMg&amp;t=940s">15:40</a>) Results from Indeed&#8217;s AI training program</p><p>(<a href="https://www.youtube.com/watch?v=iomiGESWxMg&amp;t=1113s">18:33</a>) How Indeed sustains AI adoption and learning</p><p>(<a href="https://www.youtube.com/watch?v=iomiGESWxMg&amp;t=1386s">23:06</a>) What&#8217;s next for AI enablement at Indeed</p><p>(<a href="https://www.youtube.com/watch?v=iomiGESWxMg&amp;t=1481s">24:41</a>) Q&amp;A: How coding time was calculated</p><p>(<a href="https://www.youtube.com/watch?v=iomiGESWxMg&amp;t=1525s">25:25</a>) Q&amp;A: How Indeed uses AI playbooks</p><p>(<a href="https://www.youtube.com/watch?v=iomiGESWxMg&amp;t=1600s">26:40</a>) Q&amp;A: Balancing asynchronous and live AI training</p><p>(<a href="https://www.youtube.com/watch?v=iomiGESWxMg&amp;t=1702s">28:22</a>) Q&amp;A: Psychological safety during AI adoption</p><p>(<a href="https://www.youtube.com/watch?v=iomiGESWxMg&amp;t=1904s">31:44</a>) Q&amp;A: Why AI adoption spikes after the holidays</p><p>(<a href="https://www.youtube.com/watch?v=iomiGESWxMg&amp;t=2000s">33:20</a>) Q&amp;A: The metrics Indeed tracked</p><p>(<a href="https://www.youtube.com/watch?v=iomiGESWxMg&amp;t=2122s">35:22</a>) Q&amp;A: Where the time savings are going</p><p>(<a href="https://www.youtube.com/watch?v=iomiGESWxMg&amp;t=2214s">36:54</a>) Q&amp;A: Reaching engineers who skipped the training</p><p>(<a href="https://www.youtube.com/watch?v=iomiGESWxMg&amp;t=2288s">38:08</a>) Closing thoughts</p><h2><strong>Referenced:</strong></h2><p><span>&#8226; </span><a href="https://www.indeed.com/"><span>Indeed</span></a></p><p><span>&#8226; </span><a href="https://www.anthropic.com/product/claude-code"><span>Claude Code | Anthropic&#8217;s agentic coding system</span></a></p><p><span>&#8226; </span><a href="https://cursor.com/"><span>Cursor</span></a></p><p><span>&#8226; </span><a href="https://www.windsurf.dev/"><span>Windsurf</span></a></p><p><span>&#8226; </span><a href="https://ampcode.com/"><span>Amp Code</span></a></p><p><span>&#8226; </span><a href="https://resources.anthropic.com/hubfs/The-Complete-Guide-to-Building-Skill-for-Claude.pdf"><span>The Complete Guide to Building Skills for Claude | Anthropic</span></a></p><p><span>&#8226; </span><a href="https://getdx.com/report/dx-core-4/"><span>Measuring developer productivity with the DX Core 4</span></a></p>]]></content:encoded></item><item><title><![CDATA[From PR throughput to product velocity: How Dropbox is rethinking productivity in the agentic era]]></title><description><![CDATA[How Dropbox is adapting its engineering systems, workflows, and metrics for the agentic era as AI shifts bottlenecks beyond code generation.]]></description><link>https://newsletter.getdx.com/p/from-pr-throughput-to-product-velocity</link><guid isPermaLink="false">https://newsletter.getdx.com/p/from-pr-throughput-to-product-velocity</guid><pubDate>Mon, 29 Jun 2026 13:59:38 GMT</pubDate><enclosure url="https://api.substack.com/feed/podcast/203305440/dc35434f3b2719fdc32ad6787e8d8f75.mp3" length="0" type="audio/mpeg"/><content:encoded><![CDATA[<p>Listen and watch now on <strong><a href="https://youtu.be/w0kHCjTOvyo">YouTube</a>, <a href="https://podcasts.apple.com/us/podcast/engineering-enablement-by-abi-noda/id1619140476">Apple</a>, and <a href="https://open.spotify.com/show/3NxjyIsuxeDMQtisDqBy7D">Spotify</a></strong>.</p><p><span>In this session from DX Annual, Uma Namasivayam, Senior Director of Engineering Productivity at Dropbox, shares how the company&#8217;s developer productivity efforts evolved from improving developer experience to preparing for the agentic era.</span></p><p><span>He explains how Dropbox approached AI adoption across its engineering organization, the impact it had on developer productivity, and why faster code generation is creating new bottlenecks in areas such as code review, validation, and CI/CD. He also discusses Dropbox&#8217;s efforts to rethink engineering systems, measurement, and workflows, including the development of agentic tooling and new metrics designed to move beyond PR throughput and toward product velocity.</span></p><div id="youtube2-w0kHCjTOvyo" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;w0kHCjTOvyo&quot;,&quot;startTime&quot;:null,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/w0kHCjTOvyo?rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><h2><strong>Some takeaways: </strong></h2><p><strong><span>Dropbox&#8217;s productivity journey started before AI</span></strong></p><ul><li><p><strong><span>DXI helped Dropbox identify productivity problems as system problems rather than talent problems.</span></strong><span> When the company began measuring developer experience in 2023, it found significant variation across teams in DXI scores, PR throughput, and cycle time.</span></p></li><li><p><strong><span>Measuring developer experience created a framework for prioritizing investments.</span></strong><span> The team used DXI to identify friction across areas such as debugging, documentation, and build systems while giving leadership a common language for discussing productivity.</span></p></li></ul><p><strong><span>AI adoption required more than access to tools</span></strong></p><ul><li><p><strong><span>Dropbox combined executive support, developer segmentation, enablement, and strong guardrails to drive adoption.</span></strong><span> Different teams and developer roles were matched with different tools and workflows based on their needs.</span></p></li><li><p><strong><span>The approach helped Dropbox increase AI adoption from roughly 30% to 100% within six months.</span></strong><span> During the same period, PR throughput doubled and developer satisfaction with AI tools increased significantly.</span></p></li></ul><p><strong><span>Engineers used their extra capacity to tackle neglected work</span></strong></p><ul><li><p><strong><span>As AI increased throughput, engineers naturally pulled maintenance work, migrations, and technical debt from the backlog.</span></strong><span> Dropbox saw significant growth in these categories without any specific direction from leadership.</span></p></li><li><p><strong><span>The additional capacity was often reinvested into engineering health.</span></strong><span> Teams used the opportunity to address long-standing issues that had accumulated over time rather than focusing exclusively on new feature development.</span></p></li></ul><p><strong><span>The next challenges are scale, trust, and measurement</span></strong></p><ul><li><p><strong><span>Dropbox believes the move to agentic engineering creates three major challenges: scale, validation and trust, and measurement.</span></strong><span> Existing development systems were not designed for a world where AI dramatically increases code throughput.</span></p></li><li><p><strong><span>As code generation accelerates, bottlenecks are shifting toward code review, validation, and CI/CD systems.</span></strong><span> The company is already seeing pressure move downstream in the software development lifecycle.</span></p></li></ul><p><strong><span>Agentic engineering requires redesigning the entire system</span></strong></p><ul><li><p><strong><span>Uma compared the transition to the shift from steam-powered factories to electric factories.</span></strong><span> The biggest gains came from redesigning the entire system rather than simply replacing one technology with another.</span></p></li><li><p><strong><span>Dropbox is investing in agentic workflows across the SDLC and building Nova as an orchestration layer.</span></strong><span> The company is evaluating roughly 30 development steps, and one in twelve pull requests is already being generated by Nova.</span></p></li></ul><p><strong><span>PR throughput is becoming a less useful measure of productivity</span></strong></p><ul><li><p><strong><span>Dropbox believes traditional engineering metrics need to evolve alongside AI.</span></strong><span> As agentic workflows become more common, measuring productivity through pull request volume alone provides an incomplete picture of engineering output.</span></p></li><li><p><strong><span>The company is increasingly focused on metrics such as AI contribution, loaded cost per PR, agentic workflow coverage, work distribution, and time to ship.</span></strong><span> The goal is to better connect engineering activity to customer value and business outcomes.</span></p></li></ul><h2><strong>In this episode, we cover:</strong></h2><p>(<a href="https://www.youtube.com/watch?v=w0kHCjTOvyo">00:00</a>) Intro</p><p>(<a href="https://www.youtube.com/watch?v=w0kHCjTOvyo&amp;t=57s">00:57</a>) The beginning of Dropbox&#8217;s DX journey</p><p>(<a href="https://www.youtube.com/watch?v=w0kHCjTOvyo&amp;t=154s">02:34</a>) AI adoption at Dropbox: what made it work</p><p>(<a href="https://www.youtube.com/watch?v=w0kHCjTOvyo&amp;t=286s">04:46</a>) The results of Dropbox&#8217;s AI adoption efforts</p><p>(<a href="https://www.youtube.com/watch?v=w0kHCjTOvyo&amp;t=339s">05:39</a>) What the results mean for the business</p><p>(<a href="https://www.youtube.com/watch?v=w0kHCjTOvyo&amp;t=415s">06:55</a>) The phases of AI adoption and where they are now</p><p>(<a href="https://www.youtube.com/watch?v=w0kHCjTOvyo&amp;t=480s">08:00</a>) The new bottlenecks</p><p>(<a href="https://www.youtube.com/watch?v=w0kHCjTOvyo&amp;t=556s">09:16</a>) Three challenges Dropbox faces moving into agentic engineering</p><p>(<a href="https://www.youtube.com/watch?v=w0kHCjTOvyo&amp;t=605s">10:05</a>) How Dropbox is redesigning the SDLC for agentic engineering</p><p>(<a href="https://www.youtube.com/watch?v=w0kHCjTOvyo&amp;t=946s">15:46</a>) The new metrics that matter</p><p>(<a href="https://www.youtube.com/watch?v=w0kHCjTOvyo&amp;t=1156s">19:16</a>) Final takeaways</p><h2><strong>Referenced:</strong></h2><p><span>&#8226; </span><a href="https://www.dropbox.com/"><span>Dropbox</span></a></p><p><span>&#8226; </span><a href="https://getdx.com/developer-experience-index/"><span>Developer Experience Index (DXI) | DX</span></a></p><p><span>&#8226; </span><a href="https://getdx.com/corefour"><span>DX Core 4 Productivity Framework</span></a></p><p><span>&#8226; </span><a href="https://cursor.com/"><span>Cursor</span></a></p><p><span>&#8226; </span><a href="https://www.anthropic.com/product/claude-code"><span>Claude Code | Anthropic&#8217;s agentic coding system</span></a></p><p><span>&#8226; </span><a href="https://www.jetbrains.com/"><span>JetBrains</span></a></p><p><span>&#8226; </span><a href="https://code.visualstudio.com/"><span>Visual Studio Code</span></a></p><p><span>&#8226; </span><a href="https://www.atlassian.com/software/jira"><span>Jira | Project Management for the AI Era | Atlassian</span></a></p><p><span>&#8226; </span><a href="https://github.com/"><span>GitHub</span></a></p>]]></content:encoded></item><item><title><![CDATA[Beyond the CLI: Agentic AI for async workloads and non-developers ]]></title><description><![CDATA[How Airbnb scaled AI adoption without mandates, why agentic AI is reshaping product development, and the infrastructure powering its vision for AI-first engineering.]]></description><link>https://newsletter.getdx.com/p/beyond-the-cli-agentic-ai-for-async</link><guid isPermaLink="false">https://newsletter.getdx.com/p/beyond-the-cli-agentic-ai-for-async</guid><dc:creator><![CDATA[Justin Reock]]></dc:creator><pubDate>Mon, 22 Jun 2026 13:46:55 GMT</pubDate><enclosure url="https://api.substack.com/feed/podcast/202175518/3f6d406e67f5579715913bd8049e8875.mp3" length="0" type="audio/mpeg"/><content:encoded><![CDATA[<p>Listen and watch now on <strong><a href="https://youtu.be/lL9-yATNAo0">YouTube</a>, <a href="https://podcasts.apple.com/us/podcast/engineering-enablement-by-abi-noda/id1619140476">Apple</a>, and <a href="https://open.spotify.com/show/3NxjyIsuxeDMQtisDqBy7D">Spotify</a></strong>.</p><p>In this session from DX Annual, Christopher Sanson, Product Lead, AI Developer Experience, and Madison Capps, Engineering Manager, Infrastructure at Airbnb, challenge some of the most common assumptions about AI. Is AI primarily about replacing humans? Do organizations need mandates to drive adoption? And are the productivity gains really as small as some studies suggest?</p><p>Using examples from Airbnb&#8217;s own AI journey, they share how the company achieved widespread adoption of agentic AI through AirChat, community enablement, and internal tooling rather than top-down mandates. They also discuss the impact AI is having on developer productivity, how non-developers are increasingly using coding tools, and how teams are rethinking product development in an AI-first world.</p><p>Finally, Madison takes a deeper look at the infrastructure powering Airbnb&#8217;s AI strategy, including AirChat CLI, the AirChat SDK, and AirChat Remote, along with the company&#8217;s vision for asynchronous agent workflows and the next generation of AI-powered development.</p><div id="youtube2-lL9-yATNAo0" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;lL9-yATNAo0&quot;,&quot;startTime&quot;:null,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/lL9-yATNAo0?rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><h2><strong>Some takeaways: </strong></h2><p><strong>AI adoption at scale</strong></p><ul><li><p><strong>Successful AI adoption does not require mandates.</strong> Airbnb achieved 97% weekly usage and 90% daily usage of agentic AI tools among engineers without tying adoption to performance reviews or quotas. Christopher argued that the best adoption comes when developers choose to use AI because it genuinely helps them work faster and better.</p></li><li><p><strong>Treat internal AI tools like products, not internal infrastructure.</strong> Airbnb built a recognizable brand around AirChat, invested in onboarding and workshops, created internal marketing materials, and focused heavily on user experience. That product mindset helped turn AirChat into a company-wide platform rather than just another engineering tool.</p></li><li><p><strong>Community-driven learning scales better than centralized training.</strong> AI champions, train-the-trainer programs, hackathons, workshops, and active peer-to-peer learning channels allowed knowledge to spread organically across the company. Over time, the AI community became larger and more active than the team managing the platform itself.</p></li></ul><p><strong>Productivity gains are accelerating</strong></p><ul><li><p><strong>Developers are spending more time actively coding.</strong> Christopher challenged the idea that engineers only spend a small percentage of their time writing code. As coding becomes faster and easier with agentic AI, developers can spend more of their week building software rather than working around implementation bottlenecks.</p></li><li><p><strong>The most active AI users see the largest productivity gains.</strong> Airbnb found that developers who spent four or more hours per day working with agentic AI dramatically increased their output. The relationship between AI usage and productivity became stronger as engineers learned how to incorporate agents into their daily workflows.</p></li><li><p><strong>PR throughput increased by 65% after the introduction of agentic AI.</strong> Airbnb&#8217;s data suggests that productivity gains extend well beyond the single-digit improvements often cited in industry studies. Developers who heavily embraced agentic AI moved from industry-average output to some of the highest throughput levels measured internally.</p></li><li><p><strong>AI-authored code is becoming mainstream.</strong> Roughly 59% of Airbnb&#8217;s code is now primarily authored by AI, and more than half of developers report that AI generates the majority of the code they work with. Christopher argued that this shift is happening far faster than most organizations realize.</p></li></ul><p><strong>AI is spreading beyond engineering</strong></p><ul><li><p><strong>The addressable market for AI is much larger than developers alone.</strong> Airbnb initially expected adoption to level off around its engineering population. Instead, usage continued growing as product managers, designers, finance teams, and operations teams began integrating agentic AI into their work.</p></li><li><p><strong>People will learn new workflows when the value is obvious.</strong> Some non-engineering teams adopted VS Code and terminal-based tools simply because they provided the best access to agentic AI capabilities. Rather than resisting technical tools, employees were willing to learn them in exchange for meaningful productivity gains.</p></li><li><p><strong>Domain experts are increasingly building their own AI-powered solutions.</strong> Airbnb&#8217;s internal platforms allow teams to create specialized applications tailored to their own workflows. This shifts more problem-solving into the hands of the people closest to the business problem.</p></li></ul><p><strong>Rethinking how work gets done</strong></p><ul><li><p><strong>Many existing processes were designed around expensive software development.</strong> Product reviews, lengthy requirements documents, and sequential handoffs evolved in a world where implementation was slow and costly. AI changes those economics and creates opportunities to redesign workflows from first principles.</p></li><li><p><strong>AI enables faster movement from ideas to prototypes.</strong> Rather than spending weeks refining specifications before building anything, teams can generate multiple prototypes quickly, test ideas earlier, and iterate before committing significant resources.</p></li><li><p><strong>Smaller teams can collaborate earlier and move faster.</strong> Airbnb sees opportunities to reduce handoffs between product managers, designers, and engineers by bringing teams together earlier in the process and using AI to accelerate exploration and execution.</p></li></ul><p><strong>Building for asynchronous AI</strong></p><ul><li><p><strong>Current agentic AI tooling still creates friction.</strong> Managing multiple sessions, handling long-running tasks, maintaining context, and switching between workflows remain cumbersome despite major advances in model capabilities.</p></li><li><p><strong>The next frontier is asynchronous agent workflows.</strong> Rather than interacting with a single agent in real time, developers are increasingly orchestrating multiple agents working in parallel, often across long-running tasks that continue without constant supervision.</p></li><li><p><strong>Airbnb is investing in infrastructure, not just models.</strong> AirChat CLI, migration tooling, the AirChat SDK, and AirChat Remote were all built around the belief that future gains will come from workflow orchestration, platform capabilities, and developer experience as much as from improvements in foundation models.</p></li></ul><p><strong>Preparing for an AI-first future</strong></p><ul><li><p><strong>Organizations should build for where developer workflows are heading.</strong> Madison described Airbnb&#8217;s approach as continuously forecasting how engineers are likely to work in the near future and investing in the infrastructure required to support those workflows before they become mainstream.</p></li><li><p><strong>AI-first architecture will become increasingly important.</strong> As throughput rises and more work is delegated to agents, teams will need stronger guardrails, scalable platforms, and systems designed specifically to support AI-assisted development.</p></li><li><p><strong>The biggest bottlenecks are shifting away from code generation.</strong> As AI reduces implementation costs, constraints move elsewhere in the system. Coordination, validation, infrastructure, and workflow management are becoming the new challenges organizations must solve.</p></li></ul><h2><strong>In this episode, we cover:</strong></h2><p>(<a href="https://www.youtube.com/watch?v=lL9-yATNAo0">00:00</a>) Intro</p><p>(<a href="https://www.youtube.com/watch?v=lL9-yATNAo0&amp;t=97s">01:37</a>) Myth #1: AI is about replacing humans</p><p>(<a href="https://www.youtube.com/watch?v=lL9-yATNAo0&amp;t=202s">03:22</a>) Myth #2: You need mandates to drive AI adoption</p><p>(<a href="https://www.youtube.com/watch?v=lL9-yATNAo0&amp;t=321s">05:21</a>) AirChat, agentic AI, and Airbnb&#8217;s adoption strategy</p><p>(<a href="https://www.youtube.com/watch?v=lL9-yATNAo0&amp;t=487s">08:07</a>) Myth #3: AI has little impact on productivity</p><p>(<a href="https://www.youtube.com/watch?v=lL9-yATNAo0&amp;t=573s">09:33</a>) Airbnb&#8217;s increase in coding time and PR throughput</p><p>(<a href="https://www.youtube.com/watch?v=lL9-yATNAo0&amp;t=860s">14:20</a>) Myth #4: AI coding tools are just for coders</p><p>(<a href="https://www.youtube.com/watch?v=lL9-yATNAo0&amp;t=939s">15:39</a>) How non-developers are using coding tools</p><p>(<a href="https://www.youtube.com/watch?v=lL9-yATNAo0&amp;t=1044s">17:24</a>) Rethinking product development in an AI-first world</p><p>(<a href="https://www.youtube.com/watch?v=lL9-yATNAo0&amp;t=1230s">20:30</a>) Myth #5: Vibe coding isn&#8217;t coding</p><p>(<a href="https://www.youtube.com/watch?v=lL9-yATNAo0&amp;t=1336s">22:16</a>) Unsolved problems in agentic AI tooling and how Airbnb is addressing them</p><p>(<a href="https://www.youtube.com/watch?v=lL9-yATNAo0&amp;t=1590s">26:30</a>) Airbnb&#8217;s overall AI philosophy in practice</p><p>(<a href="https://www.youtube.com/watch?v=lL9-yATNAo0&amp;t=1755s">29:15</a>) Using agentic AI to accelerate code migrations</p><p>(<a href="https://www.youtube.com/watch?v=lL9-yATNAo0&amp;t=1818s">30:18</a>) AirChat SDK: How Airbnb enables teams to build AI-powered applications</p><p>(<a href="https://www.youtube.com/watch?v=lL9-yATNAo0&amp;t=1997s">33:17</a>) AirChat Remote and asynchronous agent workflows</p><p>(<a href="https://www.youtube.com/watch?v=lL9-yATNAo0&amp;t=2167s">36:07</a>) Predictions for what&#8217;s next</p><p><strong>Where to find Christopher Sanson:</strong></p><p>&#8226; LinkedIn: <a href="https://www.linkedin.com/in/christophersanson">https://www.linkedin.com/in/christophersanson</a></p><p><strong>Where to find Madison Capps:</strong></p><p>&#8226; LinkedIn: <a href="https://www.linkedin.com/in/madison-capps-66950625">https://www.linkedin.com/in/madison-capps-66950625</a></p><h2><strong>Referenced:</strong></h2><p>&#8226; <a href="https://www.airbnb.com/">Airbnb</a></p><p>&#8226; <a href="https://hbr.org/2011/10/steve-jobss-bicycles-for-the-m">Steve Jobs&#8217;s Bicycles for the Mind</a></p><p>&#8226; <a href="https://www.linkedin.com/in/jennifer-st-pierre-4935a81">Jennifer St Pierre</a></p><p>&#8226; <a href="http://linkedin.com/in/justinreock">Justin Reock</a></p><p>&#8226; <a href="https://getdx.com/blog/ai-generated-merged-code-holds-steady-at-30/">AI-generated merged code holds steady at ~30%</a></p><p>&#8226; <a href="https://x.com/karpathy/status/2015883857489522876">Andrej Karpathy&#8217;s post on X</a></p>]]></content:encoded></item><item><title><![CDATA[The future of engineering at Nationwide, Comcast, TD, and HPE]]></title><description><![CDATA[Leaders from Nationwide, Comcast, TD Bank, and HPE share how large enterprises are building AI-first engineering organizations and preparing for the future of software development.]]></description><link>https://newsletter.getdx.com/p/the-future-of-engineering-at-nationwide</link><guid isPermaLink="false">https://newsletter.getdx.com/p/the-future-of-engineering-at-nationwide</guid><pubDate>Mon, 22 Jun 2026 13:43:30 GMT</pubDate><enclosure url="https://api.substack.com/feed/podcast/202178587/4892229a6aa383b2a148995c7f3b58d1.mp3" length="0" type="audio/mpeg"/><content:encoded><![CDATA[<p>Listen and watch now on <strong><a href="https://youtu.be/cblHlTvfNFc">YouTube</a>, <a href="https://podcasts.apple.com/us/podcast/engineering-enablement-by-abi-noda/id1619140476">Apple</a>, and <a href="https://open.spotify.com/show/3NxjyIsuxeDMQtisDqBy7D">Spotify</a></strong>.</p><p>In this session from DX Annual, Rebecca Fitzhugh, Lead Principal Engineer at Atlassian, moderates a panel featuring Nidhi Allipuram, Vice President, Enterprise Developer Experience and Platform at Nationwide, Jai Schniepp, Senior Director, DevX Product Management at Comcast, Brent Foster, Vice President and Head of Architecture and Strategy at TD Bank, and Praveena Patchipulusu, Vice President of Engineering at HPE.</p><p>Together, they discuss how large enterprises are approaching AI adoption, what it takes to build an AI-first software development lifecycle, and how engineering leaders are balancing speed, security, governance, and developer experience. They also share their perspectives on the changing role of engineers, human accountability, and how organizations can prepare for the future of software engineering.</p><div id="youtube2-cblHlTvfNFc" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;cblHlTvfNFc&quot;,&quot;startTime&quot;:null,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/cblHlTvfNFc?rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><h2><strong>Some takeaways: </strong></h2><p><strong>Building an AI-first software development lifecycle</strong></p><ul><li><p><strong>AI adoption is becoming a redesign effort, not a tooling effort.</strong> Several panelists argued that the biggest opportunity is not simply adding AI assistants to existing workflows but rethinking the software development lifecycle itself. Rather than treating AI as a coding tool, organizations are beginning to integrate it into requirements gathering, design, testing, code reviews, and deployment.</p></li><li><p><strong>Training and organizational support matter more than tool selection.</strong> Nationwide found that productivity gains came less from introducing new tools and more from providing engineers with training, coaching, playbooks, and time to learn. Teams consistently reported that air cover, psychological safety, and opportunities to experiment were more valuable than access to additional AI products.</p></li><li><p><strong>Successful adoption requires systems, not mandates.</strong> Organizations cannot simply tell teams to &#8220;go use AI.&#8221; Several panelists described building AI champion programs, governance models, embedded coaching, and structured learning opportunities that help teams develop new habits and scale adoption across large enterprises.</p></li></ul><p><strong>Keeping humans accountable</strong></p><ul><li><p><strong>Humans remain responsible for outcomes regardless of who writes the code.</strong> Every panelist emphasized that accountability does not shift to AI. Whether code is generated by an engineer, a copilot, or an agent, humans remain responsible for validating outputs, making decisions, and owning the results delivered to customers.</p></li><li><p><strong>Validation is becoming more important than approval.</strong> Traditional approval processes may matter less than ensuring the right people validate assumptions, outcomes, and risks. Teams are increasingly focused on creating workflows where humans review and challenge AI-generated work rather than simply acting as signoff gates.</p></li><li><p><strong>Decision-making is becoming a core engineering skill.</strong> As AI takes over more implementation work, engineers are spending more time evaluating tradeoffs, validating outputs, and making judgment calls. The ability to make good decisions quickly may become a larger differentiator than the ability to manually write code.</p></li></ul><p><strong>Security and governance in an AI-powered world</strong></p><ul><li><p><strong>Shift-left practices become even more important with AI.</strong> Security, compliance, and quality checks are being pushed earlier into the development process. Rather than relying on reviews at the end of the pipeline, organizations are embedding guardrails directly into workflows and development platforms.</p></li><li><p><strong>AI-generated infrastructure introduces new challenges.</strong> The conversation extended beyond application code to infrastructure. As AI increasingly generates Terraform, YAML, and cloud configuration files, organizations must build policy-driven validation and security controls to prevent vulnerabilities from entering production environments.</p></li><li><p><strong>Context is both a powerful asset and a potential risk.</strong> One of AI&#8217;s greatest strengths is its ability to use organizational knowledge and historical context. At the same time, exposing that information to AI systems creates new security concerns, making governance and access controls increasingly important.</p></li></ul><p><strong>The changing role of the engineer</strong></p><ul><li><p><strong>Engineers are becoming orchestrators rather than implementers.</strong> As AI takes over more boilerplate work, engineers are expected to focus more on system design, architecture, critical thinking, and coordinating work across humans, agents, and platforms. Success increasingly depends on defining intent and evaluating outcomes rather than writing every line of code manually.</p></li><li><p><strong>Role boundaries are becoming less rigid.</strong> The panel described a future where engineers, product managers, designers, and other builders work more closely together. AI is making it easier for individuals to contribute across traditional functional boundaries, creating smaller teams with broader responsibilities.</p></li><li><p><strong>Critical thinking and creativity become more valuable.</strong> While AI can accelerate execution, it cannot replace human judgment and problem framing. Several panelists argued that creativity, curiosity, and the ability to think differently about problems will become increasingly important as AI capabilities continue to improve.</p></li></ul><p><strong>Rethinking developer experience</strong></p><ul><li><p><strong>Developer experience is becoming workflow experience.</strong> The focus is shifting from individual tools toward creating trusted workflows that help teams move from idea to production more quickly. Organizations are increasingly measuring success by how effectively teams can deliver outcomes rather than by how efficiently they write code.</p></li><li><p><strong>Developer experience now includes agent experience.</strong> As AI agents become active participants in software delivery, organizations must consider how agents consume context, operate within guardrails, and interact with development platforms. Designing effective systems now means thinking about both human and AI users.</p></li><li><p><strong>Breaking down silos creates better outcomes.</strong> Several panelists argued that AI provides an opportunity to reduce friction between product managers, designers, developers, and security teams. The organizations that benefit most may be those that remove barriers between disciplines and enable more collaborative ways of working.</p></li></ul><p><strong>Preparing for the future</strong></p><ul><li><p><strong>The time to experiment is now.</strong> Every panelist encouraged organizations to begin learning through direct experience rather than waiting for the technology to mature. Teams that develop AI skills, workflows, and governance practices today will be better positioned as the technology continues to evolve.</p></li><li><p><strong>Institutional knowledge may become a competitive advantage.</strong> Large enterprises possess decades of documentation, decisions, diagrams, and expertise that often remain difficult to access. Several speakers highlighted the opportunity to unlock that knowledge and make it useful through AI-powered systems.</p></li><li><p><strong>Fundamentals still matter.</strong> Despite rapid technological change, the panel repeatedly returned to the same conclusion: strong engineering fundamentals, sound judgment, accountability, security practices, and critical thinking remain essential regardless of how much AI enters the software development process.</p></li></ul><h2><strong>In this episode, we cover:</strong></h2><p>(<a href="https://www.youtube.com/watch?v=cblHlTvfNFc">00:00</a>) Intro</p><p>(<a href="https://www.youtube.com/watch?v=cblHlTvfNFc&amp;t=148s">02:28</a>) The AI journey across TD Bank, Comcast, and HPE</p><p>(<a href="https://www.youtube.com/watch?v=cblHlTvfNFc&amp;t=359s">05:59</a>) Inside Nationwide&#8217;s AI-assisted development lifecycle</p><p>(<a href="https://www.youtube.com/watch?v=cblHlTvfNFc&amp;t=604s">10:04</a>) Reimagining the software development lifecycle with AI</p><p>(<a href="https://www.youtube.com/watch?v=cblHlTvfNFc&amp;t=692s">11:32</a>) Security, governance, and human accountability</p><p>(<a href="https://www.youtube.com/watch?v=cblHlTvfNFc&amp;t=927s">15:27</a>) Embedding security and guardrails into AI workflows</p><p>(<a href="https://www.youtube.com/watch?v=cblHlTvfNFc&amp;t=1075s">17:55</a>) How AI is changing the role of an engineer</p><p>(<a href="https://www.youtube.com/watch?v=cblHlTvfNFc&amp;t=1312s">21:52</a>) What developer experience looks like in the AI era</p><p>(<a href="https://www.youtube.com/watch?v=cblHlTvfNFc&amp;t=1615s">26:55</a>) What software engineering may look like in 2030</p><p>(<a href="https://www.youtube.com/watch?v=cblHlTvfNFc&amp;t=1967s">32:47</a>) How to prepare for the AI-driven future</p><p><strong>Where to find Rebecca Fitzhugh:</strong></p><p>&#8226; LinkedIn: <a href="https://www.linkedin.com/in/rmfitzhugh">https://www.linkedin.com/in/rmfitzhugh</a></p><p>&#8226; X: <a href="https://x.com/RebeccaFitzhugh">https://x.com/RebeccaFitzhugh</a></p><p><strong>Where to find Jai Schniepp:</strong></p><p>&#8226; LinkedIn: <a href="https://www.linkedin.com/in/jessicaschniepp">https://www.linkedin.com/in/jessicaschniepp</a></p><p><strong>Where to find Nidhi Allipuram:</strong></p><p>&#8226; LinkedIn: <a href="https://www.linkedin.com/in/nidhi-allipuram">https://www.linkedin.com/in/nidhi-allipuram</a></p><p><strong>Where to find Brent Foster:</strong></p><p>&#8226; LinkedIn: <a href="https://www.linkedin.com/in/engineeringthefuture">https://www.linkedin.com/in/engineeringthefuture</a></p><p>&#8226; Website: <a href="https://brentfoster.me">https://brentfoster.me</a></p><p><strong>Where to find Praveena Patchipulusu:</strong></p><p>&#8226; LinkedIn: <a href="https://www.linkedin.com/in/praveena-patchipulusu-158741">https://www.linkedin.com/in/praveena-patchipulusu-158741</a></p><h2><strong>Referenced:</strong></h2><p>&#8226; <a href="https://www.atlassian.com/">Atlassian</a></p><p>&#8226; <a href="https://www.td.com/">TD Bank</a></p><p>&#8226; <a href="https://corporate.comcast.com/">Comcast Corporation</a></p><p>&#8226; <a href="https://www.hpe.com/us/en/home.html">Hewlett Packard Enterprise (HPE)</a></p><p>&#8226; <a href="https://www.nationwide.com/">Nationwide</a></p><p>&#8226; <a href="https://github.com/github/spec-kit">GitHub Spec Kit</a></p><p>&#8226; <a href="https://www.linkedin.com/in/abinoda/">Abi Noda</a></p>]]></content:encoded></item><item><title><![CDATA[Uber’s journey of measuring AI impact on developer productivity]]></title><description><![CDATA[How Uber evolved its approach to measuring AI&#8217;s impact on engineering, why traditional productivity metrics are breaking down, and what new frameworks may be needed in an agent-driven future.]]></description><link>https://newsletter.getdx.com/p/ubers-journey-of-measuring-ai-impact</link><guid isPermaLink="false">https://newsletter.getdx.com/p/ubers-journey-of-measuring-ai-impact</guid><pubDate>Mon, 22 Jun 2026 13:42:42 GMT</pubDate><enclosure url="https://api.substack.com/feed/podcast/202179543/75f0c689e4474be76ee387b4b684a152.mp3" length="0" type="audio/mpeg"/><content:encoded><![CDATA[<p>Listen and watch now on <strong><a href="https://youtu.be/aQmnolXCH_M">YouTube</a>, <a href="https://podcasts.apple.com/us/podcast/engineering-enablement-by-abi-noda/id1619140476">Apple</a>, and <a href="https://open.spotify.com/show/3NxjyIsuxeDMQtisDqBy7D">Spotify</a></strong>.</p><p>As AI becomes embedded in software development, many of the metrics that engineering organizations have relied on for years are starting to break down.</p><p>In this session from DX Annual, Uber&#8217;s Ty Smith and Abhishek Tibrewal share how their approach to measuring AI&#8217;s impact on developer productivity has evolved over time. They walk through the different phases of their measurement journey, from adoption and engagement to measuring impact, ROI, and agentic value, explaining what they chose to measure at each stage, what worked, what failed, and how their thinking changed along the way.</p><p>They also discuss the role of qualitative feedback before telemetry existed, the challenge of identifying meaningful engagement signals, why &#8220;developer years saved&#8221; failed as an ROI metric, and how AI agents forced them to rethink traditional productivity measurements. Finally, they introduce Uber&#8217;s emerging framework built around feature velocity and explore the unanswered questions that remain as software development becomes increasingly agent-driven.</p><div id="youtube2-aQmnolXCH_M" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;aQmnolXCH_M&quot;,&quot;startTime&quot;:null,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/aQmnolXCH_M?rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><h2><strong>Some takeaways: </strong></h2><p><strong>Why AI breaks traditional productivity metrics</strong></p><ul><li><p><strong>Many measurement frameworks were built for a world where humans wrote most of the code.</strong> As AI agents become more capable, metrics that once provided useful signals can quickly become misleading.</p></li><li><p><strong>Teams should expect their metrics to break.</strong> Uber&#8217;s measurement journey required repeatedly revisiting assumptions as AI-assisted development evolved into agentic workflows.</p></li></ul><p><strong>Start with stakeholder questions</strong></p><ul><li><p><strong>The best metrics answer real business questions.</strong> Uber worked backward from questions about productivity, ROI, investment priorities, and business value instead of collecting data for its own sake.</p></li><li><p><strong>Measurement should support decision-making.</strong> Metrics influence budgets, tooling investments, enablement efforts, and long-term strategy.</p></li><li><p><strong>Use qualitative signals before telemetry exists</strong></p></li><li><p><strong>Qualitative feedback can be the fastest path to insight.</strong> Before AI tooling generated reliable telemetry, Uber relied on surveys, interviews, and experience sampling to understand adoption and guide investments.</p></li><li><p><strong>Behavioral questions are more useful than perception questions.</strong> Asking developers what they actually did produced stronger signals than asking whether they found AI helpful.</p></li><li><p><strong>Measure engagement through behavior, not demographics</strong></p></li><li><p><strong>Behavioral patterns revealed insights that demographics could not.</strong> Role, tenure, and organization offered limited signal compared to how engineers actually used AI tools.</p></li><li><p><strong>A small group of AI power users emerged early.</strong> Studying usage patterns helped Uber identify engineers who were engaging deeply with AI and generating outsized results.</p></li></ul><p><strong>Correlation is not causation</strong></p><ul><li><p><strong>High AI usage does not automatically prove AI caused higher productivity.</strong> The most productive engineers are often the first to adopt new tools.</p></li><li><p><strong>Rigorous analysis matters when making investment decisions.</strong> Uber used causal methods to better understand the true impact of AI-assisted development.</p></li></ul><p><strong>Why measuring AI ROI is difficult</strong></p><ul><li><p><strong>Developer years saved sounded compelling but failed as an ROI metric.</strong> The approach created anxiety around replacement, required constant recalibration, and did not answer the business questions leadership cared about most.</p></li><li><p><strong>Business leaders ultimately care about outcomes.</strong> Time saved is useful context, but value creation, customer impact, and business results matter more.</p></li><li><p><strong>PRs measure activity, features measure value</strong></p></li><li><p><strong>Agentic AI exposes the limitations of activity-based metrics.</strong> A single agent task can generate many pull requests without creating meaningful customer value.</p></li><li><p><strong>Feature velocity became Uber&#8217;s new North Star.</strong> The goal shifted from measuring engineering output to measuring whether valuable capabilities were actually being delivered.</p></li></ul><p><strong>Building an AI-native measurement framework</strong></p><ul><li><p><strong>Feature velocity works alongside supporting metrics.</strong> Flow efficiency, quality, and capability expansion help create a more complete picture of AI&#8217;s impact.</p></li><li><p><strong>PR classification provides important context.</strong> Understanding the type and complexity of work helps distinguish meaningful progress from routine maintenance and toil.</p></li><li><p><strong>The future belongs to outcome-based metrics</strong></p></li><li><p><strong>The most durable metrics are tied to business outcomes rather than engineering activity.</strong> As AI becomes more autonomous, output alone becomes a less reliable signal.</p></li><li><p><strong>Many important questions remain unanswered.</strong> Organizations still need better ways to measure judgment, autonomy, technical debt, and the value created by increasingly agent-driven software development.</p></li></ul><h2><strong>In this episode, we cover:</strong></h2><p>(<a href="https://www.youtube.com/watch?v=aQmnolXCH_M">00:00</a>) Intro</p><p>(<a href="https://www.youtube.com/watch?v=aQmnolXCH_M&amp;t=90s">01:30</a>) Steve Yegge&#8217;s 8 stages of AI-assisted development</p><p>(<a href="https://www.youtube.com/watch?v=aQmnolXCH_M&amp;t=202s">03:22</a>) Uber&#8217;s shift to a generative AI-powered company</p><p>(<a href="https://www.youtube.com/watch?v=aQmnolXCH_M&amp;t=260s">04:20</a>) Uber&#8217;s pre-AI productivity metrics</p><p>(<a href="https://www.youtube.com/watch?v=aQmnolXCH_M&amp;t=415s">06:55</a>) Important questions from stakeholders that previous metrics didn&#8217;t answer</p><p>(<a href="https://www.youtube.com/watch?v=aQmnolXCH_M&amp;t=505s">08:25</a>) How Uber measures AI before telemetry exists</p><p>(<a href="https://www.youtube.com/watch?v=aQmnolXCH_M&amp;t=671s">11:11</a>) Metrics used to measure adoption</p><p>(<a href="https://www.youtube.com/watch?v=aQmnolXCH_M&amp;t=769s">12:49</a>) Measuring engagement</p><p>(<a href="https://www.youtube.com/watch?v=aQmnolXCH_M&amp;t=870s">14:30</a>) Measuring impact</p><p>(<a href="https://www.youtube.com/watch?v=aQmnolXCH_M&amp;t=992s">16:32</a>) The challenge of measuring AI ROI</p><p>(<a href="https://www.youtube.com/watch?v=aQmnolXCH_M&amp;t=1172s">19:32</a>) Rethinking adoption, engagement, and impact for agentic AI</p><p>(<a href="https://www.youtube.com/watch?v=aQmnolXCH_M&amp;t=1561s">26:01</a>) The new north star: Feature velocity</p><p>(<a href="https://www.youtube.com/watch?v=aQmnolXCH_M&amp;t=1721s">28:41</a>) PR classification + feature velocity: the questions it can answer</p><p>(<a href="https://www.youtube.com/watch?v=aQmnolXCH_M&amp;t=1981s">33:01</a>) What comes next and what&#8217;s still unanswered</p><p>(<a href="https://www.youtube.com/watch?v=aQmnolXCH_M&amp;t=2070s">34:30</a>) Lessons learned and what they&#8217;d do differently</p><p>(<a href="https://www.youtube.com/watch?v=aQmnolXCH_M&amp;t=2231s">37:11</a>) Q&amp;A #1: How Uber defines a feature</p><p>(<a href="https://www.youtube.com/watch?v=aQmnolXCH_M&amp;t=2330s">38:50</a>) Q&amp;A #2: Measuring success and AI ROI</p><p><strong>Where to find Abhishek Tibrewal</strong></p><p>&#8226; LinkedIn: <a href="https://www.linkedin.com/in/aabhishektibrewal">https://www.linkedin.com/in/aabhishektibrewal</a></p><p><strong>Where to find Ty Smith:</strong></p><p>&#8226; LinkedIn: <a href="https://www.linkedin.com/in/tyvsmith">https://www.linkedin.com/in/tyvsmith</a></p><h2><strong>Referenced:</strong></h2><p>&#8226; <a href="https://steve-yegge.medium.com/welcome-to-gas-town-4f25ee16dd04">Welcome to Gas Town</a></p><p>&#8226; <a href="https://x.com/dkhos?lang=en">Dara Khosrowshahi (Uber CEO)</a></p>]]></content:encoded></item><item><title><![CDATA[From AI experiments to organizational shift: Lessons from Mercari’s transformation]]></title><description><![CDATA[What Mercari learned after mandating 100% AI adoption&#8212;and why faster code generation didn&#8217;t automatically lead to faster software delivery.]]></description><link>https://newsletter.getdx.com/p/from-ai-experiments-to-organizational</link><guid isPermaLink="false">https://newsletter.getdx.com/p/from-ai-experiments-to-organizational</guid><dc:creator><![CDATA[Justin Reock]]></dc:creator><pubDate>Mon, 15 Jun 2026 13:23:22 GMT</pubDate><enclosure url="https://api.substack.com/feed/podcast/201672920/47acd4f0792320a06193ca103e5f43a3.mp3" length="0" type="audio/mpeg"/><content:encoded><![CDATA[<p>Listen and watch now on <strong><a href="https://youtu.be/Y8LTIZcv66k">YouTube</a>, <a href="https://podcasts.apple.com/us/podcast/engineering-enablement-by-abi-noda/id1619140476">Apple</a>, and <a href="https://open.spotify.com/show/3NxjyIsuxeDMQtisDqBy7D">Spotify</a></strong>.</p><p>Michael Galloway leads Platform Engineering at Mercari, while Snehal Shinde leads Cost and Performance Engineering. Together, they have been at the center of Mercari&#8217;s effort to become an AI-native company.</p><p>In this session from DX Annual, Michael and Snehal share what happened after Mercari&#8217;s CEO mandated 100% AI adoption across the organization. While AI accelerated code generation and increased engineering output, the team quickly discovered that their existing dashboards could not answer a simple question: was AI actually improving productivity?</p><p>They discuss how Mercari built new visibility into AI usage and software delivery, the bottlenecks they uncovered across the SDLC, why faster coding did not automatically translate into faster delivery, and the lessons they learned rolling out AI at scale. They also share how Mercari is rethinking software development around agents, feedback loops, and new ways of working.</p><div id="youtube2-Y8LTIZcv66k" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;Y8LTIZcv66k&quot;,&quot;startTime&quot;:null,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/Y8LTIZcv66k?rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><h2><strong>Some takeaways: </strong></h2><p><strong>Measuring AI impact</strong></p><ul><li><p><strong>AI adoption alone does not guarantee business value.</strong> Mercari found that while AI usage increased rapidly across the organization, existing dashboards could not answer the leadership team&#8217;s most important question: whether AI was actually improving productivity.</p></li><li><p><strong>Local optimization does not necessarily improve system-wide performance.</strong> Engineers reported working faster with AI tools, but end-to-end delivery metrics remained largely unchanged because bottlenecks elsewhere in the software delivery process continued to slow teams down.</p></li><li><p><strong>Organizations need visibility into both AI usage and delivery outcomes.</strong> Mercari built new dashboards that combined AI tool data with SDLC metrics to better understand adoption, throughput, quality, and operational performance.</p></li></ul><p><strong>The reality of becoming AI-Native</strong></p><ul><li><p><strong>AI adoption required a cultural transformation, not just a tooling rollout.</strong> Mercari&#8217;s CEO mandated company-wide AI adoption, but success depended on changing workflows, habits, and expectations across engineering, finance, legal, customer support, and other functions.</p></li><li><p><strong>Different teams required different forms of enablement.</strong> Employees varied significantly in their technical backgrounds and familiarity with AI tools, making education, workshops, and support systems essential to driving adoption.</p></li><li><p><strong>The goal was to rethink work itself.</strong> Rather than layering AI onto existing processes, Mercari challenged teams to reconsider what they built, how they built it, and how people worked together.</p></li></ul><p><strong>The bottlenecks AI exposed</strong></p><ul><li><p><strong>AI revealed problems that already existed inside the organization.</strong> Review queues, CI instability, deployment friction, and support requests became more visible as coding accelerated.</p></li><li><p><strong>Code generation was not the primary constraint.</strong> Engineers often spent more time waiting for approvals, navigating organizational boundaries, and dealing with infrastructure limitations than writing code.</p></li><li><p><strong>System complexity amplified AI-related challenges.</strong> As AI-generated changes increased, existing architectural complexity and fragile workflows became harder to ignore.</p></li></ul><p><strong>Finding AI workflow opportunities</strong></p><ul><li><p><strong>Mercari mapped AI opportunities across 33 domains.</strong> The AI task force reviewed functional areas across the company to identify where AI could automate work, where it could not, and where the strongest leverage points existed.</p></li><li><p><strong>The biggest opportunities extended far beyond engineering.</strong> Role-specific workshops helped teams in finance, legal, design, operations, customer support, and other departments find practical AI use cases in their own workflows.</p></li><li><p><strong>Early wins created proof points for broader adoption.</strong> Mercari saw measurable impact from support bots, accounting workflows, platform support automation, and Socrates, an internal BI agent that made company data easier to query and use.</p></li></ul><p><strong>Rethinking software development</strong></p><ul><li><p><strong>Faster coding shifted attention upstream.</strong> As implementation became easier, planning, specification, and decision-making emerged as larger constraints on delivery speed.</p></li><li><p><strong>Agent Spec-Driven Development moves AI earlier in the lifecycle.</strong> Mercari began using agents to analyze documentation, code, and organizational knowledge before implementation work started.</p></li><li><p><strong>Future workflows will focus more on intent than execution.</strong> Teams increasingly define goals, constraints, and success criteria while agents handle larger portions of implementation and validation.</p></li></ul><p><strong>Preparing for an agent-driven future</strong></p><ul><li><p><strong>Feedback loops matter more than ever.</strong> Mercari&#8217;s multi-loop SDLC emphasizes rapid validation, iterative learning, and increasingly autonomous agent workflows.</p></li><li><p><strong>Behavioral change remains harder than technological change.</strong> Organizations must rethink ownership, accountability, and trust before they can fully benefit from agent-based development.</p></li><li><p><strong>The path to AI-native development is iterative.</strong> Mercari expects continued setbacks and learning cycles, applying what they describe as the Stockdale Paradox: maintaining confidence in the destination while remaining honest about current challenges.</p></li></ul><h2><strong>In this episode, we cover:</strong></h2><p>(<a href="https://www.youtube.com/watch?v=Y8LTIZcv66k">00:00</a>) Intro</p><p>(<a href="https://www.youtube.com/watch?v=Y8LTIZcv66k&amp;t=106s">01:46</a>) Mercari&#8217;s scale and engineering culture</p><p>(<a href="https://www.youtube.com/watch?v=Y8LTIZcv66k&amp;t=171s">02:51</a>) DX awards at Mercari</p><p>(<a href="https://www.youtube.com/watch?v=Y8LTIZcv66k&amp;t=224s">03:44</a>) Mercari&#8217;s push to become AI-native</p><p>(<a href="https://www.youtube.com/watch?v=Y8LTIZcv66k&amp;t=394s">06:34</a>) The mandate to rethink everything</p><p>(<a href="https://www.youtube.com/watch?v=Y8LTIZcv66k&amp;t=482s">08:02</a>) Mercari&#8217;s AI visibility problem and how they solved it</p><p>(<a href="https://www.youtube.com/watch?v=Y8LTIZcv66k&amp;t=690s">11:30</a>) Mercari&#8217;s early findings on AI implementation</p><p>(<a href="https://www.youtube.com/watch?v=Y8LTIZcv66k&amp;t=1127s">18:47</a>) Closing the AI awareness gap at Mercari</p><p>(<a href="https://www.youtube.com/watch?v=Y8LTIZcv66k&amp;t=1271s">21:11</a>) Mapping AI opportunities across Mercari</p><p>(<a href="https://www.youtube.com/watch?v=Y8LTIZcv66k&amp;t=1892s">31:32</a>) Unpacking the results from the second rollout</p><p>(<a href="https://www.youtube.com/watch?v=Y8LTIZcv66k&amp;t=2054s">34:14</a>) Agent spec-driven development and what&#8217;s next</p><p>(<a href="https://www.youtube.com/watch?v=Y8LTIZcv66k&amp;t=2257s">37:37</a>) A multi-loop SDLC</p><p>(<a href="https://www.youtube.com/watch?v=Y8LTIZcv66k&amp;t=2450s">40:50</a>) Some hard lessons</p><p>(<a href="https://www.youtube.com/watch?v=Y8LTIZcv66k&amp;t=2575s">42:55</a>) Closing thoughts</p><p><strong>Where to find Michael Galloway:</strong></p><p>&#8226; LinkedIn: <a href="https://www.linkedin.com/in/michaelroygalloway">https://www.linkedin.com/in/michaelroygalloway</a></p><p>&#8226; X: <a href="https://x.com/michaelgalloway">https://x.com/michaelgalloway</a></p><p><strong>Where to find Snehal Shinde:</strong></p><p>&#8226; LinkedIn: <a href="https://www.linkedin.com/in/snehal-shinde">https://www.linkedin.com/in/snehal-shinde</a></p><h2><strong>Referenced:</strong></h2><p>&#8226; <a href="https://www.mercari.com/">Mercari</a></p><p>&#8226; <a href="https://cursor.com/">Cursor</a></p><p>&#8226; <a href="https://devin.ai/">Devin</a></p><p>&#8226; <a href="https://www.anthropic.com/product/claude-code">Claude Code | Anthropic&#8217;s agentic coding system</a></p><p>&#8226; <a href="https://github.com/">GitHub</a></p><p>&#8226; <a href="https://www.datadoghq.com/">Datadog</a></p><p>&#8226; <a href="https://www.linkedin.com/in/tbozarth">Tim Bozarth - Microsoft | LinkedIn</a></p><p>&#8226; <a href="https://www.airbnb.com/">Airbnb</a></p><p>&#8226; <a href="https://jimcollins.com/concepts/Stockdale-Concept.html">Jim Collins - Concepts - The Stockdale Paradox</a></p>]]></content:encoded></item><item><title><![CDATA[Augmented, accelerated, autonomized: How Vanguard is embedding AI across the product lifecycle]]></title><description><![CDATA[How Vanguard is scaling AI across 800+ product teams by moving beyond coding assistants and transforming the entire product development lifecycle.]]></description><link>https://newsletter.getdx.com/p/augmented-accelerated-autonomized</link><guid isPermaLink="false">https://newsletter.getdx.com/p/augmented-accelerated-autonomized</guid><dc:creator><![CDATA[Justin Reock]]></dc:creator><pubDate>Mon, 15 Jun 2026 13:21:55 GMT</pubDate><enclosure url="https://api.substack.com/feed/podcast/201656519/7a8282dc93c72eb64b04c96c1ec36bbb.mp3" length="0" type="audio/mpeg"/><content:encoded><![CDATA[<p>Listen and watch now on <strong><a href="https://youtu.be/yMD4us7d_HYhttps://youtu.be/yMD4us7d_HY">YouTube</a>, <a href="https://podcasts.apple.com/us/podcast/engineering-enablement-by-abi-noda/id1619140476">Apple</a>, and <a href="https://open.spotify.com/show/3NxjyIsuxeDMQtisDqBy7D">Spotify</a></strong>.</p><p>Kelly Anne Pipe is Head of Developer Experience at Vanguard, and Nicole Scribner is a Director in the firm&#8217;s Chief Technology Office focused on engineering enablement and advancement.</p><p>In this session from DX Annual, Kelly Anne and Nicole share how Vanguard is expanding its AI strategy beyond software engineering to the entire product development lifecycle. While the company initially focused on tools like GitHub Copilot for engineers, they found that faster coding alone did not significantly improve delivery speed. Product managers, designers, QA teams, and organizational processes were still operating at a different pace.</p><p>To address this challenge, Vanguard developed a product team maturity model built around three stages: Augmented, Accelerated, and Autonomized. The framework spans six dimensions, from AI-powered delivery and AI-ready codebases to team autonomy, operations, and responsible AI.</p><p>Kelly Anne and Nicole explain how Vanguard is applying the model across more than 800 product teams, the behaviors they believe will enable faster delivery, and the lessons they have learned about measurement, organizational change, dependencies, and scaling AI across the product development lifecycle.</p><div id="youtube2-yMD4us7d_HY" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;yMD4us7d_HY&quot;,&quot;startTime&quot;:null,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/yMD4us7d_HY?rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><h2><strong>Some takeaways: </strong></h2><h4><strong>Beyond the engineering bubble</strong></h4><ul><li><p><strong>Faster coding does not automatically lead to faster delivery.</strong> Vanguard found that while engineers using AI tools reported significant productivity gains, product managers, designers, QA teams, and governance processes were still operating at traditional speeds.</p></li><li><p><strong>AI adoption becomes fragmented when it is treated as an engineering initiative.</strong> Organizations that focus solely on developer tooling risk creating an &#8220;engineering bubble&#8221; where one part of the product team accelerates while the rest of the workflow remains unchanged.</p></li><li><p><strong>The goal is to optimize the entire product development lifecycle.</strong> Vanguard shifted its focus from helping engineers code faster to helping cross-functional product teams move faster from idea to production.</p></li></ul><p><strong>The AI maturity model</strong></p><ul><li><p><strong>Vanguard built a maturity model around three stages: Augmented, Accelerated, and Autonomized.</strong> The framework gives more than 800 product teams a shared language for discussing AI adoption and long-term transformation.</p></li><li><p><strong>The model spans six dimensions of AI maturity.</strong> These include AI-powered product delivery, AI-ready codebases, agent-powered workflows, AI-augmented operations, team autonomy and enablement, and responsible AI.</p></li><li><p><strong>The goal is organizational transformation, not tool adoption.</strong> The framework focuses on how entire product teams evolve as AI becomes embedded throughout the product development lifecycle.</p></li></ul><p><strong>Building AI-ready foundations</strong></p><ul><li><p><strong>AI readiness starts with the fundamentals.</strong> Documentation, testing, CI/CD pipelines, architecture decisions, and code quality all become more important when agents are introduced into the development process.</p></li><li><p><strong>The codebase becomes the interface between teams and AI agents.</strong> Poor documentation, weak test coverage, and slow feedback loops limit the effectiveness of even the most capable AI tools.</p></li><li><p><strong>Dependencies become more visible at agent speed.</strong> Processes that were merely frustrating for humans become major bottlenecks when AI can complete implementation work in hours rather than days.</p></li></ul><p><strong>Scaling AI beyond engineering</strong></p><ul><li><p><strong>Every role on the product team needs AI-specific workflows.</strong> Vanguard is focused on helping product managers, designers, QA teams, and engineers incorporate AI into their daily work rather than limiting adoption to developers.</p></li><li><p><strong>The most valuable opportunities often begin before coding starts.</strong> AI can help transform customer conversations, discovery work, requirements, and design artifacts into implementation-ready inputs.</p></li><li><p><strong>Agent orchestration changes the role of the human.</strong> As agents take on more routine execution work, people increasingly act as orchestrators, reviewers, and strategic decision-makers.</p></li></ul><p><strong>The challenges of adoption and measurement</strong></p><ul><li><p><strong>Behavior change is harder than deploying tools.</strong> Vanguard found that fear, uncertainty, and questions about job security often create bigger barriers to adoption than technology itself.</p></li><li><p><strong>Simple productivity metrics can be misleading.</strong> Measures such as lines of code generated or time saved per developer do not capture whether AI is creating meaningful business value.</p></li><li><p><strong>Organizations need layered measurement strategies.</strong> Adoption metrics, process improvements, cycle time, quality, and customer outcomes all need to be considered together to understand AI&#8217;s true impact.</p></li></ul><p><strong>Lessons from the AI transition</strong></p><ul><li><p><strong>Agent speed exposes organizational debt.</strong> Slow approvals, review queues, onboarding processes, and governance workflows become much more obvious when implementation work accelerates.</p></li><li><p><strong>Responsible AI can accelerate delivery rather than slow it down.</strong> Investing in guardrails, governance, security, and automated controls early enables teams to move faster with greater confidence.</p></li><li><p><strong>The biggest opportunity is organizational transformation.</strong> Vanguard believes the future belongs to companies that redesign entire product teams around AI rather than simply adding AI tools to existing workflows.</p></li></ul><h2><strong>In this episode, we cover:</strong></h2><p>(<a href="https://www.youtube.com/watch?v=yMD4us7d_HY">00:00</a>) Intro</p><p>(<a href="https://www.youtube.com/watch?v=yMD4us7d_HY&amp;t=136s">02:16</a>) The state of AI one year ago at Vanguard</p><p>(<a href="https://www.youtube.com/watch?v=yMD4us7d_HY&amp;t=174s">02:54</a>) The engineering bubble</p><p>(<a href="https://www.youtube.com/watch?v=yMD4us7d_HY&amp;t=305s">05:05</a>) Building an AI maturity model for 800 product teams</p><p>(<a href="https://www.youtube.com/watch?v=yMD4us7d_HY&amp;t=504s">08:24</a>) Dimension 1: AI-powered product delivery</p><p>(<a href="https://www.youtube.com/watch?v=yMD4us7d_HY&amp;t=600s">10:00</a>) Dimension 2: AI-ready codebase</p><p>(<a href="https://www.youtube.com/watch?v=yMD4us7d_HY&amp;t=740s">12:20</a>) Dimension 3: Autonomous agent utilization</p><p>(<a href="https://www.youtube.com/watch?v=yMD4us7d_HY&amp;t=780s">13:00</a>) Dimension 4: AI-augmented operations</p><p>(<a href="https://www.youtube.com/watch?v=yMD4us7d_HY&amp;t=840s">14:00</a>) Dimension 5: Team autonomy and enablement</p><p>(<a href="https://www.youtube.com/watch?v=yMD4us7d_HY&amp;t=971s">16:11</a>) Dimension 6: Responsible AI</p><p>(<a href="https://www.youtube.com/watch?v=yMD4us7d_HY&amp;t=1095s">18:15</a>) The people problem: role evolution</p><p>(<a href="https://www.youtube.com/watch?v=yMD4us7d_HY&amp;t=1200s">20:00</a>) The measurement problem</p><p>(<a href="https://www.youtube.com/watch?v=yMD4us7d_HY&amp;t=1375s">22:55</a>) Lessons learned from rolling out the maturity model</p><p>(<a href="https://www.youtube.com/watch?v=yMD4us7d_HY&amp;t=1606s">26:46</a>) What&#8217;s ahead</p><p>(<a href="https://www.youtube.com/watch?v=yMD4us7d_HY&amp;t=1810s">30:10</a>) Q&amp;A #1: Getting your codebase ready for AI</p><p>(<a href="https://www.youtube.com/watch?v=yMD4us7d_HY&amp;t=1942s">32:22</a>) Q&amp;A #2: Audit trails and responsible AI</p><p>(<a href="https://www.youtube.com/watch?v=yMD4us7d_HY&amp;t=2056s">34:16</a>) Q&amp;A #3: Vanguard&#8217;s maturity model progress</p><p>(<a href="https://www.youtube.com/watch?v=yMD4us7d_HY&amp;t=2175s">36:15</a>) Q&amp;A #4: Measuring cycle time across 800 teams</p><p><strong>Where to find Nicole Scribner:</strong></p><p>&#8226; LinkedIn: <a href="https://www.linkedin.com/in/nicole-scribner-35b80422a">https://www.linkedin.com/in/nicole-scribner-35b80422a</a></p><p><strong>Where to find Kelly Anne Pipe:</strong></p><p>&#8226; LinkedIn: <a href="https://www.linkedin.com/in/kellyannepipe">https://www.linkedin.com/in/kellyannepipe</a></p><h2><strong>Referenced:</strong></h2><p>&#8226; <a href="https://investor.vanguard.com/">Vanguard</a></p><p>&#8226; <a href="https://www.linkedin.com/in/jennifer-st-pierre-4935a81/">Jennifer St Pierre - Dell Technologies | LinkedIn</a></p><p>&#8226; <a href="https://www.mercari.com/">Mercari</a></p>]]></content:encoded></item><item><title><![CDATA[Prioritization as code: An AI-supported framework for platform engineering]]></title><description><![CDATA[How SiriusXM built a data-driven framework for platform engineering prioritization by making assumptions visible, measurable, and easier to challenge.]]></description><link>https://newsletter.getdx.com/p/prioritization-as-code-an-ai-supported</link><guid isPermaLink="false">https://newsletter.getdx.com/p/prioritization-as-code-an-ai-supported</guid><dc:creator><![CDATA[Justin Reock]]></dc:creator><pubDate>Mon, 15 Jun 2026 13:20:13 GMT</pubDate><enclosure url="https://api.substack.com/feed/podcast/201654933/93008c1f1580ee2b30df80141a839676.mp3" length="0" type="audio/mpeg"/><content:encoded><![CDATA[<p>Listen and watch now on <strong><a href="https://youtu.be/4l2kFsSi7E4">YouTube</a>, <a href="https://podcasts.apple.com/us/podcast/engineering-enablement-by-abi-noda/id1619140476">Apple</a>, and <a href="https://open.spotify.com/show/3NxjyIsuxeDMQtisDqBy7D">Spotify</a></strong>.</p><p>In this session from DX Annual, Eleanor Millman, Senior Staff Product Manager, and Mina Tawadrous, Associate Director of Product Management at SiriusXM, share how their platform engineering organization developed a prioritization framework for platform engineering teams serving hundreds of developers across a complex cloud platform.</p><p>They explain how they define and weight platform-specific impact factors, use developer data to refine priorities, and score projects more consistently. They also explore why prioritization debates often stem from conflicting, invisible, or outdated assumptions, and how SiriusXM began treating assumptions like code by documenting, versioning, and reviewing them in source control.</p><p>Finally, they demonstrate how AI can surface assumptions, connect initiatives to existing knowledge, and support project scoring while keeping humans in the loop. Throughout the session, they offer a practical framework for making prioritization decisions more transparent, data-driven, and scalable.</p><div id="youtube2-4l2kFsSi7E4" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;4l2kFsSi7E4&quot;,&quot;startTime&quot;:null,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/4l2kFsSi7E4?rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><h2><strong>Some takeaways: </strong></h2><h4><strong>Building a platform engineering prioritization framework</strong></h4><ul><li><p><strong>Platform engineering requires different prioritization criteria.</strong> SiriusXM found that traditional product metrics did not fully capture the value of platform engineering work, leading the team to define platform-specific impact factors around development speed, reliability, security, cost, platform efficiency, user trust, and data-driven decision making.</p></li><li><p><strong>A simple scoring model created a shared language for prioritization.</strong> The framework combined impact, urgency, effort, and business needs to help teams compare projects consistently and explain why certain initiatives were prioritized over others.</p></li><li><p><strong>The framework evolved alongside the organization.</strong> As company priorities changed after a major platform launch, SiriusXM adjusted impact factor weights to reflect new goals around cost optimization, technical debt reduction, and data maturity.</p></li></ul><p><strong>Using developer data to guide decisions</strong></p><ul><li><p><strong>Developer feedback helped shape prioritization.</strong> Rather than relying solely on intuition, the team used survey data and other developer insights to determine where additional investment would have the greatest impact.</p></li><li><p><strong>Impact factor weights were revisited regularly.</strong> Quarterly reviews allowed the team to adjust priorities based on changing business objectives and improvements in areas such as reliability and security.</p></li><li><p><strong>Data increased confidence in prioritization decisions.</strong> By grounding discussions in evidence, teams were able to align more effectively on where to invest their limited capacity.</p></li></ul><p><strong>Treating assumptions like code</strong></p><ul><li><p><strong>Many prioritization conflicts stem from assumptions rather than priorities.</strong> Teams often disagreed because they were working from different, invisible, or outdated assumptions about users, workflows, and business needs.</p></li><li><p><strong>Documenting assumptions improved organizational alignment.</strong> SiriusXM began storing assumptions in source control, making them easier to discover, review, update, and validate over time.</p></li><li><p><strong>Debates became more productive when assumptions were explicit.</strong> Instead of arguing over which project mattered most, teams focused on validating the underlying beliefs that informed their decisions.</p></li></ul><p><strong>Using AI to surface organizational knowledge</strong></p><ul><li><p><strong>Assumption repositories became difficult to navigate at scale.</strong> As more assumptions were documented, it became increasingly difficult for individuals to find relevant context and connections across projects.</p></li><li><p><strong>AI helped uncover relationships humans might miss.</strong> By searching assumption repositories, OKRs, and prior project data, AI was able to surface relevant information that would otherwise be difficult to discover.</p></li><li><p><strong>AI improved information recall rather than replacing judgment.</strong> The goal was not automated decision making but helping teams access the knowledge needed to make better decisions.</p></li></ul><p><strong>Building an AI-assisted prioritization workflow</strong></p><ul><li><p><strong>AI can guide teams through the scoring process.</strong> SiriusXM built workflows that ask clarifying questions, surface assumptions, identify relevant organizational context, and generate initial project scores.</p></li><li><p><strong>Human validation remains essential.</strong> Teams review assumptions, challenge recommendations, and approve updates before information is added back into the system.</p></li><li><p><strong>Each prioritization cycle strengthens the knowledge base.</strong> New assumptions, decisions, and project context become available for future initiatives, making the system more valuable over time.</p></li></ul><p><strong>Keeping humans in the loop</strong></p><ul><li><p><strong>The framework is designed to support conversations, not replace them.</strong> Scores help teams discuss priorities more objectively, but important decisions still require context and judgment.</p></li><li><p><strong>Stakeholder disagreements often reveal useful information.</strong> When the framework produces results that feel wrong, the discussion can uncover missing assumptions, incomplete data, or opportunities to improve the model itself.</p></li><li><p><strong>The framework continues to evolve.</strong> SiriusXM treats both the prioritization model and the supporting AI tools as products that require ongoing iteration, feedback, and refinement.</p></li></ul><h2><strong>In this episode, we cover:</strong></h2><p>(<a href="https://www.youtube.com/watch?v=4l2kFsSi7E4">00:00</a>) Intro</p><p>(<a href="https://www.youtube.com/watch?v=4l2kFsSi7E4&amp;t=178s">02:58</a>) Building a platform engineering prioritization framework</p><p>(<a href="https://www.youtube.com/watch?v=4l2kFsSi7E4&amp;t=299s">04:59</a>) The seven platform engineering impact factors</p><p>(<a href="https://www.youtube.com/watch?v=4l2kFsSi7E4&amp;t=578s">09:38</a>) Using impact factors to score projects</p><p>(<a href="https://www.youtube.com/watch?v=4l2kFsSi7E4&amp;t=791s">13:11</a>) Using developer data to refine priorities</p><p>(<a href="https://www.youtube.com/watch?v=4l2kFsSi7E4&amp;t=993s">16:33</a>) Three ways assumptions fail</p><p>(<a href="https://www.youtube.com/watch?v=4l2kFsSi7E4&amp;t=1060s">17:40</a>) Assumptions as code</p><p>(<a href="https://www.youtube.com/watch?v=4l2kFsSi7E4&amp;t=1260s">21:00</a>) New problems created by assumptions as code</p><p>(<a href="https://www.youtube.com/watch?v=4l2kFsSi7E4&amp;t=1320s">22:00</a>) Using AI to surface assumptions</p><p>(<a href="https://www.youtube.com/watch?v=4l2kFsSi7E4&amp;t=1424s">23:44</a>) Building an AI-powered feedback loop</p><p>(<a href="https://www.youtube.com/watch?v=4l2kFsSi7E4&amp;t=1544s">25:44</a>) Inside the AI prioritization tool</p><p>(<a href="https://www.youtube.com/watch?v=4l2kFsSi7E4&amp;t=1698s">28:18</a>) Three steps to build your own framework</p><p>(<a href="https://www.youtube.com/watch?v=4l2kFsSi7E4&amp;t=1802s">30:02</a>) Q&amp;A #1: Evaluating high-cost projects</p><p>(<a href="https://www.youtube.com/watch?v=4l2kFsSi7E4&amp;t=1890s">31:30</a>) Q&amp;A #2: The cadence of iteration</p><p>(<a href="https://www.youtube.com/watch?v=4l2kFsSi7E4&amp;t=1930s">32:10</a>) Q&amp;A #3: When the framework conflicts with a stakeholder&#8217;s priorities</p><p>(<a href="https://www.youtube.com/watch?v=4l2kFsSi7E4&amp;t=2126s">35:26</a>) Q&amp;A #4: Using the framework for non-developers</p><p><strong>Where to find Eleanor Millman:</strong></p><p>&#8226; LinkedIn: <a href="https://www.linkedin.com/in/eleanor-millman-98b10350">https://www.linkedin.com/in/eleanor-millman-98b10350</a></p><p><strong>Where to find Mina Tawadrous:</strong></p><p>&#8226; LinkedIn: <a href="https://www.linkedin.com/in/mina-tawadrous">https://www.linkedin.com/in/mina-tawadrous</a></p><h2><strong>Referenced:</strong></h2><p>&#8226; <a href="https://aws.amazon.com/">AWS</a></p><p>&#8226; <a href="https://www.databricks.com/">Databricks</a></p><p>&#8226; <a href="https://www.intercom.com/blog/rice-simple-prioritization-for-product-managers/">RICE: Simple prioritization for product managers</a></p><p>&#8226; <a href="https://getdx.com/guide/developer-experience-surveys/">Designing developer experience surveys</a></p><p>&#8226; <a href="https://gsbpreserve.stanford.edu/view/61957/the-curse-of-knowledge">GSB Preserve | View | The Curse of Knowledge</a></p>]]></content:encoded></item><item><title><![CDATA[Doubling the productivity of your engineering team using AI]]></title><description><![CDATA[How Intercom doubled engineering throughput in nine months by making AI agents a core part of how engineers work.]]></description><link>https://newsletter.getdx.com/p/doubling-the-productivity-of-your</link><guid isPermaLink="false">https://newsletter.getdx.com/p/doubling-the-productivity-of-your</guid><dc:creator><![CDATA[Justin Reock]]></dc:creator><pubDate>Mon, 15 Jun 2026 13:18:36 GMT</pubDate><enclosure url="https://api.substack.com/feed/podcast/201652881/73dbff616ad3d0cb13dc4ee19e6dbdeb.mp3" length="0" type="audio/mpeg"/><content:encoded><![CDATA[<p>Listen and watch now on <strong><a href="https://youtu.be/iq_gvS56CUw">YouTube</a>, <a href="https://podcasts.apple.com/us/podcast/engineering-enablement-by-abi-noda/id1619140476">Apple</a>, and <a href="https://open.spotify.com/show/3NxjyIsuxeDMQtisDqBy7D">Spotify</a></strong>.</p><p>Brian Scanlan is a Senior Principal Systems Engineer at Intercom, where he works on platform engineering, developer productivity, and AI adoption across the company.</p><p>In this session from DX Annual, Brian shares how Intercom set out to double engineering throughput and ultimately achieved that goal in nine months. Rather than treating AI as an optional productivity tool, the company standardized on Claude Code, updated performance expectations, invested heavily in enablement, and adopted an agent-first approach to technical work.</p><p>Brian explains why Intercom views Claude Code as a platform rather than a tool, how the company is building domain-specific skills and workflows for agents, and why it believes agents should eventually be able to perform any technical task a senior engineer can complete on a laptop.</p><p>He also shares the data behind Intercom&#8217;s AI adoption efforts, including gains in throughput, reductions in defect backlogs, improvements in code quality, and the growing use of automated pull request approvals. Throughout the talk, Brian offers a practical look at what it takes to scale AI adoption across a large engineering organization and the lessons Intercom has learned along the way.</p><div id="youtube2-iq_gvS56CUw" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;iq_gvS56CUw&quot;,&quot;startTime&quot;:null,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/iq_gvS56CUw?rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><h2><strong>Some takeaways: </strong></h2><h4><strong>Doubling engineering throughput</strong></h4><ul><li><p><strong>Intercom set a goal to double engineering throughput.</strong> Rather than focusing on AI adoption metrics, the company chose a concrete business outcome: doubling merged pull requests per member of the R&amp;D organization.</p></li><li><p><strong>The goal was achieved in nine months.</strong> Intercom ultimately doubled PR throughput and later tripled it over a 16-month period, with no signs of the trend slowing down.</p></li><li><p><strong>Throughput was treated as a reasonable proxy for impact.</strong> While Brian acknowledged that every metric has flaws, he argued that organizations adopting AI at scale should expect to see meaningful increases in output.</p></li></ul><p><strong>Managing organizational change</strong></p><ul><li><p><strong>AI adoption became part of the job.</strong> Intercom updated expectations for engineers, designers, and product managers so that using AI tools effectively became part of performance expectations rather than an optional activity.</p></li><li><p><strong>The company combined incentives with support.</strong> Hackathons, enablement days, dedicated support teams, leadership messaging, and public recognition all helped accelerate adoption.</p></li><li><p><strong>Leadership stayed relentlessly on message.</strong> Repeating the same goals and expectations across every forum helped create clarity about the direction of the organization.</p></li></ul><p><strong>Why Intercom standardized on Claude Code</strong></p><ul><li><p><strong>Intercom chose a single AI platform.</strong> Rather than allowing teams to fragment across different tools and workflows, the company standardized on Claude Code and invested heavily in making it work well.</p></li><li><p><strong>The real value comes from context, not models.</strong> Brian argued that domain knowledge, skills, documentation, workflows, and organizational context create far more value than constantly switching between models.</p></li><li><p><strong>Agents should be treated like new employees.</strong> Intercom&#8217;s goal is to onboard agents the same way it would onboard a senior engineer by giving them access, training, tools, documentation, and clear expectations.</p></li></ul><p><strong>Building an agent-first engineering organization</strong></p><ul><li><p><strong>All technical work is becoming agent-first.</strong> Intercom believes that any task a human can perform on a laptop should eventually be accessible to agents.</p></li><li><p><strong>The focus is on durable capabilities rather than custom AI infrastructure.</strong> Teams are encouraged to build skills, access patterns, and workflows that will remain valuable even as models and tools continue to evolve.</p></li><li><p><strong>Agents should solve problems, not just execute commands.</strong> Rather than telling agents exactly which skill to run, engineers increasingly describe the problem and allow agents to determine the best workflow.</p></li></ul><p><strong>Skills as organizational knowledge</strong></p><ul><li><p><strong>Intercom has built hundreds of reusable skills.</strong> These skills capture domain expertise, troubleshooting processes, coding standards, operational procedures, and other institutional knowledge.</p></li><li><p><strong>High-quality skills create leverage across the organization.</strong> Once a skill exists, every engineer can benefit from the expertise embedded within it, even if they were not involved in creating it.</p></li><li><p><strong>Skills continuously improve over time.</strong> Engineers are encouraged to update skills whenever new knowledge is discovered so that lessons learned become available to everyone.</p></li></ul><p><strong>Measuring the impact of AI adoption</strong></p><ul><li><p><strong>Nearly all pull requests are now authored by Claude.</strong> Brian shared that more than 95% of pull requests are created with AI assistance, while automated pull request approvals continue to grow.</p></li><li><p><strong>Saved time is being reinvested into quality.</strong> As teams gained efficiency, they spent more time reducing technical debt and fixing defects, leading to a significant reduction in Intercom&#8217;s defect backlog.</p></li><li><p><strong>Code quality improved alongside throughput.</strong> Research conducted with Stanford showed that recent code changes were improving the overall quality of the codebase rather than degrading it.</p></li></ul><p><strong>The future of agentic software development</strong></p><ul><li><p><strong>Intercom wants agents to participate throughout the software development lifecycle.</strong> The company is replacing runbooks, expanding automation, and building remote agent capabilities that move work beyond individual laptops.</p></li><li><p><strong>AI adoption has expanded far beyond engineering.</strong> More than a thousand employees use Claude Code weekly, including teams in finance, operations, and other business functions.</p></li><li><p><strong>The biggest changes may still be ahead.</strong> Brian believes AI will reshape planning, team structures, workflows, and engineering roles over the coming years, not just how code is written.</p></li></ul><h2><strong>In this episode, we cover:</strong></h2><p>(<a href="https://www.youtube.com/watch?v=iq_gvS56CUw">00:00</a>) Intro</p><p>(<a href="https://www.youtube.com/watch?v=iq_gvS56CUw&amp;t=174s">02:54</a>) Intercom&#8217;s goal of doubling throughput</p><p>(<a href="https://www.youtube.com/watch?v=iq_gvS56CUw&amp;t=450s">07:30</a>) The platform strategy</p><p>(<a href="https://www.youtube.com/watch?v=iq_gvS56CUw&amp;t=570s">09:30</a>) Their agent-first strategy</p><p>(<a href="https://www.youtube.com/watch?v=iq_gvS56CUw&amp;t=658s">10:58</a>) Evergreen capabilities vs custom tooling</p><p>(<a href="https://www.youtube.com/watch?v=iq_gvS56CUw&amp;t=748s">12:28</a>) How Intercom works with agents</p><p>(<a href="https://www.youtube.com/watch?v=iq_gvS56CUw&amp;t=1003s">16:43</a>) What the data reveals about AI adoption and impact</p><p>(<a href="https://www.youtube.com/watch?v=iq_gvS56CUw&amp;t=1160s">19:20</a>) Using session data to improve AI workflows</p><p>(<a href="https://www.youtube.com/watch?v=iq_gvS56CUw&amp;t=1220s">20:20</a>) Cutting the defect backlog in half</p><p>(<a href="https://www.youtube.com/watch?v=iq_gvS56CUw&amp;t=1364s">22:44</a>) Inside Intercom&#8217;s Claude Code setup</p><p>(<a href="https://www.youtube.com/watch?v=iq_gvS56CUw&amp;t=1689s">28:09</a>) Claude Code beyond engineering</p><p>(<a href="https://www.youtube.com/watch?v=iq_gvS56CUw&amp;t=1849s">30:49</a>) Q&amp;A #1: Token cost</p><p>(<a href="https://www.youtube.com/watch?v=iq_gvS56CUw&amp;t=1972s">32:52</a>) Q&amp;A #2: Preparing for AI pricing changes</p><p>(<a href="https://www.youtube.com/watch?v=iq_gvS56CUw&amp;t=2054s">34:14</a>) Q&amp;A #3: Stress testing and auditing skills</p><p>(<a href="https://www.youtube.com/watch?v=iq_gvS56CUw&amp;t=2191s">36:31</a>) Q&amp;A #4: Criteria for agents approving PRs</p><p><strong>Where to find Brian Scanlan:</strong></p><p>&#8226; LinkedIn: <a href="https://www.linkedin.com/in/scanlanb">https://www.linkedin.com/in/scanlanb</a></p><p>&#8226; X: <a href="https://x.com/brian_scanlan">https://x.com/brian_scanlan</a></p><p>&#8226; Website: <a href="https://brian.scanlan.ie">https://brian.scanlan.ie</a></p><h2><strong>Referenced:</strong></h2><p>&#8226; <a href="https://www.intercom.com/">Intercom</a></p><p>&#8226; <a href="https://www.nytimes.com/2026/02/14/business/dealbook/software-companies-ai.html">Software? No Way. We&#8217;re an A.I. Company Now! - The New York Times</a></p><p>&#8226; <a href="https://www.anthropic.com/">Anthropic</a></p><p>&#8226; <a href="https://www.snowflake.com/en/">Snowflake</a></p><p>&#8226; <a href="https://linear.app/">Linear</a></p><p>&#8226; <a href="https://launchdarkly.com/">LaunchDarkly</a></p><p>&#8226; <a href="https://fin.ai/">Fin AI</a></p><p>&#8226; <a href="https://copilot.microsoft.com/">Microsoft Copilot</a></p><p>&#8226; <a href="https://cursor.com/">Cursor</a></p><p>&#8226; <a href="https://www.anthropic.com/product/claude-code">Claude Code | Anthropic&#8217;s agentic coding system</a></p><p>&#8226; <a href="https://x.com/Steve_Yegge?lang=en">Steve Yegge (@Steve_Yegge) / Posts / X</a></p><p>&#8226; <a href="https://www.honeycomb.io/">Honeycomb</a></p><p>&#8226; <a href="https://ideas.fin.ai/">Fin Ideas</a></p><p>&#8226; <a href="https://fin.ai/cli">Fin CLI | AI Agent Command Line Interface</a></p>]]></content:encoded></item></channel></rss>