Engineering Enablement
Engineering Enablement by DX
Doubling the productivity of your engineering team using AI
0:00
-39:30

Doubling the productivity of your engineering team using AI

How Intercom doubled engineering throughput in nine months by making AI agents a core part of how engineers work.

Listen and watch now on YouTube, Apple, and Spotify.

Brian Scanlan is a Senior Principal Systems Engineer at Intercom, where he works on platform engineering, developer productivity, and AI adoption across the company.

In this session from DX Annual, Brian shares how Intercom set out to double engineering throughput and ultimately achieved that goal in nine months. Rather than treating AI as an optional productivity tool, the company standardized on Claude Code, updated performance expectations, invested heavily in enablement, and adopted an agent-first approach to technical work.

Brian explains why Intercom views Claude Code as a platform rather than a tool, how the company is building domain-specific skills and workflows for agents, and why it believes agents should eventually be able to perform any technical task a senior engineer can complete on a laptop.

He also shares the data behind Intercom’s AI adoption efforts, including gains in throughput, reductions in defect backlogs, improvements in code quality, and the growing use of automated pull request approvals. Throughout the talk, Brian offers a practical look at what it takes to scale AI adoption across a large engineering organization and the lessons Intercom has learned along the way.

Some takeaways:

Doubling engineering throughput

  • Intercom set a goal to double engineering throughput. Rather than focusing on AI adoption metrics, the company chose a concrete business outcome: doubling merged pull requests per member of the R&D organization.

  • The goal was achieved in nine months. Intercom ultimately doubled PR throughput and later tripled it over a 16-month period, with no signs of the trend slowing down.

  • Throughput was treated as a reasonable proxy for impact. While Brian acknowledged that every metric has flaws, he argued that organizations adopting AI at scale should expect to see meaningful increases in output.

Managing organizational change

  • AI adoption became part of the job. Intercom updated expectations for engineers, designers, and product managers so that using AI tools effectively became part of performance expectations rather than an optional activity.

  • The company combined incentives with support. Hackathons, enablement days, dedicated support teams, leadership messaging, and public recognition all helped accelerate adoption.

  • Leadership stayed relentlessly on message. Repeating the same goals and expectations across every forum helped create clarity about the direction of the organization.

Why Intercom standardized on Claude Code

  • Intercom chose a single AI platform. Rather than allowing teams to fragment across different tools and workflows, the company standardized on Claude Code and invested heavily in making it work well.

  • The real value comes from context, not models. Brian argued that domain knowledge, skills, documentation, workflows, and organizational context create far more value than constantly switching between models.

  • Agents should be treated like new employees. Intercom’s goal is to onboard agents the same way it would onboard a senior engineer by giving them access, training, tools, documentation, and clear expectations.

Building an agent-first engineering organization

  • All technical work is becoming agent-first. Intercom believes that any task a human can perform on a laptop should eventually be accessible to agents.

  • The focus is on durable capabilities rather than custom AI infrastructure. Teams are encouraged to build skills, access patterns, and workflows that will remain valuable even as models and tools continue to evolve.

  • Agents should solve problems, not just execute commands. Rather than telling agents exactly which skill to run, engineers increasingly describe the problem and allow agents to determine the best workflow.

Skills as organizational knowledge

  • Intercom has built hundreds of reusable skills. These skills capture domain expertise, troubleshooting processes, coding standards, operational procedures, and other institutional knowledge.

  • High-quality skills create leverage across the organization. Once a skill exists, every engineer can benefit from the expertise embedded within it, even if they were not involved in creating it.

  • Skills continuously improve over time. Engineers are encouraged to update skills whenever new knowledge is discovered so that lessons learned become available to everyone.

Measuring the impact of AI adoption

  • Nearly all pull requests are now authored by Claude. Brian shared that more than 95% of pull requests are created with AI assistance, while automated pull request approvals continue to grow.

  • Saved time is being reinvested into quality. As teams gained efficiency, they spent more time reducing technical debt and fixing defects, leading to a significant reduction in Intercom’s defect backlog.

  • Code quality improved alongside throughput. Research conducted with Stanford showed that recent code changes were improving the overall quality of the codebase rather than degrading it.

The future of agentic software development

  • Intercom wants agents to participate throughout the software development lifecycle. The company is replacing runbooks, expanding automation, and building remote agent capabilities that move work beyond individual laptops.

  • AI adoption has expanded far beyond engineering. More than a thousand employees use Claude Code weekly, including teams in finance, operations, and other business functions.

  • The biggest changes may still be ahead. Brian believes AI will reshape planning, team structures, workflows, and engineering roles over the coming years, not just how code is written.

In this episode, we cover:

(00:00) Intro

(02:54) Intercom’s goal of doubling throughput

(07:30) The platform strategy

(09:30) Their agent-first strategy

(10:58) Evergreen capabilities vs custom tooling

(12:28) How Intercom works with agents

(16:43) What the data reveals about AI adoption and impact

(19:20) Using session data to improve AI workflows

(20:20) Cutting the defect backlog in half

(22:44) Inside Intercom’s Claude Code setup

(28:09) Claude Code beyond engineering

(30:49) Q&A #1: Token cost

(32:52) Q&A #2: Preparing for AI pricing changes

(34:14) Q&A #3: Stress testing and auditing skills

(36:31) Q&A #4: Criteria for agents approving PRs

Where to find Brian Scanlan:

• LinkedIn: https://www.linkedin.com/in/scanlanb

• X: https://x.com/brian_scanlan

• Website: https://brian.scanlan.ie

Referenced:

Intercom

Software? No Way. We’re an A.I. Company Now! - The New York Times

Anthropic

Snowflake

Linear

LaunchDarkly

Fin AI

Microsoft Copilot

Cursor

Claude Code | Anthropic’s agentic coding system

Steve Yegge (@Steve_Yegge) / Posts / X

Honeycomb

Fin Ideas

Fin CLI | AI Agent Command Line Interface

Discussion about this episode

User's avatar

Ready for more?