AI-assisted engineering: Q4 impact report
Data and insights from 400+ organizations navigating their AI transformation.
Welcome to the latest issue of Engineering Enablement, a weekly newsletter sharing research and perspectives on developer productivity.
đ On November 20th, Abi and I are hosting a live discussion on running data-driven evaluations of AI engineering tools. Register to join.
âAre we falling behind on AI?â
Without fail, this question makes its way into nearly every conversation I have with software engineering executives these days. Click-worthy headlines claim that Coinbase has 40% of its code written by AI, that two-thirds of AI-native startups are letting AI write nearly all of their code, and dozens of other success stories have surfaced just in the past week alone. Combine that with pressure from the boardroom to improve efficiency, and itâs easy to see why so many leaders feel like theyâre already behind.
But are those headlines true? And are they a good reference point for your own organisationâs performance?
At DX, we have the unique vantage point of working closely with more than 400 organisations and seeing firsthand how their AI strategies are unfolding. To help you better contextualise your own performance, weâre releasing a new report today: AI-Assisted Engineering: Q4 Impact Report.
Each quarter, weâll analyse the prior quarterâs data to offer a clear, moment-in-time view of AI adoption and impact across the industryâreviewing the good and the bad, not just cherry-picked success stories that make for good headlines. We also include our analysis and interpretation, so you can draw meaningful insights faster and spend less time wondering what the numbers actually mean for your organisation.
In this quarterâs report, there are a few big themes.
First, AI tool use is now ubiquitous, with over 90% of developers using AI tools at least once a month. Earlier in the year, many leaders focused their AI strategies on adoption and activation, and itâs paid off. But now that adoption has ramped up, the focus needs to shift toward driving real impact.
We also looked at the extent to which AI-authored code is being merged without significant human rewrites, and found that the percentage of AI-authored code is lower than what you see in some of the major headlines. Getting reliable data here is a challenge for many organisations, but an area of tool telemetry thatâs evolving rapidly, including through a solution weâre developing at DX.
Finally, enterprises continue to lead on AI impact. Engineers at large organisations report saving more time with AI tools than those at startups or smaller companies. However, AI tool adoption within enterprises still lags behind, suggesting thereâs untapped potential once adoption catches up.
You can read the full report here.
A final thought to leave you with is that there is no âaverageâ experience with AI impact. Looking at industry trends and averages like those in this report can help contextualise your performance and do some pattern-matching, but I want to caution against using these averages as a guide to what your performance should look like. The truth is that AI impact is very asymmetrical and looks very different for each organisation. Some are doing very well when it comes to throughput, but quality is suffering; some have done a great job of using AI for migrations, but are struggling to incorporate AI in testing and release processes. Averages abstract away the nuances of each companyâs transformation journey. The best way to use this information is to enhanceânot replaceâyour own metrics and AI strategy.
If youâre looking to improve the way your organisation is measuring AI impact, the AI Measurement Framework offers practical guidance on what to measure and how to collect those measurements, and is a great place to start.
Whoâs hiring right now
This weekâs featured DevProd job openings. See more open roles here.
Capital One is hiring a Product Manager - Developer Experience | Plano TX; McLean VA; Richmond VA
Whatnot is hiring a Software Engineer - Platform | San Francisco, Los Angeles, Seattle, NYC
Dropbox is hiring a Senior Infrastructure Software Engineer | Remote - United States
Reddit: Staff Software Engineer DX | Remote - United States
Thatâs it for this week. Thanks for reading.