What do developers want from AI?
Google's approach for guiding AI investments.
This is the latest issue of my newsletter. Each week I share research and perspectives on developer productivity.
This week I read What Do Developers Want From AI?, a recent installment of Google’s Developer Productivity for Humans series. The paper shares findings from their work exploring how developers want to use AI. If you’re responsible for rolling out GenAI tooling at your organization, you may find this paper’s findings, as well as Google’s approach, useful.
My summary of the paper
The discussion around AI in software engineering often focuses on boosting developer productivity, but there's less talk about how developers actually want to use these tools. Google's research team cautions that leaders might overlook the most effective applications of AI if they're too wrapped up in the technology itself.
To understand where AI tools could be deployed, Google’s research team started by learning about how developers want to work with AI. Then they considered how use cases for AI may evolve in the future. Finally, they considered which of Google’s developers’ top pain points would be good opportunities where AI could help.
How developers want to work with AI
To uncover what developers want from AI, the research team conducted interviews and user studies with developers using AI-based developer tooling. Here are the common themes that emerged:
Developers do not want AI to fundamentally change their workflows. They want AI to help them do what they are already doing, getting from point A to point B.
They want AI-based tools to save them time and energy by helping them do their job more efficiently. They aren’t just focused on speed: efficiency is also about making things easier while maintaining or improving quality.
They want AI to support simpler tasks and reduce toil. By automating repetitive or trivial tasks, they can focus more energy on the complex problem solving and creative aspects of their jobs.
In essence, developers want to stay in the driver’s seat with AI as opposed to it fully automating or changing their workflow (at least not yet). They also want to keep doing the most rewarding aspects of their work while using AI to expedite the less rewarding aspects.
Where AI-based developer tools may be heading
By examining potential directions for AI tools, we can better prepare for and anticipate their uses. The research team looked to the history of advancements in car technology as inspiration for where advancements to AI tools may be made. (Cars were chosen as inspiration because their introduction was a large technological innovation.)
This approach ultimately led them to identify three key areas for AI enhancement: enhancing, extending, and delegating human capabilities.
Most advancements with AI today either enhance or extend human capabilities. For example, AI-powered code completion tools enhance developers’ ability to write code quickly, while leveraging LLMs to find answers extend developers’ abilities to search for information. There's a lot of potential for continued innovation here to make software development more secure, of higher quality, easier to learn, and more efficient.
Longer term, developers express excitement for AI to do more. That could include using AI to help them elevate their thinking and solve more complex problems, or it could include delegating tasks.
Where AI should be deployed
Equipped with insights into what developers expect from AI, as well as possible future use cases with AI, the research team then pinpointed the major pain points where AI could help. They used data from their quarterly engineering survey to identify these areas. They explain their reasoning for using this approach:
“While AI could be applied to all aspects of the development lifecycle, at Google we are driven by what developers tell us are their pain points as a starting point for our AI investments.”
Their surveys have surfaced the top hindrances to developers at Google. The most common hindrance is tech debt, followed by poor or missing documentation, and learning a new platform, infrastructure, framework, or technology. #2 and #3 are potential opportunities for AI.
These results have informed where Google has made investments in AI. For example, to address the pain point of learning new platforms and frameworks, they are investing in helping engineers find answers to their questions using their Gemini Code Assist tool. This tool provides chatbot and troubleshooting support to developers. Google is also investing in AI support for reviewing and testing code to address pain points with those tasks.
Final thoughts
The potential for developer efficiency gains through GenAI tools may be enormous, but actually realizing these gains is difficult. This is in part due to how challenging it is for leaders to identify valuable use cases for AI, and getting everyone to benefit.
This paper from Google describes a good approach to tackling this problem: first their research team asked developers how they wanted to work with AI tools. Then they used their quarterly survey to identify problems that were hindering productivity. They ultimately used these insights to inform where AI investments were likely to yield the most benefit.
Identify use cases for GenAI tools
We recently published a free guide on identify common use cases for AI tools and how to measure their adoption. Get a copy of the guide here.
Who’s hiring right now
Here’s a roundup of new Developer Experience job openings:
ClickUp is hiring a Staff Infrastructure Engineer - DevX/EngProd | US
dbt Labs is hiring a Senior Software Engineer - Developer Experience | US
LinkedIn is hiring a Sr. Staff Software Engineer - DevProd & Happiness | San Francisco
Pfizer is hiring a Director, Principal Engineer - Stratus Programs | NYC
Plaid is hiring a Product Manager - Developer Platform | San Francisco
Rocket Money is hiring a Team Leader - Developer Experience | US
SEB Bank is hiring a Head of DSI Developer Experience | Stockholm
theScore is hiring a Sr. Software Engineer - DevEx Enablement | Toronto
The New York Times is hiring a Sr. TPM - Developer Experience | NYC
Find more DevEx job postings here.
That’s it for this week. Thanks for reading.
-Abi