Barriers to AI adoption
7 obstacles to adopting GenAI tools in engineering organizations.
This is the latest issue of my newsletter. Each week I share research and perspectives on developer productivity.
This week I read AI Tool Use and Adoption in Software Development by Individuals and Organizations by researchers at the University of Victoria. The study examines the factors impacting the adoption of AI tools like ChatGPT, Copilot, and Gemini in software development.
My summary of the paper
Previous research has shown that developers who use tools like Copilot and ChatGPT should reap significant productivity improvements and become more effective in their work; however, not all companies or developers are embracing these tools to their fullest potential. This study aimed to identify what factors limit organizations’ ability to see the level of AI tooling adoption that they expect.
The authors conducted a mixed-methods study involving interviews and a survey of 395 respondents. They identified several challenges at both individual and organizational levels, which are summarized below.
Challenges that hinder AI tool adoption
The researchers identified four challenges at the individual level and three at the organizational level that impact the level of AI tool usage within companies.
Individual challenges include:
1. Fear of decreased skills. A large percentage of engineers in the study expressed fears of decreased coding ability and losing learning ability by using AI tools, and especially by over-relying on them. Some participants expressed worry for more junior developers who could lose the opportunity to grow or learn about the codebase if they rely too heavily on AI tools.
2. Limited capabilities relative to expectations. Most participants aspire for AI to do more in certain situations. AI tools lack awareness of the user’s operational environment, including specific software versions, hardware configurations, and the nuances of the programming languages in use often becomes a bottleneck. This leads to AI-generated responses that may be incompatible with the developer’s actual working environment. They also often don’t have access to the user’s codebase, so they’re unable to generate correct results. This gets in the way of people using these tools more often.
3. Lack of prompting skill. Another notable challenge involves the difficulty of crafting the right prompts.
4. Potential judgment from peers. A smaller but still significant percentage of engineers fear judgment by their peers for using AI tools. A fear of judgment exists in junior developers who may worry about being judged by their senior colleagues for not being proficient enough to code without AI assistance. There may also be a perception that using AI indicates incompetence or laziness.
Challenges at the organizational level include:
5. Lacking a culture of sharing. Many organizations don’t actively promote communication between their people about AI tool use and best practices, and this inhibits tool adoption. In larger organizations or organizations with hybrid or remote work, it’s common that engineers aren’t even aware of what tools their colleagues are using—much less the usefulness or best practices for using these tools.
6. Company does not cover the cost of the tool. If individuals need to cover the costs of the premium version for an AI tool, many won’t.
7. Lack of company guidelines. Prior to AI tools, organizations implemented restrictions preventing unauthorized sharing of sensitive information on platforms like GitHub, but not all organizations have implemented such guidelines to address the emerging challenges of AI tools. These concerns from developers may limit their inclination to use the tools.
One of the most important relationships identified in the study are between the culture of sharing aspect and the potential judgement from peers. In environments where sharing and collaboration are encouraged, knowledge of AI tools and their potential benefits disseminates more rapidly, fostering a more widespread and effective adoption.
Final thoughts
A lot of the conversation around improving GenAI adoption centers around identifying the right use cases for these tools. This study suggests a similar but different strategy, which is to focus on promoting a culture where people share with each other how they’re using AI tools.
These findings may provide inspiration for those responsible for rolling out tools like Copilot within their organizations.
Who’s hiring right now
Here is a roundup of DevEx job openings. Find more open roles here.
Citi is hiring a Director - Engineering Excellence | TX
Netflix is hiring a Product Manager - Platform | CA
REI is hiring an Senior Manager - Platform Engineering | US
Sentry is hiring an Engineering Manager - Developer Experience | CA
That’s it for this week. If you know someone who might like this issue, consider sharing it with them:
-Abi