Structured rollout boosts Copilot adoption and satisfaction by 20%
The impact of structured Copilot training and rollout at a 2,000 engineer company.
This is the latest issue of Engineering Enablement, a weekly newsletter covering the data behind world-class engineering organizations.
A few weeks ago, we reviewed data from a customer that was seeing a measurable lift in engineering velocity thanks to Copilot. While analyzing the data, we noticed another interesting trend: Copilot users who were a part of a structured rollout had higher satisfaction and adoption levels than those who did not.
This company rolled out Copilot to its 2,000+ developers in two phases: first starting with a pilot group of early adopters who received support and training. Then followed by Copilot licenses being provided to the rest of the organization.
Several months later, they ran a developer survey to collect data on satisfaction (CSAT) and utilization levels. As shown in the charts below, there was a significant difference (17%) in the percentage of users who report being satisfied with Copilot (81% vs 64%).
Data analysis also yielded a correlation (r=0.39) between Copilot usage and satisfaction. This makes sense as developers with higher satisfaction with a tool will utilize the tool more—and it also reinforces the idea that satisfaction is a good signal and predictor of adoption.
The takeaway here is that structured Copilot rollout and onboarding can drive higher utilization, helping organizations fully capitalize on their Copilot license spend. Providing training and support requires effort, but the payoff is worth it.
Below are some examples of how to provide training and support. For more best practices on rolling out Copilot, check out my interview with Eirini from GitHub’s research team.
Training sessions that help users understand Copilot’s capabilities and key use cases
Interactive learning events, such as lunch-and-learns or “Prompt-a-thons”
Dedicated support channels for addressing user questions and resolving issues
A community channel where users can share experiences and tips with each other
Regular nudges to see if and how developers are using Copilot, and to offer support and encouragement
For more best practices on rolling out Copilot, check out our guide on measuring Copilot’s adoption and impact.
Who’s hiring right now
Here’s a roundup of Developer Productivity job openings. Find more open roles here.
Snowflake is hiring an Engineering Manager - DevEx | San Mateo
Pinterest is hiring a Senior PM - Infrastructure | San Francisco
Airbnb is hiring a Senior Staff Engineer - AI for Dev Productivity | US
That’s it for this week. Thanks for subscribing to Engineering Enablement. This newsletter is free so feel free to share it.
-Abi