How Spotify maintains team autonomy at scale
An in-depth look at Spotify’s strategies for maintaining team autonomy and the challenges they’ve faced.
This is the latest issue of my newsletter. Each week I cover the latest research and perspectives on developer productivity.
This week I read Decentralized decision-making and scaled autonomy at Spotify by researchers at the Blekinge Institute of Technology. This paper reports on how team autonomy is maintained at Spotify, including the degree to which autonomy exists in the company, barriers to team autonomy, and mechanisms to improve it.
My summary of the paper
Finding the right balance between team autonomy and maintaining necessary rules and structures is challenging. Spotify has openly shared their journey in this area, starting with the viral Spotify Model and continuing with various talks (Yip, Ivarsson, Sundén) and publications (HBR, Smite, Kniberg) that show how the model has evolved. Since the industry still struggles with understanding how to distribute decision-making and enable autonomy at scale in large organizations, the authors of this paper aimed to provide an up-to-date and thorough report on Spotify's current practices.
To understand how Spotify manages team autonomy at scale, researchers conducted a detailed study using data from various sources, including interviews with squad managers, retrospectives conducted with squad members, surveys, pull request data, and information from the company’s employee database. Their goal was to see how much control teams have, what challenges they face, and what strategies help them stay autonomous while still working well together.
Here’s what the study found:
How much autonomy teams have at Spotify
The researchers started by developing a diagram representing the distribution of authority at Spotify. This was developed after coding data from retrospectives and interviews, and verified with representatives from management.
Here’s a summary of what leaderships decides, what squads can decide, and where compromises are made.
Leadership has control over decisions related to how the organization is structured and plans for growth. In turn, tribe leaders plan and lead recruitment efforts, especially when hiring and firing managers. Senior leadership also has control over strategic decision regarding the long-term vision and key priorities. As part of this, senior leadership reviews new initiatives quarterly and sets priorities for strategic work, which are called company bets.
Squads can independently make decisions regarding work execution. To a large extent, squads independently decide how to code a feature or service, with few exceptions.
In theory, squads are able to choose the technology best suited for solving their tasks. However there’s a need to align the use of technologies, languages, and infrastructure. For that reason, Spotify established a central committee of very senior engineers who own the list of accepted technologies on their TechRadar. They also have a well-defined process for proposing technologies and trying them out.
Squads are also constrained by technical dependencies with other squads, of which there are “surprisingly many.” That is despite the company’s attempts to decouple the architecture by design (microservice architecture principles) and by action (code ownership principles and continuous re-engineering). These technical dependencies are a known major challenge for autonomy. When a squad’s work impacts other squads, engineers are expected to consult the others either informally or through formalized action. Large changes are documented via RFCs, and smaller changes are handled through pull requests.
Squads also have control over decisions related to:
Managing and monitoring processes and progress,
Recruiting or firing squad members, and
Work planning and prioritizing.
To clarify that last bullet: squads do not decide which projects or product areas they work on, but they do have relatively large freedom to contribute and innovate within the business area they belong to.
Interestingly, the researchers also noted that the well-performing squads were perceived to have higher levels of autonomy, while the challenged squads were less autonomous and had more involvement from management.
There are also work groups that come together to discuss and pilot new processes or practices. For example, working groups at Spotify have produced a template for squad retrospectives, brought together groups of managers to learn from each other, introduced No-meeting-Wednesdays, monitor quality practices, and introduced the aforementioned TechRadar.
Barriers to autonomy
During their study, the researchers also examined barriers to squad autonomy beyond managerial control. They identified the following factors to negatively impact autonomy:
Squad immaturity (lack of knowledge redundancy, lack of competence)
Injured teamwork and failure to self-manage (too broad scope, unclear goals, missing process improvements, lack of process for getting things done, poor onboarding)
Architectural and legacy constraints, technical debt, the need to compromise with old systems and clients, and undocumented tribal knowledge
Weak ownership principles lead to squads depending on others
Mechanisms for maintaining autonomy at scale
Spotify employs several strategies to manage autonomy at scale while ensuring alignment and coordination among its squads. Here are the methods the researchers identified:
1. Formalized code ownership
The absence of code owners hinders squads, so Spotify emphasizes formalized code ownership. Engineers are allowed to change code owned by other squads, but because there is an owner, there is a clear point of contact for questions as well as a clear quality guard when changes emerge.
The tradeoff with this approach is that it can lead to engineers waiting on others to review their pull requests. In the study, engineers agreed that this could be addressed by establishing better contracts between squads specifying mutual expectations and responsibilities, or just better processes for getting the pull requests done.
2. Strategies for creating alignment
Team autonomy at scale requires constant work to create alignment. Here are the practices that are working for Spotify:
Using RFCs for the collective review of important changes
Promoting work groups that provide communication, authority, and control, such as the Technology Architecture Group that owns the TechRadar
Constrained choice of technologies
Golden Paths, which create alignment through tutorials and templates
3. Encouraging information sharing
Spotify encourages interaction, socialization, and knowledge sharing through practices like embedding, lunch and learns, encouraging on-site visits, and designing office spaces to facilitate networking.
All of these strategies help Spotify empowering its squads with a large degree of autonomy, even at scale.
Final thoughts
Many organizations aim to maintain a sense of speed and agility as they grow—this paper provides an in-depth look at one company's approach. While many of the strategies are familiar, the paper is inspiring for two key reasons. First, it demonstrates the impact that leadership can have in clearly defining how various decisions should be made. Second, it highlights the power of working groups or internal-facing teams in increasing team autonomy and agility. In this case, these teams led initiatives such as the Tech Radar, Golden Paths, monitoring quality practices, and creating more time for deep work.
Who’s hiring right now
Here’s a roundup of Developer Productivity job openings. Find more open roles here.
Citi is hiring a Director - Engineering Excellence | Texas
Costco is hiring a Platform Engineer | WA, TX
DoorDash is hiring multiple Developer Platform roles | CA, WA, NY
Opendoor is hiring a Senior Software Engineer - DevEx | Canada
REI is hiring an Senior Manager - Platform Engineering | Seattle
Samsara is hiring a Staff Software Engineer - Infrastructure | London
Sentry is hiring an Engineering Manager - DevEx | California
That’s it for this week. If you know someone who might like this issue, consider sharing it with them:
-Abi