Inside Etsy’s Multi-Year DevEx Initiative
An interview with former Etsy CTO, Mike Fisher, on the company’s initiative to improve their developer experience.
This is the latest issue of my newsletter. Each week I cover research, opinion, or practice in the field of developer productivity and experience. This week we’re covering “practice” by looking at how Etsy launched an initiative focused on improving their developer experience.
In 2020, Etsy kickstarted an initiative focused on their developer experience. The company was seeing historic growth, with plans to go from having 250 people in engineering to almost 1,000. The goal for this initiative was to help the organization maintain velocity as they grew.
Today it may be obvious to focus such an initiative on developer experience, but back in 2020 this wasn’t as common. Eager to learn more, I spoke with Mike Fisher, Etsy’s former CTO and the leader of this initiative, to understand why they focused on DevEx and how the initiative worked.
This is a summary of our conversation. You can listen to the full conversation with Mike here.
Abi: You were telling me that Etsy’s journey with developer productivity began with the pandemic. Take me back to that time. What was happening at Etsy and why did you kick off an initiative to tackle developer productivity?
Mike: In March of 2020, when the CDC issued guidance that people should wear masks, people turned to Etsy to buy masks. Our traffic nearly doubled overnight.
Fortunately we had just completed our migration to the cloud, so we could handle the massive surge in traffic. But after a while, we realized the traffic wasn’t letting up: new visitors were coming to Etsy for masks and then staying because they found all of these other wonderful things on the platform. That was great, but that also meant we needed to hire engineers aggressively to keep up.
We knew that hiring aggressively would cause its own issues. No matter where you’re scaling from — from five to 50, 50 to 500, or 500 to 5,000 — you’re going to have scaling challenges. Specifically, I’ve learned that you have to think about how people, process, and technology interact with each other. Scale any one of them without the others and you create waste. If you scale technology too early, you’re wasting money. Scale your processes too early and you’re bureaucratic. Scale people without investing in technology or process to support that growth, and your productivity takes a hit.
If you focus on one and not the others, you’ll be in trouble. So we knew that we needed to invest in process and tech to support our headcount growth.
Why Etsy focused on “DevEx”
A: You mentioned earlier that as you began identifying these problems, you started focusing on developer experience. At that time, I don’t think the term “developer experience” was widely used by companies focusing on developer productivity. How did that term come into the picture?
M: I don't know exactly who came up with it first, but Etsy has always thought about the human side of things. Our mission is to “keep commerce human”; we’ve always cared about supporting and helping humans.
When we started thinking about developer productivity, the conversation wasn’t just “hey let’s focus on making developers more efficient.” We knew there was something more holistic. By taking that approach we could not only maintain our velocity but also make developers’ lives better. So developer experience was a natural fit for us.
A: I have one more question about the concept of developer experience. I was recently having a debate about whether developer experience is a new approach or an old approach. What’s your take?
M: That’s interesting. I think this work has to be done, whether we organize a team to do it efficiently or we require engineers to do it on their own. The work has always existed.
For example, if my IDE is not working, I as a developer need to get it working. If there’s not a team that can pitch in and help me because they’re the experts on this, then I’ll need to figure it out myself. Either way, the work is still there. I think about other parts of DevEx in this same way. People are already doing this work in the background. Maybe as an industry we’ve just put a new name on it.
The four pillars of the DevEx initiative
A: I want to ask you about how the DevEx initiative worked. What were the core pillars or components of this initiative?
M: Initially, we approached developer experience as a single, broad initiative. However, as we delved deeper, we discovered that the necessary improvements spanned areas such as engineers' use of data, code deployment, and after-hours notifications. This led us to realize that the initiative crossed multiple engineering leaders' domains and required division into smaller, more manageable parts.
Specifically, we identified four core areas of work. One of the engineering leaders dubbed these areas "pillars," and the name stuck.
Help me craft products focused on making it easier to deliver features to both sellers and buyers on Etsy. The word “craft” is intentionally used, as Etsy has always thought of software engineering as a craft and not something that can be easily measured. In practice, this meant focusing on projects like modernizing their frontend so developers could more easily work in that environment.
Help me develop, test, and deploy focused on development environments, test patterns, and deployment tooling and processes. Etsy is famously a monolith, and one of the secrets to being able to scale a monolith in terms of traffic and number of developers is fast deployment.
Help me build with data focused on enabling product managers and engineers to leverage the company’s massive amount of data to experiment more quickly and guide next iterations.
Help me reduce toil focused on reducing tedious or repetitive tasks. For example, making sure there are accessible and updated runbooks and documentation for on-call engineers.
We had the trust of our product and business partners, enabling us to tackle all four pillars simultaneously. If you don't have the same situation, my advice would be to focus on one pillar, like "reducing toil," and work on it for a few quarters or sprints, sharing the positive results.
Etsy’s investment in the initiative
A: Another common question around these types of initiatives is the investment that goes into it. So what was the investment that was allocated towards the DevEx initiative?
M: When we did our multi-year journey to the cloud from 2017 until 2020, we dedicated about 25% of our engineering capacity to that initiative. So the company was familiar with that level of investment. In this case, it came in at around 20% of engineering capacity.
I’ll also add that there is often a tendency to underinvest in developer experience because it can be challenging for leaders to demonstrate the impact of such work to executives. Two strategies I recommend are 1) translating estimated results into financial impact and 2) showcasing incremental results quickly. When discussing engineering projects with business partners, focus on incremental revenue or reduced costs. For example, instead of explaining that you want to change the way you deploy code using packages, tell the CEO that the work will save 50% of engineering hours spent waiting for deployment, translating to 500 hours of saved engineering time or roughly a quarter of a full-time engineer. This approach is more likely to pique their interest. Once you've captured their attention, don't wait until the project's completion to provide updates. Share incremental results every month or quarter through status emails or meetings, even if the progress is simply lessons learned.
A: Figuring out how to talk about the potential payback for an initiative like this can be a challenge some tech leaders run into. What's your advice for doing that?
M: Most of the time you’ll be the most technical person on the executive team, so as soon as you start talking about something technical, the other execs’ eyes are going to roll back in their head. It doesn’t make sense to them. So you have to translate what you’re saying into a language they do understand, which is typically finance. Money. I’m not a huge fan of monetarily-driven metrics, but if you use them to translate the impact of an initiative on the business, they help.
We were able to do that with deploys, for example. If you can take a deploy from 15 minutes down to 7 minutes, we can see how substantial that is by taking into account the average salary for engineers, how frequently they deploy, and the time saved that could be every week. This calculation doesn’t account for the cost of disrupting flow. It also is harder to calculate for things like reducing pages that disrupt developers’ evenings so they’re less productive the next day. But that’s why I call these calculations “back of the envelope": they don’t have to be precise to communicate the opportunity to the executive team.
You can also look at longer-term problems like attrition. If someone leaves my company, my estimate is you lose at least 6 months of productivity, or 12 if they’re senior. You can start telling your leadership team: “the industry average is 10% attrition per year. If we can cut that in half to 5%, we save X months of productivity.” These numbers really start to matter as you have more engineers.
The important point is to translate the impact of these problems into numbers that everyone can understand. And know that it doesn't have to be precise; it can be rough math.
How projects were carried out
Abi: I want to transition now and ask you about who was actually leading and driving this work. Was it existing teams across the company doing the work, or did you put together dedicated groups to focus on this work?
M: The initiative was mine as CTO, and each of the four DevEx pillars were owned by a VP. There was a VP focused on the data pillar, another focused on reducing toil, and so on. Each of the VPs built teams around their pillar.
As for dedicated teams, we tried and failed with platform teams a few different times for various reasons. We ultimately realized that there is a place for these centralized teams — but we didn't call them “platform” teams, we actually called them “enablement” teams. That way, right off the bat people would know what their purpose was. Their purpose is to enable the teams that are closer to the customer to be more efficient.
So the first step for us was to call them enablement teams and help them understand what their primary purpose was. The second thing we found to be really successful was to put product managers on those teams. We hadn’t done that in the past, and when you don’t, you’re then asking the engineering manager to do multiple jobs. Not only do they have to manage their teams and projects, but they also have to meet with their customers to understand what should be on their roadmap. Having product managers removed that burden and helped enormously.
Measuring the initiative’s success
A: What signals did you use to determine whether the initiative was successful?
M: Measuring the success of such initiatives with hard metrics can be challenging, but it's still important to try. As leaders, we spent considerable time initially brainstorming and debating the right metrics to gauge success. We used various metrics, including deploy times and an NPS-like developer score. The primary measurement I monitored was experimentation velocity, or the number of experiments launched per team per time period (usually a month). Maintaining the same velocity as your team grows should be viewed as a success, considering the increased communication channels (Mythical Man Month) needed for coordination.
Another indicator of success is when an initiative like Developer Experience becomes "evergreen" or simply part of the organization's standard workflow, which is what happened at Etsy.
That’s it for this week! Share your thoughts and feedback anytime by replying to this email.
If this email was forwarded to you, subscribe here so you don’t miss any future posts:
-Abi
Thanks Abi. That was a really interesting article!