March 28, 2023

Etsy, a web based market for distinctive, handmade, and classic gadgets, has
seen excessive progress during the last 5 years. Then the pandemic dramatically
modified consumers’ habits, resulting in extra shoppers purchasing on-line. As a
end result, the Etsy market grew from 45.7 million patrons on the finish of
2019 to 90.1 million patrons (97%) on the finish of 2021 and from 2.5 to five.3
million (112%) sellers in the identical interval.

The expansion massively elevated demand on the technical platform, scaling
visitors nearly 3X in a single day. And Etsy had signifcantly extra clients for
whom it wanted to proceed delivering nice experiences. To maintain up with
that demand, they needed to scale up infrastructure, product supply, and
expertise drastically. Whereas the expansion challenged groups, the enterprise was by no means
bottlenecked. Etsy’s groups have been in a position to ship new and improved
performance, and {the marketplace} continued to offer a wonderful buyer
expertise. This text and the following kind the story of Etsy’s scaling technique.

Etsy’s foundational scaling work had began lengthy earlier than the pandemic. In
2017, Mike Fisher joined as CTO. Josh Silverman had lately joined as Etsy’s
CEO, and was establishing institutional self-discipline to usher in a interval of
progress. Mike has a background in scaling high-growth corporations, and alongside
with Martin Abbott wrote a number of books on the subject, together with The Art of Scalability
and Scalability Rules.

Etsy relied on bodily {hardware} in two information facilities, presenting a number of
scaling challenges. With their anticipated progress, it was obvious that the
prices would ramp up shortly. It affected product groups’ agility as that they had
to plan far upfront for capability. As well as, the info facilities have been
primarily based in a single state, which represented an availability danger. It was clear
they wanted to maneuver onto the cloud shortly. After an evaluation, Mike and
his workforce selected the Google Cloud Platform (GCP) because the cloud companion and
began to plan a program to maneuver their
many techniques onto the cloud.

Whereas the cloud migration was taking place, Etsy was rising its enterprise and
its workforce. Mike recognized the product supply course of as being one other
potential scaling bottleneck. The autonomy afforded to product groups had
precipitated a difficulty: every workforce was delivering in several methods. Becoming a member of a workforce
meant studying a brand new set of practices, which was problematic as Etsy was
hiring many new folks. As well as, that they had observed a number of product
initiatives that didn’t repay as anticipated. These indicators led management
to re-evaluate the effectiveness of their product planning and supply

Choosing a companion

Etsy had by no means labored with a software program improvement consultancy. To find a way
to scale product supply and usher in some exterior experience, they began
searching for a vendor. After an intensive search, they chose Thoughtworks,
primarily as a result of they perceived an in depth cultural match and deep alignment
relating to fashionable software program improvement.

Thoughtworks has an identical strategy to Agile as Etsy – being Agile
relatively than doing Agile. Etsy’s workforce was not seeking to undertake a
methodology that blindly follows strict rituals and practices, and wished
a companion that bases its strategy on agile rules and tradition.
Thoughtworks is understood for technical excellence and management in DevOps,
Steady Integration, and Steady Supply, all of which Etsy depends
on closely. As well as, the 2 corporations shared comparable rules
relating to worker care, variety, open-source, and know-how

The Thoughtworks workforce began by embedding into product groups, working with
each purchaser and vendor groups, on varied essential initiatives resembling cost
mannequin modifications, Search Engine Optimization (website positioning), and a notification platform.
This enabled Thoughtworks to know how Etsy labored, created a direct
influence on supply, and improved improvement practices resembling automated

For Thoughtworks, Etsy was fairly totally different from the standard shopper.
They’re a totally digital firm, and their technical course of and product
practises are very mature. Not like most purchasers who interact Thoughtworks,
Etsy had no giant transformation to undertake, and no ingrained habits
that wanted a significant shift. What they have been looking for was exterior perspective
and supplementive expertise, to assist them incrementally enhance and sort out
their urgent initiatives. The tradition and rules have been already very
aligned with Thoughtworks’ personal, so the partnership grew to become extra of an
trade of concepts drawn from totally different experiences.

Scaling Product Supply and Discovery

Thoughtworks and Etsy shaped a cross-functional Product Supply
Tradition (PDC) workforce to investigate the present product supply and discovery
course of. They analyzed the complete software program improvement worth stream,
wanting intently on the complaints of managers and workforce members. Chosen
occasions have been replayed to achieve a deep understanding of “how” it occurred
rather than who?. The Thoughtworks
workforce, as a result of they hadn’t lived via the issues, have been in a position to convey
an goal outsider perspective.

There have been quite a few issues Etsy was doing rather well:

  • Cross-functional groups: They structured their groups round “4 desk legs”:
    Product, Design, Engineering, and Analytics. All planning and supply practices
    occur with collaboration among the many teams.
  • Incremental supply of worth to customers: Steady supply is the core
    apply of their strategy. A code change sitting in a repository and never
    deployed to customers shouldn’t be beneficial, and successfully prices the corporate cash. Etsy
    deploys a whole bunch of instances a day, with a 30-minute decide to manufacturing lead
  • Options are prioritized utilizing information: Product managers and information analysts
    collaborate to find out an anticipated worth, measured by KPI modifications. If the
    function doesn’t ship sufficient worth versus its value, as evidenced by an A/B
    check, it isn’t rolled out, and the code is eliminated.

The workforce additionally uncovered alternatives for enchancment:

  • Every workforce was doing supply in several methods, having disparate processes
    and rituals, making onboarding new staff and altering groups tough. Some
    groups have been utilizing a course of like Scrum, whereas others didn’t use any formal
    course of.
  • There have been quite a few initiatives that had not created worth as
    they anticipated. For instance, Etsy tried a present wrap function that had
    decrease than anticipated adoption charges by sellers and patrons.
  • Previous to 2018, Etsy had been via a transitional interval, with
    pivots in product technique and management modifications main some product
    managers to expres a need for larger possession over their product

Time to studying

The workforce created an enchancment program, primarily based on lean pondering and the
concepts of Marty Cagan, who had lately offered at Etsy. The management workforce was
studying his ebook Impressed.

I promise you that at the very least half the concepts
in your roadmap should not going to ship what you hope. (By the way in which, the
actually good groups assume that at the very least three quarters of the concepts received’t
carry out like they hope.)

Marty Cagan, Inspired (p19)

After some preliminary analysis the workforce got here up with a metric they known as Time
to Studying – the time it took for a product workforce to validate an thought with a
buyer and acquire a greater understanding of its worth. That they had a baseline of
50 days that they wished to scale back.

They have been additionally different metrics.

  • Bettering the time to market
  • Bettering the effectiveness of merchandise (e.g. income, buyer
  • Worker happiness

To influence the KPIs, they got here up with quite a few resolution hypotheses:

Lightweight prototypes that incorporate extra direct consumer analysis

Etsy has robust experimentation infrastructure and analytics capabilities.
They run many concurrent A/B exams on {the marketplace}. The issue the PDC
workforce noticed is that the suggestions cycle may be fairly gradual. For an thought to be
prepared for an A/B check it needed to be close to production-level high quality. It additionally had
to have sufficient information to indicate statistical relevance. A/B exams would run for
months in lesser-used elements of {the marketplace}.

To set your expectations, robust groups usually
check many product concepts every week—on the order of 10 to twenty or extra per week. I
wish to emphasize that these are experiments, sometimes run utilizing prototypes.
A prototype shouldn’t be one thing that’s prepared for prime time and positively not
one thing your organization would attempt to promote and stand behind. However they’re
immensely helpful, as they’re all about studying quick and low cost.

Marty Cagan, Inspired (p27)

To scale back the Time to Studying metric, the PDC workforce utilized a lean UX
strategy. First, they began by shortly creating lo-fi prototypes and
instantly displaying them to customers. Second, the workforce elevated consumer suggestions
classes to each two weeks and decreased the extent of ceremony. Third, they
created a dual-track system, with a design and analysis workforce frequently
working forward of a supply workforce. The thought was to effectively acquire confidence
in an thought earlier than it qualifies for an A/B check. By lowering the price of
experimentation, they might enhance the range and amount of experiments,
discovering concepts that confirmed extra worth.

Determine 1: Experiment validation course of

Blueprint for product supply

The PDC, along side product workforce leaders, got here up with a set of
product supply rules and associated supply apply modifications. The
practices included: smaller extra targeted tales, redesigning story partitions to
higher match the workforce course of, utilizing spikes to scale back technical uncertainty, and
higher methods of managing tech debt. They didn’t wish to take away the autonomy of
groups. Steady enchancment is a giant a part of Etsy tradition. These practices
are supposed to be handled as wise defaults that may be tailored and improved
for every workforce’s particular context.

The PDC workforce first examined the apply modifications on pilot product groups to
set up their effectiveness after which iterated. As soon as the apply
demonstrated an enchancment, the workforce added to a data base and offered
gentle teaching to introduce the brand new practices to all product groups throughout

Lean Portfolio administration

Some giant initiatives have been nonetheless being carried out, regardless that there was
not clear proof of the initiative creating worth. That is the standard sunk
value drawback. To unravel this, the PDC workforce
advised taking a lean strategy to portfolio administration.

Our runway must be an inventory of hypotheses to check, not an inventory of
necessities to construct. Once we reward our groups for his or her means to ship
necessities, it’s simple to quickly bloat our merchandise with pointless
options—resulting in elevated complexity, increased upkeep prices, and
restricted means to alter. Options delivered should not a measure of success,
enterprise outcomes are. Our runway is a sequence of questions that we have to
check to scale back uncertainty and enhance our understanding of progress

Humble et al, Lean Enterprise (p96)

As an alternative of making prolonged mission specs and plans, the PDC workforce
coached product groups to develop final result statements for every initiative. With
final result statements, beginning with a buyer or enterprise final result permits the
workforce to experiment with many alternative concepts to attain the result by tying
them to measurable KPIs. The much less exact an final result is, together with its KPIs,
the tougher it’s to specify. When making ready for doable initiatives, every
product workforce created a two-page temporary summarizing the result, the measurable
influence, and the preliminary funding required.

The manager workforce prioritized the result statements, and every assertion
was seen as a “wager” amongst a portfolio. Impressed by the Lean Worth Tree
idea, the manager workforce would apply a daily cadence to find out if a
wager was paying off, giving them the chance to pivot early.

Determine 2: An instance initiative assertion

Stronger product & engineering collaboration

One other enchancment was to incorporate engineers early within the product ideation
course of and to have them be part of consumer suggestions classes. The workforce examined
engineers becoming a member of the speculation era and validation classes and
working intently with designers throughout design sprints. The outcomes of the check
confirmed a number of advantages. Engineers improved their understanding of the
buyer, constructing empathy for his or her wants. They might present on-the-spot
suggestions on the feasibility of an strategy and shortly create prototypes to
display concepts to customers. Cross-functional engagement allowed engineers to
contribute quite a few nice concepts that made it to the product.

After the preliminary PDC initiative Etsy continued to strengthen the
collaboration between teams. At each stage of the group, product and
engineering are companions, which suggests they’re in a position to have wholesome trade-off
discussions. The platform group expanded using product managers for
infrastructure and developer tooling groups – encouraging groups to consider
technical capabilities as merchandise and end-users being Etsy engineers. The
technical product managers do analysis and prioritization actions, simply as
they might for a buyer product.

What was the influence of product supply tradition?

Testing with pilot groups indicated that the Time to Studying went from 50
days to five days. Through the pilot, one product supervisor stated, “What I like about
the [direct user research sessions] pilot is that it lets us check our concepts
quicker, it was very nice to not spend as a lot time diving deep into
one thing that we didn’t even know we have been going to make use of.” An initiative on
stock high quality that was predicted to ship a seven million greenback income
enhance as a substitute delivered eleven million {dollars} in incremental worth utilizing
the PDC advised enhancements. Following this pilot success, the workforce coached
the complete product improvement group on the product supply strategy.

In accordance with Mike Fisher, “It is extremely onerous to isolate the advantages of a
course of change on a corporation of a number of hundred engineers since we don’t
run A/B experiments of this sort. Nevertheless, we did have a look at excessive stage
productiveness metrics after having PDC applied for over 2 years, and all of
our metrics have been very constructive. We had improved our velocity by doing this
work, and we have been in a position to produce in the end extra advantages for the patrons and
sellers – extra options, higher search, higher processes; the entire
market grew to become higher for patrons and sellers and that was the objective.”