The Journey to Continuous Deployment in Microservices
In 2009, Etsy did something radical. They started deploying changes not just weekly or daily, but several times a day. Let's break down why this was a big deal and how it made sense in their setup using microservices and continuous deployment.
What are Microservices?
Microservices are a way to design software applications as a collection of loosely connected services. Each service handles a specific function, making the entire system more flexible and easier to update.
Think of it like a restaurant where each dish is prepared by a specialist chef. If you need to change one dish, you just talk to that chef — you don't need to overhaul the entire kitchen.
Step 1: The Problem with Big Releases
When software teams bundle many changes together, releases can become huge and complex. Imagine moving a giant boulder (big release) instead of several pebbles (small releases). If something breaks, it’s tough to figure out what went wrong.
Big Releases: They Can Be Trouble
| Big Release: A Wall of Changes |
------------------------------------
| Change 1 | Change 2 | Change 3 | ... | Change N |With many changes, if something fails, finding the issue is like searching for a needle in a haystack. Everything gets mixed up.
Step 2: Continuous Deployment — The Etsy Approach
Etsy decided to do the opposite of big releases. They made changes small and frequent — like moving pebbles one at a time.
What is Continuous Deployment?
Continuous Deployment means automatically deploying changes to production as soon as they are ready. No waiting, no bundling with other updates. It's like having a conveyor belt where each part gets added as soon as it's cooked.
Etsy's Small, Frequent Deploys
Etsy did this using several techniques:
- Deployinator: A tool for rapid, frequent deploys.
- StatsD and Graphite: Tools for observing system behavior in real-time.
Step 3: Using Feature Flags
Imagine you’re experimenting with a new menu item at the restaurant. You prepare it, but only serve it to a few customers at first. If the feedback is good, you roll it out to everyone.
What is a Feature Flag?
Feature Flags are like switches in your code that let you turn features on or off without deploying new code.
if feature_enabled('checkout_redesign'):
use_new_checkout()
else:
use_old_checkout()This allows you to gradually introduce changes and see their impact before they affect everyone.
Step 4: Database Changes with Expand/Contract
Changing the database is more delicate. You don't want to break things for users. Etsy used a method known as expand/contract.
- Expand: Add a new field or column without removing the old one.
- Backfill: Populate the new field with data.
- Contract: Slowly transition operations to use the new field, then remove the old one when everything is stable.
This approach ensures a smooth transition without disruptions.
Step 5: Staying Alert with Observability
Every deployment requires careful monitoring. Observability is like a surveillance system in your restaurant kitchen. You watch every corner to ensure nothing goes wrong without your notice.
What is Observability?
Observability involves tools and practices to understand what's happening inside your system (like StatsD and Graphite). It's crucial to know when something goes wrong so you can fix it quickly.
Successful continuous deployment shifts the challenge from deploy day to watching and managing the system continuously.
The core lessons
- Smaller is safer: Frequent, small changes reduce risk and make solving issues easier.
- Feature flags offer control: They allow gradual feature roll-out and instant roll-back.
- Observability is key: With continuous deployment, staying alert to system behavior is vital.
- Expand/contract for databases: Makes database changes smoother and less risky.
Understanding and applying these principles helps ensure smooth, reliable updates to your software systems, making life easier for developers and better for users.