Software Architecture in the AI & Cloud Era

← the whole book on one page

Chapter 5: Microservices — Building, Operating, Succeeding

Microservices are an organizational technology that happens to run on computers.

Encore has grown. Three years after Chapter 1, it has 70 engineers in nine teams, and the modular monolith that served nine people so well has become a machine for coordination. Releases ride a shared deploy train, so a finished change waits for everyone else’s. Freeze windows stop every team at once, whether or not its own work is risky. And more and more of each team’s week is spent waiting on another team to review, merge or release something. None of this is a failure of the monolith’s code. It is a failure of arithmetic, and Fred Brooks described that arithmetic in The Mythical Man-Month (1975). In a group of n people there are n(n − 1)/2 possible pairs who may need to talk, so the paths of communication grow roughly as the square of the group’s size. Nine people have 36 such paths; 70 people have 2,415. Adding people adds conversation faster than it adds work.

That arithmetic is the real problem microservices exist to solve. Their purpose is to let many teams change one product without asking each other’s permission, which is to say, to cut the number of conversations a single change requires. If the Catalog team can ship a new search filter on Tuesday without consulting the Orders & Payments team, a conversation has been removed from that change, not merely made shorter. Measured against that purpose, the reasons more often given (high request volume, or the appeal of a fashionable line on a résumé) are weak ones.

The independence is not free. Every benefit microservices offer is paid for in operational complexity: more deployables to build and release, more network calls that can fail, more dashboards to watch, more versions to keep compatible. This is Chapter 1’s first law of software architecture, that everything is a trade-off, now charged at fleet prices. The worst bargain is the one many teams strike without noticing: the distributed monolith. It is a system split into services that are still so coupled they must be changed and deployed together, in lockstep. Such a system pays every cost of distribution (the network, the operations, the partial failures) and collects none of the benefit, because no team can move alone.

We have been here before. The service-oriented architecture (SOA) of the 2000s, as Chapter 2 told, also promised independent services, and often delivered a middleware committee that every change had to pass through. Microservices are relearning SOA’s lessons with smaller pieces, and the first lesson is that independence must be designed into the boundaries; it cannot be declared by drawing boxes and calling them services.

5.1 Modeling and Communication

Designing for independent deployment

Let us begin with a definition precise enough to argue with. A microservice is an independently deployable unit of business capability: a piece of the system that one team can change, test and release on its own schedule, and that does one coherent thing the business would recognize. The word that carries the weight is independently, and here lies the first surprise for teams new to the style. Independence is not something we achieve at deployment time, with clever pipelines and container images. It is decided much earlier, at the whiteboard, when we decide where one service ends and the next begins. By the time anything ships, the outcome has already been settled.

Where, then, should the lines go? The answer this book has been building toward is Chapter 3’s bounded contexts. A bounded context follows a real seam in the business, a place where the language, the rules and the people change. Because the business itself rarely needs to change on both sides of such a seam at once, a service drawn along it can usually change alone. A new refund policy belongs to Orders & Payments; a new seating layout belongs to Seat Inventory & Ticketing; neither change needs the other team’s permission.

The alternatives fail in instructive ways. One is to cut by entity, giving us a “User service,” a “Ticket service” and an “Event service.” This is Chapter 1’s entity trap, now repeated at the scale of a whole fleet. The trouble is that a business change almost never concerns one noun. Consider a modest request: let verified fans transfer a ticket to a friend. That touches the user (who is verified), the ticket (who owns it now) and the event (whether transfers are allowed for this show). In an entity-cut fleet, the feature makes a tour through three repositories and lands in two or three teams’ sprint planning, and nobody can ship until everybody has. The other alternative, cutting by technical layer (a front-end service, a business-logic service, a data service), fails the same way for the same reason: every feature runs vertically through every layer, so every feature touches every team. In both cases the services are separate on the diagram but joined in every change.

A city offers a useful picture of the difference. Imagine a city built as neighborhoods, each with its own shops, school and clinic: most of daily life happens within a short walk of home, and the roads between neighborhoods stay quiet. Now imagine a city zoned strictly by function, with every bakery in one district, every school in another and every house in a third. Nothing is wrong with any single district, yet everyone must drive across town every morning, and the city’s real business becomes traffic. Services cut by entity or layer build the second city, and their traffic is network calls: each one slower than a local function call, each one able to fail. Here, though, the analogy breaks in our favor. Streets are poured in concrete and last for centuries, while service boundaries can be redrawn. Redrawing them is not free (data must migrate, contracts must be renegotiated, teams must be reorganized), but it is possible, which means a wrong first cut is a cost to be paid, not a sentence to be served.

Encore’s own cut reads almost exactly like its context map from Chapter 3. There are seven services: On-Sale Admission (the Sale Gate that meters fans into an on-sale), Seat Inventory & Ticketing, Catalog, Orders & Payments, Support, Notifications, and Bot Screening. Sales Analytics is deliberately not on that list. It remains an event consumer (Chapter 9): it listens to what the other services announce, such as seats sold and orders paid, and builds its reports from that stream, but no service ever calls it and waits for an answer. That keeps it off every critical path, so a slow report can never slow a sale. The mapping from the past is not one-to-one, either. Chapter 1 sketched eight components; passed through Chapter 3’s bounded contexts and three years of watching where the business actually changes, they have become seven services. The new cut is truer than the old one for a simple reason: it is drawn from evidence of how Encore really evolves, not from a first guess.

The organization follows the same lines. Seven stream-aligned teams each own one service, where a stream-aligned team is one organized around a flow of business work rather than around a technology. A further team runs the platform the others build on, and one builds the fan app, which makes nine teams in all. No service is shared between teams, and that rule is stricter than it sounds. A service with two owning teams has, in practice, none: each assumes the other is watching the alerts, each hesitates to make a change the other might object to, and decisions stall in the space between them. One consequence of these lines deserves attention. Seat Inventory and Orders, which Chapter 3 deliberately kept in one deployable, now belong to separate teams and therefore to separate services with separate databases. While they shared a deployable, reserving a seat and recording the order could happen in a single database transaction, all or nothing. Across two services that guarantee is gone, so it must be rebuilt in another form, the saga of Section 5.2. Notice what has happened: a decision about who works on what has reappeared as a problem in transaction design: an organizational decision has become a technical one. It will not be the last time in this chapter.

One rule completes the modeling: information hiding, Parnas’s criterion from Chapter 1. Parnas taught us to draw module boundaries around decisions likely to change, and to hide each such decision behind an interface so that changing it disturbs nobody else. For a service, the largest such decision is its database: which tables exist, what the columns are called, how the data is shaped. That schema should be as private as a class’s fields are private to the class. The reason is mechanical. If the Support service reads the Orders tables directly, then the Orders team can no longer rename a column, split a table or change a type without first checking with Support and then releasing in lockstep with them. The two services have become one deployable in all but name, and they stay that way for as long as the shared tables remain.

Getting out of a shared schema is usually the hardest part of any migration to services, and the order of operations matters. First give every table exactly one writing service, its owner; other services may still read it for a while, but only the owner changes it. Then build bridges while the data moves to where it belongs: database views that present a stable shape to the remaining readers, or change data capture (CDC), which streams every committed change from the owner’s database so that other services can keep their own copies up to date (Chapter 9). Only when no one reads the old tables directly is the boundary truly closed. In my experience most distributed monoliths are born here, in a shared database, long before anyone draws a service diagram. The diagram shows separate boxes; the database knows better.

The communication decision

Once the boundaries are drawn, every line between two services raises a second question: how do they talk? There are three modes to choose from, each with a situation it suits and a cost it charges for as long as the link exists. Because this choice is made again for every single link, and Encore has many links, it pays to have a default and to depart from it only for a reason:

Figure 5.1 — The three communication modes, each with the situation it suits and the cost it keeps charging. The default: events for facts, synchronous calls for moments of truth and orchestration for named processes. Encore has very few moments of truth; Exercise 4.3 asked you to find the main one, seat reservation.

Each row deserves a word of explanation. A synchronous call is right when the caller genuinely cannot continue without the answer. When a fan taps a seat, the system must know, then and there, whether that seat is now theirs, because the next screen depends on it. The price is temporal coupling: the caller can only work while the service it calls is working too. Chain several such calls together and the availabilities multiply, as Chapter 4 showed, so the chain is always less available than its weakest link. Suppose three services are each up 99.9% of the time: a chain through all of them is up only about 99.7% of the time. Events suit the opposite situation, where other services need to know that something happened but have no say in whether it did. When a seat sells, Seat Inventory publishes SeatSold; analytics updates its counts and Notifications sends the confirmation email, and the sale does not wait for either of them. The price here is eventual consistency (for a short while, the analytics dashboard is behind the truth) and the extra effort needed to trace a flow that no single piece of code describes. An orchestrated workflow fits a named, multi-step process, such as a purchase that must reserve, charge and then issue, where someone must be accountable for driving each step and for cleaning up when one fails. Its price is the orchestrator itself, one more component to run, monitor and keep healthy.

Beneath the table lies a simpler distinction, and it is the one I would ask you to remember. A synchronous call asks, and the caller waits for the reply. An event tells, and the publisher moves on with its work. Most of what Encore’s services need from one another is of the second kind: Notifications needs to hear that a ticket was issued, Support needs to see that an order was refunded, and none of them needs to be consulted first. A fleet that asks when it could simply tell ends up with services standing in line behind one another, each waiting for the next, and a slowdown anywhere in the line becomes a slowdown everywhere. The habit to build is to tell by default and to ask only at the few moments when the answer truly matters.

Recap. Services are bounded contexts made deployable, and whether they can really deploy independently is won or lost when we model them, not when we ship them. Each service keeps its database private, because shared tables quietly fuse services back together. How services communicate is decided link by link, and since most of Encore’s links carry facts that others only need to learn, events serve most of them best, with synchronous calls kept for the few moments of truth.

Exercise 5.1. Take two services, either from a system you know or from Encore, and list every way a change in one forces a deploy of the other. Label each coupling by its kind: schema (shared tables or data formats), timing (one must be up for the other to work) or shared library (common code both must upgrade together). Then decide which coupling you would sever first, and explain why.

5.2 Workflow, Transactions, and Contracts

Sagas

Splitting Seat Inventory & Ticketing and Orders & Payments into separate services, as Section 5.1 did, costs us something we used to get for free: a single transaction around a purchase. Consider what buying a ticket now involves. Seat Inventory & Ticketing reserves the seat and, later, issues the ticket. Orders & Payments charges the fan’s card through a payment service provider (PSP), a company outside Encore altogether. That is two services, each with its own database, plus an external party, and no database transaction can span all three. If everything lived in one database, we could wrap “hold the seat, take the money, issue the ticket” in a single transaction and trust that either all of it happened or none of it did. That promise is gone, and we need something to put in its place.

The something is the saga. A saga is a sequence of local transactions, each committed on its own in one service’s database, where every step comes paired with a compensating action: a step that undoes the business effect of the earlier one if some later step fails. Hector Garcia-Molina and Kenneth Salem described the idea in 1987, and, interestingly, not for distributed systems at all. They were concerned with long-lived transactions inside a single database, the kind that would hold locks for minutes or hours and choke everyone else. Their answer was to break the long transaction into short ones and accept that, on failure, you clean up by compensating rather than by rolling back. The idea then waited the better part of three decades for microservices to make it famous, because microservices turned every business process into exactly that kind of long, multi-part transaction.

The best way I know to feel the difference between a rollback and a compensation is to book a trip. Suppose a traveler books a flight, a hotel and a rental car from three different companies, and the car turns out to be unavailable. Nobody turns back the clock; the flight and the hotel were really booked, and both companies know it. Instead, the traveler cancels the hotel and claims a refund on the flight, and each company applies its own rules: perhaps the hotel cancels for free, and perhaps the airline keeps a fee. Those rules are business policy, not technical mechanics. That is the heart of the matter: a compensation is a business act, and the person who knows what a cancellation ought to cost, whether a fee applies, whether the fan gets an email, is rarely the engineer writing the code. So compensations must be designed with the business, not improvised by developers.

Figure 5.2 — An orchestrated saga compensating a declined card. The seat was held, then the card was declined. Compensation cannot undo the decline, which has already happened; instead it takes a new step forward, releasing the hold so the seat makes business sense again. That is why compensations are designed with the business rather than invented in code review.

Not every step in a saga can be compensated, and the design turns on knowing which can. Every saga has a pivot: the point of no return. In Encore’s purchase, the pivot is the successful charge. The steps before it, such as holding the seat, must be compensatable, because if the charge fails we have to be able to walk them back (release the hold). The steps after it, such as issuing the ticket, cannot simply be walked back, because once the fan has paid we owe them a ticket; instead, they must be retriable until they succeed. An arch makes a good picture of this. Its keystone is the last stone set at the top. Until it is set, the stones can be lifted away one by one; once it is set, the arch stands, and the only work left is to finish it. The pivot is the saga’s keystone. One further rule applies to every step and every compensation: each must be idempotent (Chapter 4), meaning that running it twice has the same effect as running it once. This matters because the component driving the saga will retry steps whenever it is unsure whether they finished, and a retried “charge the card” must never charge twice.

Sagas also leak, and the leak is worth understanding before it surprises us. A database transaction is isolated: its intermediate work stays hidden from everyone else until it commits, so other readers see either the old state or the new one and never the mess in between. A saga gives that up, because each of its local transactions commits immediately and is visible to the world at once. Here is what that means for fans. Suppose a fan’s purchase of seat 14B holds the seat, and then the card is declined. For the moments between the decline and the compensating release, every other fan looking at the seat map sees 14B as “held”, for a purchase that has already failed. Usually that is a brief annoyance, but under an on-sale it can also lead other steps to act on state that is about to change.

There are three standard techniques for containing this leak, and we choose among them step by step; Encore’s seat map uses all three.

• A semantic lock is a pending status that the saga writes and that readers agree to honor. Instead of showing 14B as simply taken, the map shows it as “being purchased”, and other flows know not to treat that status as final in either direction.
• Commutative updates are updates designed so that their order does not matter. Adding one to a count and then subtracting one gives the same result as subtracting first and adding afterward. If the updates two sagas make to the same record commute, their interleaving cannot leave that record in a corrupt state.
• Re-read-and-verify has the final step re-check its assumptions before it acts, treating anything read earlier in the saga as possibly stale. Before issuing the ticket for 14B, for example, the last step confirms that the hold is still this fan’s.

Someone, or something, has to move a saga from step to step, and there are two ways to arrange that. In orchestration, one accountable owner drives the steps, as a conductor directs an orchestra: it tells each service what to do next and decides what happens on failure. In choreography, there is no conductor; each service listens for the previous service’s event and reacts to it, like dancers moving on each other’s cues. Choreography has no central bottleneck, but it has a matching weakness: the dance as a whole is written down nowhere, and to learn what happens end to end you must read every dancer’s part. Encore chooses by what is at stake. It orchestrates the flows that touch money and carry deadlines and dunning (the scheduled retries and reminders that chase a failed payment), because those need one owner who can be held to account and asked “where is this order?” It choreographs the propagation of facts, such as letting other services know that a ticket was issued, where no single owner is needed. The orchestrator itself is usually a durable-execution engine (Temporal, AWS Step Functions and similar), which persists every step as it completes, so a workflow can pick up exactly where it left off after a crash or a redeploy. Chapter 6 covers both styles in depth.

Service contracts

Everything we have gained from splitting services rests on independent deployment, and independent deployment ends the day service A’s release breaks service B. At that point the two teams must coordinate their releases again, and we are back to a distributed monolith. What protects us is the contract between services. It helps to think of that contract as closer to a treaty than to a function signature. Inside one codebase, if you change a function’s signature, the compiler finds every caller and you fix them all in one commit. Between services there is no such luxury: the contract is an agreement between parties who cannot be made to move together, each deploying on its own schedule. So it has to be designed to change gracefully.

Two practices make that possible. First, schemas must be explicit, written down, rather than implied by whatever the code happens to send. Second, they must evolve by expand–contract. In the expand phase, you may add new optional fields freely, because no existing consumer depends on them. Removing a field, or changing what an existing field means, comes only in the contract phase, after every consumer has migrated away from the old form. Suppose Orders & Payments wants to replace a single amount field with amount plus currency. It first adds currency alongside the old field, waits while each consuming team updates its code, and only then retires anything old. Engineers widen a busy bridge the same way: they build the new lanes first, shift the traffic onto them, and demolish nothing until the traffic has moved.

The consuming side has duties too. Consumers must be tolerant readers: they read the fields they need and ignore any they do not recognize, so a provider’s new optional field cannot break them. This is a descendant of Postel’s robustness principle, and it has a price that Chapter 7 weighs. To find out before release whether a change breaks anyone, we can use consumer-driven contract tests. Each consumer publishes the specific things it relies on (these fields, these types, this behavior), and the provider runs all of those expectations in its continuous integration (CI) pipeline. If Orders & Payments makes a change that would break Notifications, its own build fails before the change ships. The question “did we break anyone?” becomes a test failure instead of a meeting.

One tempting shortcut undoes all of this: a shared “common model” library, a package of domain classes such as Order or Fan that every service imports. It looks like reuse, but it couples every service that imports it, because any change to the shared model forces every importer to upgrade and redeploy together. SOA’s canonical data model (Chapter 3) taught that lesson at scale: a model everyone shares, no one can change. The rule that follows is simple. Share primitives and protocols, such as money types, identifiers and message formats, and keep domain models inside their own services.

Recap. Once a purchase spans several services, sagas replace the distributed transaction: a sequence of local transactions with compensations designed with the business, a pivot after which steps are retried rather than undone, and techniques such as semantic locks to contain the loss of isolation. Encore orchestrates accountable, money-touching processes and choreographs the spread of facts. Explicit schemas that evolve by expand–contract, tolerant readers and consumer-driven contract tests in CI keep services independently deployable.

Exercise 5.2. Design the saga for Encore’s refund. It has three parts: reversing the payment, re-listing the seat for sale if more than 48 h remain before showtime, and invalidating the ticket. Put the steps in order and give each a compensation or a retry policy. Which step is the point of no return? What compensates a failure after it?

5.3 Delivery — Build, Test, Observe

Making deployment routine

Independence that has been designed into the services still has to be exercised, and the place it is exercised most often is the deployment pipeline. In a healthy fleet, deploying is so ordinary that nobody mentions it anymore. Each service has its own pipeline, which carries a merged commit all the way to production in under an hour. Nobody has to coordinate with another team first, no release train has to be caught, and no freeze window has to be waited out. That is the payoff the chapter opened with: when the Orders & Payments team fixes a bug at ten in the morning, fans have the fix before lunch, and no other team needed to know it happened.

Speed alone does not take the fear out of deploying, though. A team that can ship in an hour can also ship a mistake in an hour. Progressive delivery deals with that remaining fear by never exposing everyone to a new version at once. The new version first runs as a canary: it receives a small share of real traffic, a few percent, while the old version keeps serving everyone else. If the canary behaves, its share widens step by step until it serves everyone. All the while, the pipeline compares the canary’s error rate and latency with the old version’s, which serves as the baseline. If the two diverge, an automated rollback sends all traffic back to the old version, with no human in the loop. The name comes from the canaries miners once carried underground. The bird was more sensitive to bad air than the miners were, so it fell ill first, while there was still time to get out. A software canary plays the same part, failing first and on a small scale. The analogy breaks in one important place: the canary’s traffic is not a bird but real fans trying to buy real tickets. Every minute a bad canary keeps running, some of them are failing to pay, and that is why the rollback is automatic rather than waiting for someone to notice a graph.

The second tool, feature flags, decouple deploy from release. Deploying means putting new code on the servers; releasing means turning on the behavior that code contains. With the new code wrapped in a flag, the two become separate decisions, made at different times and often by different people. Engineers decide when the code is safe to deploy; product owners decide when fans should see it. Here is how that plays out at Encore. The team ships the new seat picker dark on Tuesday: the code is in production, but its flag is off, so no fan sees it. On Thursday the product owner turns the flag on for 5% of fans and watches how they get on. If something goes wrong, turning the flag back off is a configuration change that takes effect at once, with no redeploy. And because no big switch-over is scheduled for Friday night, nobody works the weekend.

Testing a fleet of services

Testing changes shape once one product becomes many services, but the old guidance still applies. Mike Cohn’s test pyramid says to write many small, fast tests at the bottom and only a few large, slow ones at the top, and it still holds; distribution just raises the price of every layer above the first. At the base, unit tests check a single function or class. They stay cheap and plentiful. One level up, service tests run a whole service with its neighbors replaced by stubs, simple stand-ins that return canned answers. A service test of Orders & Payments, for instance, exercises the real order logic against a fake payment provider and a fake Seat Inventory. These tests carry most of the confidence, because they test everything a team owns and nothing it doesn’t. Contract tests (Section 5.2) guard every boundary between services, so that no service can change a message another one depends on without a test failing.

At the top sit end-to-end tests, which run a real flow across many real services at once. In a fleet they are slow, because every service in the flow has to be running. They are flaky, because any one service, network hop or piece of test data can fail them for reasons unrelated to the change under test. And they are politically expensive. When an end-to-end test fails, it rarely says which service caused it, so each failure starts a debate among teams about whose fault it is, which is exactly the kind of cross-team conversation microservices were meant to remove. The advice, then, is to keep only a handful of them, written as journeys that matter to the business (“a fan buys a ticket”), and to push every other check down the pyramid, to a level where a failure points at one team.

Even a well-built pyramid leaves gaps, and the last layer of testing happens in production itself. Testing in production sounds reckless, so it is worth being precise about why it is needed. Contract tests check the shape of messages but not their meaning. A contract test can confirm that a field called amount is a number; it cannot confirm that both services agree whether that number is in euros or cents. Staging has a matching gap. No staging environment reproduces a Friday on-sale, with its real traffic, real payment providers and a crowd of fans all arriving in the same minute. So the canary, kept small, becomes the final integration check: the first time the new version meets the real world, and only a few percent of fans are exposed to it.

Encore adds a second production check, synthetic buyers. These are scripted fans that run the real purchase path every hour, from finding a concert to paying for a seat. They buy against a hidden test event that no real fan can see, and they pay through the providers’ sandbox, so no money moves. Because the event is hidden and its sales are kept apart from real ones, they never hold a seat a real fan could have bought, and they never inflate the sales numbers on anyone’s dashboard. What they do catch is the class of failure that only appears in production, such as an expired certificate, a misconfigured route or a payment provider that changed its behavior overnight, and they catch it within the hour, usually before a real fan runs into it.

From monitoring to observability

Tests tell us whether a change is safe before and as it ships; once it is running, we need to see what it is doing. Here two words are often used interchangeably, and the difference between them matters. Monitoring answers questions we knew to ask in advance: “Is CPU usage high?” “Is the error rate above 1%?” Someone decided ahead of time to measure those things and set an alert. Observability answers questions nobody thought to ask until something went wrong: “Why have Dutch fans’ payments been slow since 14:02?” No one built a dashboard for Dutch payments after 14:02, so the answer has to be assembled from whatever the system already records.

The word comes from control theory, where Rudolf Kálmán gave it a precise meaning around 1960: a system is observable if its internal state can be inferred from its outputs alone. The software sense keeps that idea. A service is observable when an engineer can work out what it is doing, and why, from what it already emits, without shipping new code to ask the question. If answering “why are Dutch payments slow?” requires adding a log line and waiting for a deploy, the service was not observable; it was merely monitored.

That ability rests on three kinds of output, each suited to a different question. Structured, correlated logs record individual events as data rather than free text, with fields such as order ID, country and payment provider, so they can be filtered and grouped (“all payment events from the Netherlands since 14:00”). They are correlated because each entry carries the identifier of the request it belongs to. Metrics are cheap numerical aggregates, such as request counts, error rates and latency percentiles. Metrics are what dashboards and alerts are built from, because they summarize millions of requests in a few numbers. Traces record the path a single request takes through every service it touches, with the time spent at each step. A trace is what shows that a slow payment spent most of its time waiting on one provider call deep inside Orders & Payments. In a monolith, a stack trace often tells us where a problem lies. In a fleet, one fan’s purchase crosses several services, and no single service’s logs tell the whole story. That is why, without traces, debugging a fleet comes down to guesswork.

Observability cannot be left to each team’s good intentions, because a single service that does not pass the trace along breaks the trail for every request that crosses it. So the architect builds all three outputs into the service templates (Section 5.5), and every new service inherits them on its first day. The thread that ties a request together is trace context: a small identifier attached to the request when it first arrives at Encore’s edge. Encore carries it in the traceparent header standardized by the World Wide Web Consortium (W3C), and OpenTelemetry instrumentation in each service reads it from incoming requests and attaches it to outgoing ones. The context travels through every hop, not only direct calls between services but also queue message headers and each step of a saga, so asynchronous work stays on the same trail. The payoff is concrete. When order #881 goes wrong at 3 a.m., any Encore engineer on call can follow it across seven services from a single trace ID, and needs to wake no one to find out where it stalled.

Recap. Each service has its own pipeline, and canaries and feature flags make deploys boring and reversible: a canary limits how many fans a bad version can reach, and a flag separates shipping code from turning it on. Most testing confidence comes from service and contract tests, end-to-end tests cover only a few business journeys, and testing in production, through small canaries and synthetic buyers, is deliberate and isolated from real fans. Every service propagates trace context, so that any request can be followed across the whole fleet.

Exercise 5.3. Measure how long a merged commit in your system takes to reach production, and count how many people have to touch it along the way (approve, merge, schedule or deploy it). If it takes more than an hour or involves more than one person, name the single constraint (a manual approval, a shared release train, a freeze window, a slow test suite) you would remove first, and say why that one.

5.4 Production Realities

So far we have designed the fleet, connected it and learned to deliver it. Now it has to survive real traffic, and four concerns decide whether it does: resilience, security, scaling and the user interface. Each gets a chapter of its own later in the book, so here we only survey them. What they have in common is worth seeing first. In a fleet, each of these properties is only as strong as the weakest service that carries it, so all four demand uniformity. A protection that exists in six services out of seven is, for practical purposes, a protection the fleet does not have.

Resilience, aggregated. Chapter 4 gave us the chain-breakers: timeouts, retries with backoff, bulkheads and circuit breakers, the devices that stop one failing dependency from dragging down everything that calls it. In a monolith, they were installed once, in one codebase, and every module shared them. In a fleet, they must exist in every service, configured the same way, because the services are coupled at runtime even when they deploy independently. A bulkhead missing in one service is a hole in everyone’s hull. Suppose Catalog has no timeout on a slow call during a Friday on-sale: its threads pile up waiting, the Sale Gate’s calls to Catalog start to wait in turn, and the fans queued behind the Sale Gate feel a failure that began somewhere they have never heard of.

Leaving this to each team’s discipline means nine teams must each get it right, and keep getting it right, on every release. The alternative is a service mesh: a layer of network proxies that sits beside every service and handles its traffic on its behalf. Each service talks only to its local proxy, and the proxies talk to one another. Because every request passes through them, the proxies are the natural place to enforce timeouts and retries, to encrypt and authenticate every connection with mutual Transport Layer Security (mTLS, in which both ends of a connection prove their identity, not only the server), and to emit telemetry about every call. The proxies can be deployed as a sidecar, a small proxy container running in each service’s pod, or, increasingly, as sidecar-less proxies shared by all the pods on a node, which cost less memory and are easier to upgrade. Either way, the platform team writes the policy once (“calls to Seat Inventory time out after this long and are retried at most this often”), and the mesh applies it to every service. Resilience stops being the diligence of nine teams and becomes fleet policy.

A city makes the same move when it lays water mains instead of trusting each household to dig a safe well. The water is clean everywhere because it no longer depends on every household doing its job well. The analogy also shows where the mesh’s reach ends: plumbing cannot fix a house’s wiring. The mesh sees requests passing on the network, not what they mean to the business, so some decisions stay with the architect and the service teams. Retries must still happen in exactly one layer, under a budget, because, as Chapter 4 showed, three layers each retrying three times turn one failing request into 27. And no mesh can make a non-idempotent call safe to retry. If Orders & Payments sends a charge request and the response is lost, the mesh cannot tell whether the charge happened; retrying it safely needs an idempotency key designed into the service itself, so that a second attempt is recognized as a repeat and not taken as a second payment.

A mesh is heavy machinery. It adds a hop to every call, consumes resources on every node and gives the platform team one more complex system to run and upgrade. For a handful of services, that price may be too high. At Encore’s scale, though, the alternative is resilience by memo: a wiki page of recommended timeouts that each team is asked to follow, and that the next new service quietly forgets. Between those two, the machinery is the better bargain.

Security, previewed. The same logic of uniformity applies to security, with higher stakes. Every boundary between services is a place where an attacker who has gained a foothold can try to go further, so each one is an attack surface. And “internal” is not a trust level: a request is not safe merely because it comes from inside Encore’s network. Believing otherwise is the fallacy that “the network is secure,” one of the fallacies of distributed computing from Chapter 4. If a compromised Notifications instance can call Orders & Payments simply because both sit on the same network, then one weak service exposes the most sensitive one. So each service gets its own identity, mTLS on every connection, so that both caller and callee know whom they are talking to, and least-privilege credentials that let it do its own job and nothing more. Notifications can send emails about an order; it cannot issue refunds. The mesh supplies much of the mechanism; Chapter 10 takes up the design in full.

Scaling, per service. Uniform policy does not mean uniform size, and here the fleet earns one of its clearest benefits. On a Friday on-sale, the Sale Gate scales to 400 instances to admit the crowd, while Support idles on 2, because nobody files a support ticket in the first minutes of an on-sale. In the modular monolith, scaling for the Sale Gate meant scaling every module along with it, Support included. In the fleet, each service scales on its own, according to its own load. Once every service exposes the same metrics and runs on the same platform, the scaling decisions become routine: rules the platform applies automatically, not judgment calls made by an engineer watching a dashboard.

UIs and backends-for-frontends. The last concern sits at the edge, where people meet the system. We have taken care to keep the services independent, and the frontend can quietly undo that work. If one screen of the fan app calls nine services directly, then the app knows the shape of the whole fleet, every change to a service’s application programming interface (API) risks breaking the screen, and a fan on a weak mobile connection pays for nine round trips. A backend-for-frontend (BFF) solves this by giving each experience, such as the fan app or the organizer web, a thin server of its own. The BFF makes the many service calls on the frontend’s behalf, close to the services, and returns exactly the data that one screen needs in one response. It is owned by the frontend team, because it changes whenever the screens change, and that keeps the service teams free to evolve their APIs without coordinating with every UI. Chapter 8 develops the pattern.

Recap. In a fleet, production properties must be uniform, because the weakest service sets the level for all of them. Resilience is enforced by the mesh, not left to each team’s memory, although retry layering and idempotency remain design decisions. Every service has its own identity and least-privilege credentials. Each service scales independently on the same platform, and BFFs stand in front of the UI so that screens do not couple themselves to the whole fleet. These properties belong in infrastructure, where no team has to remember them.

Exercise 5.4. Pick two services in your organization and compare how they handle timeouts, retries and TLS. Write down, for each, the timeout values, the retry count and backoff, and whether connections are encrypted and authenticated. If the two differ, their resilience depends on each team’s diligence. Sketch what moving these settings into infrastructure would take: which layer would enforce them (a mesh, a shared library or a gateway), who would own the policy, and what the first migration step would be.

5.5 The Organization That Microservices Require

Conway’s law as a design tool

Most of this chapter so far has been about software: how to cut services, how they talk, how they ship and how they survive. This section is about the people who build them, because a microservice architecture is as much an organizational design as a technical one. Mel Conway made the essential observation in 1968, in a paper called “How Do Committees Invent?”: an organization that designs a system will produce a design that copies the organization’s own communication structure. The reason is mechanical, not mystical. Two pieces of software can only fit together well if the people building them talk, so the interfaces in the system end up wherever the conversations in the organization happen, and the walls end up wherever the conversations stop. Team boundaries and service boundaries are therefore one drawing seen twice. When the two drawings disagree, the org chart usually wins. Suppose an architect draws Seat Inventory and Orders as separate services but leaves one team building both. That team will, quite reasonably, reach across the line whenever it is convenient, through a shared table here and a private shortcut there, until the two services behave like one.

Conway described a tendency; he did not prescribe a technique. Using that tendency on purpose, by shaping the teams first so that the architecture we want is the one they naturally produce, came much later. It is called the “inverse Conway maneuver,” a name coined at ThoughtWorks and popularized by Matthew Skelton and Manuel Pais in their book Team Topologies. Encore made exactly this move. Its team map was drawn to match its service map: seven stream-aligned teams, each owning one service, plus one platform team and one fan-app team (Section 5.1). Because each service has one team and each team talks mostly to itself, the boundaries the teams draw in their daily work are the boundaries the architecture asks for. Conway’s force, which so often pulls a design apart, here holds it together.

What kind of team makes this work? Skelton and Pais call them stream-aligned teams: a team aligned to one stream of business change, such as selling seats or taking payments, that builds its services, runs them in production and carries the pager for them. The pager matters more than it might seem. When the people who wrote the code are the people woken at 3 a.m. when it fails, the lesson arrives at once and reaches exactly the people who can act on it. No postmortem document, no ticket queue and no hand-off between a development team and an operations team is faster. That is why I think of the pager as the shortest feedback loop an organization has. If the Orders & Payments team ships a change that makes failover to the second PSP slow, the same team feels it during the next on-sale and fixes it the next morning.

Owning everything from code to pager carries a cost, and that cost is cognitive load: the amount a team must know and keep in mind to do its job. A team that must also master build pipelines, observability tooling, the service mesh and the conventions for starting a new service has less attention left for seats and fans. This is where the platform comes in (Chapters 11 and 15). A platform team takes those shared burdens and turns them into paved roads: ready-made, well-maintained paths, such as a golden-path template that creates a new service with its pipeline, dashboards, alerts and mesh configuration already wired in. A stream team that follows the road gets all of that without having to become expert in any of it. The arithmetic is simple. A team can hold only so much in its head, so every concern the platform absorbs is attention handed back to the business problem, which is the only thing the stream team is there to solve.

Governance without a review board. Any organization of this size still needs some way to keep its services consistent and safe, and the traditional answer is a central architecture review board that must approve each new service or significant change. The trouble is that such a board recreates exactly the coordination that microservices exist to remove. If seven teams must each queue for the same committee before they ship, we have reintroduced the deploy trains and waiting from the monolith, only now the waiting happens in a meeting room. A board that approves every service is a monolith rebuilt out of meetings.

Encore governs instead with paved roads and fitness functions, the automated architectural checks introduced in Chapter 1. The two do different jobs. The paved road works before the fact: because the golden-path template already includes the logging, tracing, mesh policy and security settings Encore wants, the right thing is also the easiest thing, and most teams do it without being asked. The fitness functions work after the fact: Chapter 1’s governance, now applied across the whole fleet, runs checks in every pipeline that catch drift. A service that starts sharing another service’s database, or ships without the required timeouts, fails a check and is stopped before it reaches production, and no meeting is needed. A team may still deviate from the road if it has a real reason. The condition is that it pays for the alternative and owns the result, including the 3 a.m. pages when the alternative breaks. Encore’s rule of thumb puts it plainly: you may leave the paved road whenever you are prepared to build your own.

Deciding whether to adopt microservices

All of this describes an organization that has already made microservices work. The harder question comes earlier: should an organization adopt them at all? The answer depends less on technology than on whether certain prerequisites are in place, so it helps to ask about them explicitly.

Microservices are a prerequisite-gated purchase. Ask four questions, in order. (1) Do you have a team-coordination problem, measured in waiting? (2) Are your domain boundaries stable enough to cut along (Chapter 3 done properly)? (3) Can you operate a fleet (observability, on-call, automated deploys) or fund a platform that can? (4) Can leadership accept team autonomy, including its mistakes? With four yeses, proceed incrementally, strangler-style. With any no, a modular monolith with enforced boundaries gives most of the benefit at a fraction of the cost and keeps the option open.

Each question guards against a different way of failing. The first asks whether the problem exists. Microservices cure one ailment, teams waiting on each other, so the evidence should be measured in waiting: days lost to deploy trains, features held for freeze windows, teams blocked on another team’s release. Encore, with 70 engineers in nine teams, could point to all three. The second asks whether we know where to cut. If the domain boundaries from Chapter 3 are still shifting, services cut along them today will need to be recut tomorrow, and recutting across a network is far more expensive than moving code between modules. The third asks whether we can run what we build. A fleet needs observability to see what is happening, an on-call rotation to respond when it goes wrong and automated deploys so that shipping stays cheap, or else a funded platform that provides those things. The fourth asks whether leadership will let teams make their own decisions, including the wrong ones. If all four answers are yes, proceed one service at a time, strangler-style, as Chapter 3 described, so that each extraction can be reversed. If any answer is no, a modular monolith with enforced boundaries delivers most of the benefit, clean separation and clear ownership, at a fraction of the cost, and it keeps the option to split later.

The order of the questions matters, much as it does in medicine. A good surgeon first establishes that the ailment is present, then that the anatomy is understood, and only then whether the hospital is equipped to operate; a skilled operating team is no reason to operate on a patient who is not ill. In the same way, a fine platform team is no argument for microservices if nobody is waiting, and it is little help if the boundaries are still moving.

Of the four, the question about leadership is most often answered too quickly, because saying yes costs nothing on the day it is said. The test comes later. Suppose Encore’s Catalog team ships a bad release that breaks event pages for an hour, and the response from above is a new rule that every release now needs sign-off from a senior manager. The teams still carry everything that autonomy cost them: separate pipelines, separate on-call rotations, a network between services where there used to be function calls. What they have lost is the independence that justified those costs. Autonomy withdrawn after the first bad release leaves the organization with only its burdens.

None of this makes a modular monolith a consolation prize. Chosen deliberately, it is a sound architecture in its own right, the one that served Encore well for its first three years. It keeps the boundaries, because the modules stay separate and automated checks can enforce the lines between them, and it defers the network, with all its partial failures and operational cost, until the organization can afford to pay for it. When that day comes, the boundaries are already drawn, and the split follows lines that have been tested for years.

Recap. Conway’s law means team boundaries become service boundaries, so Encore drew its teams to match its services. Stream-aligned teams own their services from build to pager, and a platform absorbs the shared burden so that each team’s cognitive load stays manageable. Governance runs on paved roads and fitness functions instead of meetings: the road makes the right thing easy, and automated checks catch drift. Adopting microservices requires coordination pain, stable boundaries, operational capability and leadership’s acceptance of autonomy; without all four, the modular monolith is the wiser purchase.

Exercise 5.5. Answer the four adoption questions for your organization in writing, giving evidence for each answer: for example, how long changes wait on other teams, how often your domain boundaries have moved in the past year, who carries the pager today, and how leadership responded to the last bad release. Then mark any “yes” that is really a “hope,” and say what evidence would turn it into a genuine yes.

Kata: The Adoption Verdict

Your brief: “Grainhouse.” Grainhouse is the 60-developer software group of a grocery chain. Everything it builds lives in one twelve-year-old monolith that runs the stores, the warehouse, e-commerce and the loyalty program. The monolith ships on a release train every three weeks, and its six teams spend much of that time waiting: a feature that is ready must queue behind the features of other teams until the whole train is ready to leave. E-commerce feels this most, because during every release freeze it cannot ship anything, and it loses money each time. The CEO has read an airline magazine and now wants “microservices by Q3.” Three facts about the system matter for your verdict. The warehouse module is stable and well liked. The loyalty module changes every week. And the database is a single schema of 700 tables, none of which has an owner.

Deliverables: (1) The four-question adoption verdict from the section above, with each answer backed by evidence from the brief rather than by opinion. (2) If your verdict is “yes, with conditions,” name the first two extractions. Choose them by two tests: how much coordination pain each one removes, and how few tables and calls would cross its new boundary. For each, give a strangler plan (the incremental, reversible migration of Chapter 3) and a table-ownership plan that assigns an owner to every table the extraction touches. (3) The communication design for the extracted pieces, applying the discipline of Figure 5.1: say which interactions become events, which stay synchronous calls and which become orchestrated processes. (4) The organizational prerequisites: what Grainhouse must build or hire before its first service, service #1, ships. (5) One architecture decision record (ADR), the short written record of a decision and its reasons, titled “why we are not extracting the warehouse.”

Rubric:

Where you now stand

We can now cut services along real business boundaries, define the contracts between them, deliver and observe them, and organize teams so that the structure holds. One piece, though, we have taken on trust since Section 5.1: the events that let one service learn a fact, such as a sold seat or a canceled concert, without having to call another service and ask. Events are the fleet’s nervous system. They carry news from the service where something happened to every service that needs to react, and we have been relying on them without yet examining how they work. Chapter 6 takes them up, covering event-driven architecture and streaming.

References

• Sarah Wells — Enabling Microservice Success. O’Reilly, 2024.
• Neal Ford, Mark Richards, Pramod Sadalage, Zhamak Dehghani — Software Architecture: The Hard Parts. O’Reilly, 2021.
• Sam Newman — Building Microservices, 2nd ed.. O’Reilly, 2021.
• Nicole Forsgren, Jez Humble, Gene Kim — Accelerate. IT Revolution, 2018.
• Chris Richardson — Microservices Patterns. Manning, 2018.
• Jez Humble, David Farley — Continuous Delivery. Addison-Wesley, 2010.
• Matthew Skelton, Manuel Pais — Team Topologies, 2nd ed.. IT Revolution, 2025.
• Charity Majors, Liz Fong-Jones, George Miranda — Observability Engineering. O’Reilly, 2022.
• David L. Parnas — On the Criteria To Be Used in Decomposing Systems into Modules. Communications of the ACM, 1972.
• Melvin E. Conway — How Do Committees Invent?. Datamation, 1968 — free online.
• Hector Garcia-Molina, Kenneth Salem — Sagas. ACM SIGMOD, 1987.