Building Products in the AI & Cloud Era

← the whole book on one page

Chapter 1: The Product and the System — Deciding What to Build at 400 Engineers

“The hardest single part of building a software system is deciding precisely what to build.” — Frederick P. Brooks Jr.

A company with 400 engineers can build almost anything it decides to build. That is its strength, and it is also the source of a quiet danger, because the capacity to build makes the decision about what to build feel cheaper than it is. Forty teams will always have work. Their boards will be full, their releases frequent and their roadmaps confident, and none of that tells us whether the work is changing anything for the people it was meant to serve. The question of this chapter, and of the whole book, is whether an organization of this size can decide well what to build: how it notices that it has been measuring the wrong things, who in it should own the decision, how the decision is tested before the money is spent, and what changes now that writing code has become cheaper than it has ever been.

We begin with a failure that had every output a project plan could ask for. On 1 October 2013 the United States government opened HealthCare.gov, the website through which millions of Americans were to buy health insurance. Dozens of contracts had been let, the code had been written, and the site went live on the day the White House had promised. On that first day, according to the government’s own inspector general, six people managed to submit an application and select a plan. The repair that followed, and the government agency that grew out of the repair, teach the central lesson of this chapter more plainly than any framework could. A product can be delivered on time while its outcome fails, and the difference between the two is a matter of ownership and measurement far more than of engineering skill.

Encore, the concert-ticketing company we follow through this series, has never had a day like that, and this chapter will not invent one. Its trouble is milder and more common. Encore reached 400 engineers in its seventh year, and it has stayed at about that size since. It is now the winter of its ninth year, a few months after the Vesper Hale on-sale and the added stadium dates that the Operations book followed. Some of its teams decide well what to build; others work through a queue of requests that nobody ever checks against what customers then did. And on the table is a proposal that will test all of them. It is called Verified Resale: fans who cannot attend a show would resell their tickets through authorized partners, Encore would reissue every resold ticket under a new barcode, and organizers would set the rules and share the fee. This chapter introduces Encore for readers who start here, sets out the proposal, and asks what Encore would have to be good at to decide whether it should exist. Chapter 2 takes up whether it should.

1.1 The Build Trap

Six people on the first day

HealthCare.gov was built to let people in 36 states, those that had not set up an exchange of their own, compare and buy health insurance under the Affordable Care Act. The Centers for Medicare & Medicaid Services (CMS), part of the Department of Health and Human Services (HHS), was responsible for it. The law required that coverage begin on 1 January 2014. It did not require the website to open on 1 October 2013. That date came from a public commitment by the White House, and the HHS Office of Inspector General (OIG) later found that some staff and contractors worked under “the misimpression that the deadline for website functionality was statutory.” Nobody among CMS’s leadership, the OIG reported, seriously discussed moving it.

The work was divided among about 60 contracts, 55 of them awarded under contracts that already existed for other purposes. CGI Federal built the core of the marketplace. No contractor was hired to act as the systems integrator, the party responsible for making the pieces work together, because CMS believed it was doing that job itself. And nobody owned the whole. “CMS senior leadership never declared a clear ‘business owner’ with overall responsibility for the Federal Marketplace,” the OIG wrote, and when its investigators asked staff which division had led the project, they named four different ones. The Government Accountability Office (GAO) reached a similar conclusion in 2014, finding that CMS had issued task orders while key technical requirements were still unknown. The costs grew as such projects’ costs do: six key contracts estimated at $464 million had reached $824 million by early 2014.

On the morning of 1 October about 250,000 people tried to use the site at once, far more than it had been built to carry, and it began failing within two hours. “In the end,” the OIG wrote, “only six consumers were able to submit an application and select a plan on the first day.” Notice the precise wording, because the popular retelling blurs it. Six people did not enroll and pay; six managed to apply and choose. Every output in the plan had been delivered. The contracts had been let and fulfilled, the code had been written and deployed, and the launch date had been met. The outcome, people able to buy insurance, had not happened at all.

The repair is the more instructive half of the story. Steven Brill’s account in Time, published in February 2014, describes a small group of engineers from outside government, among them Mikey Dickerson, on leave from Google’s site reliability engineering, who arrived in the third week of October under Jeff Zients and the US chief technology officer, Todd Park. One of their first discoveries was that nobody could see what users were experiencing: the site had no dashboard of its own health, so the first thing the newcomers did was build one. Dickerson’s rules for the daily stand-up meeting, as Brill reports them, are worth quoting, because they describe a culture as much as a process: “The ones who should be doing the talking are the people who know the most about an issue, not the ones with the highest rank.” By early December, by Brill’s account, the site could serve at least 50,000 people at once, and its uptime had risen from 43% to 95%. The first open enrollment ended with 5.4 million people having selected a plan through the federal marketplace.

It is tempting to tell this as the story of a brilliant small team rescuing a failed large one, and the OIG’s report, based on 86 interviews, takes care to correct that version. What it credits is a reorganization of everyone already involved: a “badgeless” culture in which CMS staff and contractors worked together as one team, whatever their employer, together with “ruthless prioritization” of the work that mattered most. Its summary of the cause is a single sentence: “Most critical was the absence of clear leadership.” In August 2014 the White House founded the US Digital Service (USDS), with Dickerson as its administrator, and published the Digital Services Playbook the same day. Its sixth play states the lesson of the whole affair in a paragraph: “Assign one leader and hold that person accountable. There must be a single product owner who has the authority and responsibility to assign tasks and work elements; make business, product, and technical decisions; and be accountable for the success or failure of the overall service.”

Figure 1.1 — HealthCare.gov from launch to the US Digital Service, in order but not to scale. Above the line the launch date was met; below it, six people got through on the first day, and nobody inside could see that until the repair built a dashboard.

Figure 1.1 sets the story out in order, with what the organization did above the line and what users met below it. Read as a product failure, HealthCare.gov had four faults, and each will recur in this book in a milder form. A date was mistaken for the goal. The work was organized as contracts with deliverables, so that every party could succeed while the whole failed. Nobody measured what users experienced, so bad news had no road to travel up. And nobody owned the product from end to end. The repair restored each missing piece: an owner, a measure of outcomes and one team across organizational boundaries.

Outputs and outcomes

The distinction at the heart of that story needs precise names. An output is something an organization produces and ships: a feature, a release, a completed contract, a closed ticket. An outcome is a change in what customers do that creates value for them and for the business. Josh Seiden, whose short book Outcomes Over Output (2019) gave the distinction its plainest form, defines an outcome as a change in human behavior that drives business results. The definition is demanding on purpose. It excludes things we produce, because those are outputs, and it excludes the business results themselves, such as revenue, because those arrive too late and are caused by too many things to steer a team. What remains is behavior: organizers setting up events in half the time, fans who come back for a second show, partners whose integrations stop breaking.

Melissa Perri, in Escaping the Build Trap (2018), named the condition that follows when an organization loses this distinction. The build trap is an organization’s habit of measuring its success by what it ships rather than by what its customers then do. The trap is comfortable, which is why it is common. Outputs are easy to count, they arrive on a schedule, and they belong clearly to the team that made them, while outcomes are slow, noisy and shared. A roadmap of features can be finished, and finishing it feels like success. An organization in the trap is rarely idle; it is usually busier than its neighbors.

The evidence that outputs are a poor proxy is old and consistent. At Microsoft, Ron Kohavi and his colleagues ran controlled experiments on ideas that teams had already judged worth building, and they reported in 2012 that “only one third of ideas tested at Microsoft improved the metric(s) they were designed to improve.” The other two-thirds were neutral or made things worse, and without the experiments every one of them would have counted as a delivered feature. A vendor study points the same way from a different angle. Pendo’s Feature Adoption Report of 2019 estimated that 80% of features in the average software product are rarely or never used. That figure comes from one company’s analysis of its own customers’ products, so it indicates a direction and does not measure the industry. Taken together, the two sources suggest that a team which ships and moves on is, most of the time, unaware of whether its work did anything at all.

Encore’s organizer console shows the symptoms clearly enough to be useful. Over the last four quarters the seven teams that build for organizers shipped 61 items from a shared request queue. Forty-four of them entered the queue from the organizer sales team, usually attached to a contract being negotiated, and they arrived as solutions: a new export format, a second approval step for price changes, a dashboard tile. Only for five of the 61 did anyone look afterwards at whether organizers used the result. The teams are capable and hard-working, and nothing here is anyone’s fault in particular. The trouble lies in the queue itself, which records what was asked for and what was shipped, and holds no place for what happened afterwards.

The value exchange system

Perri’s account of why organizations fall into the trap rests on a simple model. A company and its customers are in a value exchange: customers have problems and desires, and when the company’s products solve them, the customers give something back, usually money, loyalty or attention. The company captures value only by creating it for customers first. The model sounds too obvious to need stating, until we notice how many forces inside a company push it out of shape. Perri names three kinds of organization in which one force dominates, and Figure 1.2 sets them beside Encore’s own history.

Figure 1.2 — Perri’s three distorted organizations, with an Encore example of each. Each kind trades the customer’s problem for a different proxy.

Perri’s own example of the visionary-led company is Apple under Steve Jobs, which she treats as the rare exception that works while the visionary is present. Chapter 2 tells the story of Jobs’s return to Apple in 1997, so here we note only her warning: a company whose product judgment lives in one person has a single point of failure in the place it can least afford one. The framework itself is a practitioner’s lens, drawn from experience; nobody has tested it in a controlled study. It earns its place because it lets a team name the force bending its own roadmap, and once the force is named, the team can argue about whether it should prevail.

Projects, products and services

A deeper cause of the trap lies in how work is organized and funded. A project is a bounded piece of work with a scope, a budget and an end date; it succeeds when its scope is delivered. A product is a long-lived capability with an owner, which succeeds when it changes what its customers do and goes on succeeding only as long as it keeps doing so. A service, in Perri’s sense, delivers value mainly through people’s labor, each instance done afresh. The three shapes reward different things. A project rewards finishing, a product rewards the outcome, and a service rewards the hours its people put in.

HealthCare.gov was the project shape at its most expensive: some 60 contracts, each with deliverables, none responsible for whether a family could buy insurance. Encore’s own example is milder and instructive in a different way. Onboarding a new venue onto Encore’s SaaS platform is still partly a service, since a person on the Tenant Onboarding team configures each new venue’s seat maps and branding by hand. Perri’s point is that a service which needs a person in every instance is not yet a product, and that it becomes one only when the organization decides to own the outcome, venues live and selling, and invests in making it repeatable. Mik Kersten’s Project to Product (2018) adds a way of seeing what a product team actually spends its time on, by sorting work into four flow items: features, defects, risks and debt. The funding mechanics that follow from his argument belong to the Software book’s Section 10.4, and we leave them there.

The hardest single part

The Architecture book introduced Fred Brooks’s distinction between essential complexity, which belongs to the problem, and accidental complexity, which we bring to it, and this book does not redefine either. What it adds is the passage from the same essay, “No Silver Bullet” (1986), that the Architecture book did not quote, and which is this book’s starting point. “The hardest single part of building a software system is deciding precisely what to build,” Brooks wrote, and a few sentences later: “No other part of the work so cripples the resulting system if done wrong. No other part is more difficult to rectify later.” He went on to say why: “the client does not know what he wants.” His remedy was the iterative extraction and refinement of requirements, through rapid prototypes put in front of the people who would use the system.

Brooks called deciding what to build the hardest single part. This book goes a step further and argues that most of a product’s essential complexity is decided there, in the choice of which problem to solve and for whom, and that everything built afterwards inherits the quality of that choice. The claim is ours, and Brooks should not be held to it. But it explains the shape of this book, which spends its first part on deciding before it spends its second on designing. It also explains why Brooks’s remedy reads, forty years later, like a description of what product teams now call discovery.

Diagnosing the trap

How does an organization tell whether it is caught? Perri closes her book with a short set of diagnostic questions, and two of them do most of the work. One asks who came up with the last feature the team built. The other asks what was the last product the company decided to kill. A company that cannot answer the second has no mechanism for learning that something failed, because outputs, once shipped, are never examined again. Marty Cagan’s essay contrasting good and bad product teams, first published in 2007 and collected in INSPIRED (2017), and John Cutler’s “12 Signs You’re Working in a Feature Factory” (2016) offer longer lists of symptoms. None of these is a validated instrument. They are checklists drawn from experience, useful because the honest answers are uncomfortable.

At Encore the answers differ by team, and that is itself the most useful finding. The fan storefront teams run experiments, watch what fans do and retire features that fail. The organizer console teams, as we saw, work through a queue, and when asked what they last decided to kill, they could not name anything. The same company, with the same leaders and the same engineering platform, can be product-led in one part and caught in the trap in another, because the trap lives in how each team’s work arrives and how it is judged.

The simplest proven tool for changing how work arrives is to ask, before it starts, what outcome it serves. Cagan’s opportunity assessment in INSPIRED does this with four questions, asked of any proposed piece of work: what business objective it addresses, how we will know it succeeded, what customer problem it solves, and for which customers. The Organizer Console team now attaches a filled-in assessment to every item that enters its queue. Here is one, for a request that arrived from sales.

OPPORTUNITY ASSESSMENT: OC-212 "Second approval step for price changes"
Requested by     Organizer sales, for a festival promoter's renewal
Objective        Retain mid-size promoters at renewal (team objective 2)
Customer problem Promoters' finance staff fear unauthorized price changes
                 by event staff; two incidents reported in the last year
Target customers Promoters with separate finance and events staff
                 (about 120 of our 900 active organizers)
Success measure  Share of those promoters using price-change approval
                 after 90 days; renewal rate in the segment
Alternatives     Audit log with alerts; per-role permission on prices
Decision         Discovery first: interview four promoters, test alternatives
Review           Outcome check booked for 90 days after release

The last line matters as much as the others. An assessment written before work begins is only half the discipline. The other half is the review after release, when someone looks at the success measure and decides whether the work stays, changes or goes. That review is how a company comes to have an answer to Perri’s question about the last thing it killed. Measuring the outcome well is the subject of Chapter 4; here it is enough that the review is booked, with a date and the people who will attend, at the moment the work is accepted.

Recap. HealthCare.gov delivered its outputs on time and failed its outcome, because nobody owned the product end to end and nobody measured what users experienced. The build trap is the habit of judging work by what is shipped, and the evidence that most shipped work moves nothing is old and consistent. Sales-led, visionary-led and technology-led organizations each bend the exchange of value in their own way, and the project shape rewards finishing over outcomes. The cure begins with two habits: ask what outcome a piece of work serves before it starts, and look afterwards at whether it did.

Exercise 1.1. Take your team’s last ten shipped items. Label each with the outcome it was meant to move, and with whether anyone checked afterwards that it did. Then answer Perri’s question for your own organization: what was the last thing you decided to kill?

1.2 Encore at 400

The products

For readers meeting Encore for the first time, a short introduction. Encore sells concert tickets across the European Union, in euros, and also sells its ticketing platform to venues as software as a service (SaaS). Three facts have shaped everything it has built. When a famous artist’s tour goes on sale, traffic rises 500× within minutes. Every seat must be sold exactly once. And bots arrive before the fans do. The Architecture book followed Encore from nine engineers to 400, the Resilience book showed how it prepares for its biggest mornings, and the Operations book showed one of those mornings, the Vesper Hale on-sale, going wrong and being recovered. This book asks a different question of the same company: given that it can build and run almost anything, how does it decide what is worth building?

At 400 engineers Encore has four kinds of customer, and it serves each with a product of its own. Fans use the storefront on the web and the native app, to find events, buy tickets and carry them to the door. Organizers, the promoters and festival companies whose events Encore sells, use the organizer console to set up events, set prices and watch sales. Partners, among them venues, a festival app and the resale partner Rowmate, call the public and partner application programming interfaces (APIs) at api.encore.com. And more than 500 venues run their own box offices on Encore’s platform as SaaS tenants, in three tiers. A fifth product serves only Encore itself: the internal platform, the paved roads on which its teams build and deploy. Each of these products has different customers, different measures of success and different economics, and each has one product leader accountable for its outcome. That last fact is USDS’s sixth play, applied to a company: whatever else changes, someone can always say who owns the product.

The 40 teams on one map

Figure 1.3 places Encore’s teams by the product and customer they serve. It is the roster this book keeps; the Software book reads the same roster as code ownership and as team types, in its Sections 1.2 and 9.2. Encore has 40 stream-aligned teams, each owning a slice of a product or a shared domain end to end, with about nine engineers each, and a platform group of nine teams and 40 people beneath them. The platform group sits at the top of the 5–10% of engineering that the Architecture book’s Section 15.2 gave as a common rule of thumb.

Figure 1.3 — Encore’s products, customers and teams on one map. Forty stream-aligned teams serve four kinds of customer, directly or through the 14 shared domains, and the platform group serves the teams.

The shared domains need a word of explanation, because they are where the map stops being a simple tree. The Inventory team owns Seat Inventory & Ticketing, the service that sells each seat exactly once, and every product depends on it. So do Orders & Payments, Bot Screening and Fan Messaging, which delivers tickets to phones. These teams have customers in every column, and their outcomes are the service levels the Resilience book taught Encore to set with the teams that consume them. A map like this is never neutral. It records which customers have a team of their own and which are served only on the way to someone else, and the exercise at the end of this section asks the reader to draw one for that reason.

How Encore got here

Every stage of Encore’s growth made product decisions by default, through the shape of the organization, without anyone choosing them. When Encore had nine engineers in one team, the two founders decided what to build in conversation. They were, without using the word, a product trio of the kind Section 1.3 describes, and they were close enough to fans and organizers to be right most of the time. The decision they made by default was that the organizer side would be built when someone had time. For three years, organizers were served by whatever the fan product needed them to have.

Three years later Encore had 70 engineers in nine teams, each owning one service. Product decisions now followed service boundaries: whoever owned a service decided what it did, and nobody owned journeys that crossed several services. The public API opened in this period because partners asked for it, and in its sixth year Encore agreed to a request from partners that became, without anyone deciding so, a product. Under Partner Listings, authorized partners could read Encore’s event listings and post resale offers that showed as links on sold-out event pages, while the ticket itself changed hands outside Encore. It was approved as an integration and owned by nobody as a product, and nobody measured what it did to fans. Chapter 2 reads the consequences.

By its seventh year Encore had 400 engineers in 40 teams, the shape Figure 1.3 shows, and the decision made by default at this size was the request queue. Where a product had a strong leader and good data, as the fan storefront did, teams were given problems. Where the loudest input came from contracts being negotiated, as on the organizer side, teams were given lists. The Software book’s Section 9.1 tells the organizational history in full. Here the point is narrower: at each size, the decisions Encore made by default were product decisions, and it only noticed them afterwards.

The new bet: Verified Resale

The proposal came from Encore’s chief product officer and its head of partnerships, in the weeks after the Vesper Hale on-sale, when tickets for the tour appeared on unofficial resale sites within a day at prices well above face value and fans who bought them were turned away at the door. Its parts answer the four weaknesses of Partner Listings that Chapter 2 sets out: a barcode could be sold to several buyers, organizers had no control, organizers took no share, and buyers went unscreened.

Fans may list tickets they bought through the Sale Gate, Encore’s admission controller for on-sales, on Encore’s own app or on an authorized partner. Rowmate would be the first partner and Tessera the second. Partners are marketplaces where fans list their tickets; no partner owns inventory, and Encore admits no professional resellers. When another fan buys a listing, that buyer passes the same Bot Screening as any primary buyer, and Encore reissues the ticket: it issues a new ticket identity to the buyer and revokes the seller’s old barcode at every gate. The price is capped at face value, meaning what the seller originally paid, and organizers can only tighten the rules for their events: by setting a lower cap, disabling resale, shortening the resale window or lowering the per-fan limits. The buyer pays a 10% fee on top of the resale price, split four points to the partner through which the ticket was bought, three to the organizer and three to Encore; when the buyer buys in Encore’s own app, Encore keeps the partner’s four points as well. A €60 ticket resold at face value costs the buyer €66; the partner receives €2.40, the organizer €1.80 and Encore €1.80, and the seller gets the full €60.

Reissue on transfer is established practice in the industry, and Encore did not invent it. When Ticketmaster announced its SafeTix system in May 2019, it described a mobile barcode that refreshes every few seconds and a new digital ticket issued to the recipient whenever a ticket is transferred. What Encore proposes to add is the rest of the system around reissue: the partners, the organizer’s rules, the cap and the shared fee.

It is important to say what the proposal is at this point, because the rest of Part I depends on it. Nothing has yet been decided. Leadership has asked for a written bet memo by the end of the winter, and has asked three people to spend the weeks until then finding out whether the bet deserves to be made. Chapter 2 tests its strategy, Chapter 3 its assumptions, Chapter 4 how its success would be measured and Chapter 5 its price. Verified Resale will be built in the end, and Chapter 10 follows it from the memo to its launch and review. But at this moment nobody at Encore knows that, and the chapters that follow should be read as the company finding out.

What already exists

Verified Resale does not start from nothing, and an honest assessment of a bet begins with the assets it can reuse. When Encore opened its public API, in the Architecture book’s Chapter 7, resale partners were among the first to ask for a feed of available inventory, and Partner APIs has served such feeds ever since. Since the Rowmate key leak, which the Resilience book’s Section 7.2 tells, every partner has authenticated with short-lived tokens bound to a client certificate, so the partner credentials a new product would need already exist and have already been tested by an incident. Fan Messaging delivers tickets to phone wallets and updates passes. And since the eighth year, the barcode in Encore’s app has rotated, generated on the phone from a secret held for each ticket, so a screenshot of a ticket stops working within seconds. Each of these was built for another purpose, and each lowers the cost of the bet. The mechanics stay in the books where they were built; what matters here is that the trio can count them as assets when it estimates what Verified Resale would cost.

Recap. Encore serves four kinds of customer with four products, plus an internal platform, and each product has one accountable leader. Its 40 stream-aligned teams and nine platform teams sit on one map, which shows which customers have teams of their own. At each stage of its growth Encore made product decisions by default, Partner Listings among them. Verified Resale is proposed to replace that arrangement, as a bet still to be tested.

Exercise 1.2. Draw your own organization’s products, customers and teams on one page, in the manner of Figure 1.3. Mark every team that serves no customer you can name, and every product with no single accountable owner.

1.3 The Four Risks and the Product Trio

Value, usability, feasibility, viability

Every product idea is a bundle of guesses, and a useful way to sort them is by what could make the idea fail. Marty Cagan’s essay “The Four Big Risks” (2017) gives the standard sorting. Value risk is the risk that customers will not buy the product or choose to use it. Usability risk is the risk that they cannot figure out how to use it. Feasibility risk is the risk that the engineers cannot build it with the time, skills and technology the company has. Business viability risk is the risk that the solution does not work for the other parts of the business: sales, finance, legal, partners, brand. Discovery, the work this book teaches in Chapter 3, is the activity of reducing these four risks before the cost of delivery is spent.

The four risks reward being written down for a particular idea, because each idea carries them in different proportions. Figure 1.4 does this for Verified Resale, as the trio first wrote it.

Figure 1.4 — The four risks of Verified Resale, with the fifth question added, as the trio first wrote them. The last column is mostly empty, and saying so plainly is the point of the exercise.

Viability is many stakeholders

Viability is the risk teams most often see last, because it does not show up in a prototype. A fan can be shown a resale screen and asked what she thinks; a regulator cannot. Cagan spells viability out as a set of constraints from the rest of the business: whether the product can be sold and marketed, whether it fits existing contracts, whether it is legal, whether it can be paid for and whether it keeps the brand’s promises. For Verified Resale each of these has a name. Consumer law constrains resale prices in some markets: Ireland’s Sale of Tickets Act 2021, in force since 31 July 2021, bans resale above the original price for designated events and venues, and Chapter 5 takes up what that means for Encore’s fee. Finance must decide how a fee shared among three parties is recognized as revenue. Organizers’ contracts must be amended to give them rules and a share. The European Union’s Digital Services Act treats Encore, for resale, as an online marketplace, which is another reason to admit fans and no professional resellers, since traders would bring obligations to trace and verify them. And Encore must not end up holding other people’s money, which would require a payment license; the trio’s early answer is to route resale payments through the primary payment provider’s marketplace product.

None of these constraints is an engineering question, and every one of them could kill the product. Cagan and his co-authors argue in TRANSFORMED (2024) that the people who hold these constraints, in legal, finance, sales and partnerships, are partners of the product team, to be brought into discovery early; a team that visits them only at the end for approval learns their constraints when it is most expensive to meet them. The way they actually join a team’s discovery is the Software book’s Section 9.3. Here it is enough to say that viability is a whole list of risks, each owned by a different part of the company, and that the product manager’s job is to know them all before the memo is written.

The ethical risk

There is a fifth question, older in Cagan’s writing than is often assumed. The second edition of INSPIRED (2017) already asks, in a short passage on ethics, whether we should build something at all, and TRANSFORMED returns to it under the heading of ethical risk. Teresa Torres, in Continuous Discovery Habits (2021), lists ethical assumptions as one of five kinds a team should test. The question differs from the other four. Value, usability, feasibility and viability ask whether the product will work. The ethical question asks whether it should, and who bears the cost if it does.

For Verified Resale the ethical question is sharp. Encore’s brand rests on fighting scalpers, on the promise that real fans get a fair chance at seats. Even a resale market with a face-value cap, run only for fans, could be read as normalizing the secondary market Encore has always opposed. And if bots get through the screening, it is a real fan who is outbid for a seat that should have been hers. The trio cannot answer this question with an interview or a prototype. It is a strategic question as much as an ethical one, and Chapter 2 gives it the strategic answer the company eventually reached. What this chapter insists on is that it is asked out loud, in the memo, by the people proposing the product, before anyone else has to ask it for them. Encore has already settled one ethical question permanently: it will never run individual price experiments on fans.

The trio

Who carries these risks? Cagan’s answer is that each belongs to a role. The product manager owns value and viability, the product designer owns usability, and the tech lead owns feasibility. The three together are what Torres calls the product trio: the product manager, the designer and the tech lead who share responsibility for discovering what their team should build. The definition contains its own instruction. The three work as one unit from the first interview, and none of them waits for another to finish. A designer who receives requirements from a product manager and passes screens to an engineer is in a relay, and each handover loses what the previous runner learned.

A surgical team is a useful picture, up to a point. The surgeon, the anesthetist and the scrub nurse each own a different risk to the patient, and none of them could take over another’s job. Yet they work on the same patient at the same time, each watching what the others are doing, and an operation in which they took turns would be absurd. The analogy breaks where the trio’s work is least like surgery: the surgical team knows what operation it is performing, and the trio is still finding out. That uncertainty is exactly why the three must be in the same conversations, since each will notice different things in the same interview.

Encore’s resale trio was formed for the winter’s work. Amira, the product manager, came from the Partner APIs team, where she had inherited Partner Listings and its complaints. Paolo, the designer, came from the Organizer Console team, and knew organizers’ working days better than anyone else at Encore. Katrin, the tech lead, came from Orders & Payments, where she had worked on the purchase saga, and had worked against the barcode core from the purchase side for years, which made her one of the few people outside the Inventory team who understood it. All three keep their old teams for now; if the bet is made, they will form the core of a new Resale team. Cagan and Torres both treat the trio as practitioner consensus, without a controlled study behind it, and this book presents it the same way. It recommends the trio because a relay loses, at each handover, what the previous role learned, and because discovery depends on what each role notices.

The tech lead in discovery

Of the three, the tech lead is the one most often left out of discovery, and leaving her out costs the most. Engineers who join only when delivery begins arrive after the product has been shaped, and the most valuable thing they know, what has recently become possible and what is harder than it looks, arrives too late to change the shape. Cagan’s tool for bringing it in early is the feasibility prototype: a small piece of throwaway code written to answer one question about whether something can be built, before anyone commits to building it.

Katrin’s first question was the one Figure 1.4 records: once a ticket is resold, can Encore make the seller’s copy stop working? In Encore’s own app the answer was already yes, because the barcode there rotates and the app can be told to stop generating it. Wallet passes were less certain, so she spent a week finding out. For Google Wallet, an issuer can mark a ticket pass as expired, and the pass moves to the expired section, but only once the phone has synchronized. For Apple Wallet, the update mechanism is a push notification asking the phone to fetch the new pass, and Apple’s own documentation warns that such notifications are not guaranteed to be delivered. The prototype’s finding was therefore that no phone could be trusted to remove the old ticket. The only place where revocation could be guaranteed was the gate, by giving every venue’s scanners a list of revoked barcodes they hold even when offline.

That one finding changed the product before any of it was built. It added the Venue Entry team, which owns the door scanners, to the teams the bet would touch. It changed what the seller would be told, since the honest message is that the old ticket will be refused at the door, whatever the phone still shows. And it turned a promise the trio had been about to make in the memo, that the old ticket would vanish from the seller’s phone, into one Encore could keep. The engineering of revocation belongs to the Software book; Chapter 3 returns to this prototype as an example of its kind.

Discovery and delivery as two tracks

If discovery reduces risk before delivery, it is tempting to treat the two as phases: first we discover, then we build. Practice has settled on a different arrangement, and its origin is worth knowing. In 2007 Desirée Sy described in the Journal of Usability Studies how her team at Alias, later part of Autodesk, ran user research and design one iteration ahead of development, in parallel tracks, so that designs were tested before they were built while the previous iteration’s designs were being built. Jeff Patton, in an essay of 2017 that credits Sy and notes that Cagan spread the term, calls the arrangement dual-track development: one team running discovery, which produces validated ideas, alongside delivery, which produces releasable software, continuously and at the same time. His summary of why it matters is hard to improve: “The most expensive way to test your idea is to build production quality software.” Patton insists on one condition, and Figure 1.5 draws it as a single frame around both tracks. The two tracks are not two teams and not two phases; the same people work in both, every week.

Figure 1.5 — Dual-track development as a schematic: each idea is tested in discovery one iteration before delivery builds it, and an idea that fails its test, like C here, is dropped before anyone writes production code for it.

A touring production offers a good picture of the arrangement. While the crew loads in and runs tonight’s show, an advance team is already at next week’s venues, checking the stage dimensions, the power supply and the loading dock, so that problems are found while they are still cheap to solve. The analogy holds for the rhythm: the advance work never stops while the tour is running. It breaks on one important point. The advance team visits venues that already exist, while discovery explores products that do not exist yet, and much of what it learns is that the planned destination should change. Feasibility questions, such as Katrin’s, are where architecture work enters discovery; the Architecture book’s Chapter 1 showed how the characteristics a system must have are heard in what the business says, and the tech lead is the person in the trio who listens for them.

Recap. Cagan’s four risks, value, usability, feasibility and viability, sort what could make a product fail, and the ethical question asks whether it should be built at all. The product manager, designer and tech lead own the risks together as a trio, working as one unit from the first interview. A tech lead in discovery can change a product before it is built, as Katrin’s wallet prototype did. Discovery and delivery run as two continuous tracks worked by the same people.

Exercise 1.3. For one feature on your roadmap, write one sentence per risk naming the evidence you have, or saying plainly that you have none. Then add a sentence for the ethical question, and name who in your organization would answer it if you did not.

1.4 The Product Operating Model

Three changes

Most companies that adopt the practices of this chapter adopt them one at a time, and get less from each than they expected, because the practices depend on one another. Cagan and his co-authors, in TRANSFORMED (2024), describe the whole as the product operating model: a way of building, solving problems and deciding which problems to solve, in which teams are given problems to solve and judged by outcomes. They describe the move to it as three changes. The first changes how a company builds: small, frequent, independent releases instead of large and infrequent ones. The second changes how it solves problems: discovery before delivery, by the team that will build the solution. The third changes how it decides which problems to solve: from a list of features chosen by stakeholders to a strategy, made from insight and turned into objectives for teams.

The evidence behind these three changes is uneven, and it is worth being exact about it. The first change has strong quantitative support from a decade of research by DORA (DevOps Research and Assessment), which the Software book’s Section 1.1 examines. The second and third rest on case stories, many of them gathered in TRANSFORMED itself, and on long practice; nobody has run a comparative study. The best large-scale demonstration is public. In 2014 the British government made the same three changes into policy with its Digital by Default Service Standard, and its current version, of 2019, has 14 points, among them “Understand users and their needs”, “Have a multidisciplinary team”, “Iterate and improve frequently” and “Define what success looks like and publish performance data”. Central government services in the United Kingdom are expected to meet the standard, and many are formally assessed against it before they go live. Encore has made the first change thoroughly; it deploys about 180 times a working day. It has made the second change only where trios exist, and the third only in parts of the company.

Empowered teams, feature teams, delivery teams

The difference that matters most is what a team is given. Cagan’s essay “Product vs. Feature Teams” (2019), developed in EMPOWERED (2020) with Chris Jones, names three kinds of team, and Figure 1.6 sets them side by side.

Figure 1.6 — Three kinds of team, after Cagan and Jones. The difference lies in what leaders assign, and the same people can work in any of the three.

An empowered product team is a cross-functional team given a problem to solve and an outcome to reach, and trusted to find the solution. Cagan’s description of feature teams as “all about output” is exact: they deliver what is asked of them, and someone else carries the risk that it will not work. The most important observation in his essay is about cause. A team is usually a feature team, he writes, because “the leadership does not yet trust that particular team.” Whether a team is empowered is decided by what its leaders assign, and that is why a team cannot simply decide to become empowered. At Encore the fan storefront’s teams are given problems such as “more admitted fans complete a purchase”, and the organizer console’s teams have been given lists. The difference between them lies in what each was given to do, whatever the talent of the people.

Strategic context

Empowerment alone is dangerous at scale. Forty teams, each solving its own problem in its own way, will pull in forty directions unless something holds them together. Cagan and Jones’s answer is strategic context: the shared knowledge, provided by leaders, that lets empowered teams make decisions consistent with each other and with the company’s direction. They name five elements, and this book uses their names and gives each a precise meaning.

A product vision is the future the company is trying to create for its customers, described far enough ahead, usually several years, to inspire and to guide many teams at once. Product principles are the beliefs that guide trade-offs when the vision does not settle them; one of Verified Resale’s will be that the fan never pays more than the organizer allows. The team topology is how the teams are divided, and what each is responsible for, the subject of Figure 1.3. The product strategy is the choice of which problems to solve, in what order, to move toward the vision; it is the subject of Chapter 2. And team objectives are the specific problems each team is asked to solve, with the outcomes by which progress will be judged, which Section 2.5 turns into practice.

Of the five, strategy is the one most often missing, and its absence is the hardest to see. A company can have an inspiring vision, sensible principles and a well-drawn topology, and still hand its teams a list of features each quarter, because nobody has decided which problems matter most. That gap is where the organizer console’s queue came from. The Software book’s Section 9.3 covers the engineering roles that make strategic context work across 40 teams; the context itself, and the strategy at its heart, are this book’s subject.

Working with sales, finance and leadership

A product team does not decide alone, and the product operating model changes its relation to the rest of the company in three ways. With sales, commitments to customers become inputs to discovery, and stop being placed directly on a team’s work. The organizer sales team still hears what organizers want, and its knowledge is invaluable, but what it brings back is now a problem to be assessed, as in the opportunity assessment of Section 1.1, where it used to be a solution with a date. Chapter 3 shows how such requests are handled in discovery. Organizer sales is Encore’s hard case, because its requests are attached to contracts with renewal dates, and saying “we will look into the problem” to a commercial director with a renewal on Friday takes nerve and support from above.

With finance, the change is from funding projects to funding teams, so that a team can stay with a problem long enough to solve it. Anyone who has lived through a large bank’s annual budget round, as I have, knows how strongly the project shape is held in place by the way money is approved. The mechanics of the change are the Software book’s Section 10.4. With leadership, the change is from reviewing roadmaps to reviewing outcomes, and from approving features to providing context. Verified Resale’s leaders will see it as a bet memo, with a diagnosis, the outcome it aims for, the guardrails it must not break and the conditions under which it would be stopped. Amazon’s PR/FAQ, the press release and questions written before any work begins, which Colin Bryar and Bill Carr describe in Working Backwards (2021), is a heavier form of the same document. Chapter 10 follows Encore’s memo from the day it is written.

Innovation theater

There is a failure that looks like success, and it deserves a name. A company can adopt the whole vocabulary of this chapter, trios, outcomes, discovery and objectives, and change nothing about who decides what is built. TRANSFORMED warns of it in a passage on what it calls “innovation theatre”, and the term is in wide use for visible innovation activity, labs and hackathons among them, that never reaches the product teams. I use theater here, in a slightly broader sense, for any adoption of the product operating model’s language without its transfer of decisions.

The signs are recognizable. A roadmap renamed an “outcome roadmap” still lists features, each with a metric attached after the fact. Discovery is run after the decision has been made, to confirm it. Objectives are set for teams and then overridden whenever a large customer asks. The trio exists on the organization chart, but the product manager’s real job is to take requests from stakeholders and turn them into tickets. Each sign is a gap between what the company says and who actually decides. Encore’s organizer console was showing two of them when the Verified Resale proposal arrived: its teams had trios and objectives, and their work still arrived through the queue.

Recap. The product operating model consists of three changes, in how a company builds, solves problems and decides which problems to solve, and only the first has strong quantitative evidence. Empowered teams are given problems and outcomes; feature teams are given features; the difference is set by leaders. Strategic context, made of vision, principles, topology, strategy and objectives, keeps empowered teams pulling the same way. Theater is adopting the vocabulary without moving the decisions.

Exercise 1.4. List everyone in your company who can add an item to a team’s work without that team’s agreement. For each path, decide whether it should remain a direct path or become an input to discovery, and say who would have to agree to the change.

1.5 What the AI Era Changes, and What It Does Not

Cheaper construction, unchanged essence

AI assistance has lowered the cost of producing code, and the Architecture book’s Section 15.4 argued what that means for architecture: as code gets cheaper to produce, judgment becomes the scarce part of the work. The product consequence follows directly, and it is the only new thing this section adds. If building gets cheaper while deciding stays as hard as it was, then building the wrong thing becomes a larger share of what a product costs. Brooks’s remedy for the hardest part, prototypes put in front of the people who will use the system, becomes cheaper to apply. It does not become unnecessary, because the prototype still has to be shown to someone, and someone still has to decide what it showed.

The build trap, faster

An organization caught in the build trap measures itself by what it ships. Give it tools that make shipping cheaper, and we should expect it to ship more: more features, more surface area, more for users to learn and more for teams to own. The cost of ownership is already known. Robert Glass’s widely cited estimate, which the Architecture book’s Section 13.3 uses, puts maintenance at 40% to 80% of software’s cost, about 60% on average. If two-thirds of ideas fail to move the metric they were meant to move, as Microsoft’s experiments found, then an organization that ships twice as many ideas without examining them acquires twice as many unused features to maintain. No published study has yet measured whether AI assistance inflates feature counts in this way, so this is an expectation, drawn from known costs. It is a reason to put the outcome review of Section 1.1 in place before the cheaper tools arrive in force.

The evidence so far, in brief

The evidence on AI assistance and developer productivity is young, mixed and moving, and the Software book’s Section 5.1 examines it in full. In a laboratory task, Peng and colleagues found in 2023 that developers with GitHub Copilot finished one programming task 55.8% faster. In field experiments with 4,867 developers at three companies, Cui and colleagues, in Management Science in 2026, found about 26% more completed tasks, with larger gains for less experienced developers. In contrast, METR’s study of 16 experienced open-source developers working in their own mature repositories found them 19% slower in early 2025, although they believed they had been faster; METR has since marked that result out of date, and reported in February 2026 that its newer data are too weak to size the effect. DORA’s report of September 2025 linked AI adoption to higher delivery throughput, a reversal of its 2024 finding, while delivery stability still declined. The honest summary is that the gains are real but uneven, and that perceived speed tends to run ahead of measured speed.

Where AI helps deciding

For product work, the most promising effect of AI is on discovery itself: prototypes that once took a sprint can now be produced in hours, so a trio can put more versions of an idea in front of customers before choosing. That practice is young, and Section 3.5 presents it as emerging, with the practitioner accounts it rests on. AI also changes the four risks when the product itself contains an AI feature, and here the practice is established. Feasibility then includes whether the feature passes evaluations against a set of real cases, and viability includes what each use costs in model tokens, which Encore already counts in its cost per ticket, as the Resilience book’s Section 2.5 shows. Verified Resale will have one such feature, an assistant that turns an organizer’s plain-language rules into the typed rules the platform enforces, and Chapter 6 designs it.

The rule for this book

Every claim this book makes about practice, AI or otherwise, is held to one rule, which the Software book shares: proven practices only; anything emerging is labelled and carries a real, specific account of use. For AI claims in particular, that means treating each one as a product team treats a hypothesis. It needs a named source, a stated setting and a measured result, and the reader should be told whether it is proven or emerging. The rule applies to the book the same discipline the book asks of product teams.

Recap. Cheaper construction leaves the hardest part, deciding what to build, as hard as it was, so the cost of deciding wrongly grows as a share of the whole. An organization in the build trap will use cheaper tools to ship more unexamined work, and the remedy is the outcome review, installed early. The evidence on AI productivity is mixed and moving, and this book labels every claim as proven or emerging.

Exercise 1.5. Find one claim about AI productivity made inside your company. Trace it to its evidence: which study or measurement, in what setting, measuring what. If it traces to nothing, write down what measurement would settle it.

Kata: The Policy Factory

Your brief: “Haverlock Insurance.” Haverlock sells home and motor insurance in three EU countries, through its own website and app and through 2,000 independent brokers who use a broker portal. Its 150 engineers work in 22 teams, and every one of them is funded project by project. Each autumn the business units bid for the next year’s projects, a steering committee ranks them, and teams are assembled to deliver the winners and disbanded when the projects close. Last year Haverlock delivered 31 of its 34 approved projects on schedule, and the steering committee called it the best year on record. In the same year online quotes that became policies fell from 14% to 11%, broker complaints about the portal doubled, and nobody could say which of the 31 projects had been meant to change either number. The largest broker network has threatened to move its business unless the portal “gets the features we were promised”. The chief executive has asked you to explain what went wrong in a year when everything was delivered, and to propose a different way of working for three areas: online quotes, the broker portal and claims.

Deliverables:

1. The diagnosis — A one-page account of Haverlock’s build trap. Name its outputs and the outcomes nobody measured, say which of Perri’s three distortions dominates in each area, and answer her two questions: who came up with the last feature built, and what was the last thing killed.
2. The product map — A map of Haverlock’s products, customers and teams in the manner of Figure 1.3, marking which customers have teams of their own and which products lack a single accountable owner.
3. Three empowered teams — For online quotes, the broker portal and claims, the problem each team would be given, the outcome by which it would be judged and the trio that would lead it.
4. The four risks — For the broker network’s demand, an opportunity assessment and a table in the manner of Figure 1.4, stating the evidence for each risk or its absence.
5. The strategic context — A draft of Haverlock’s product vision, three product principles and the objective for each of the three teams.
6. The outcome review — A one-page procedure for reviewing every release against its outcome, with who attends, when, and what happens to work that moved nothing.

Rubric:

Where you now stand

You can now tell an output from an outcome, and recognize the build trap in a team’s work by how that work arrives and whether it is ever judged afterwards. You can draw an organization’s products, customers and teams on one map, sort a product idea’s risks into value, usability, feasibility, viability and ethics, and say who in a trio owns each. You can name the elements of strategic context, and tell a company that has adopted the product operating model from one that has adopted its vocabulary. What you cannot yet do is decide whether Verified Resale deserves to exist, because that depends on a question this chapter has deliberately left open: what problem Encore is trying to solve, and what it will refuse to do in order to solve it. That is a question of strategy, and Chapter 2 takes it up, beginning with four words on an Apple whiteboard in 1997.

References

• HHS Office of Inspector General — HealthCare.gov: Case Study of CMS Management of the Federal Marketplace, OEI-06-14-00350. US Department of Health and Human Services, 2016 — free online.
• Melissa Perri — Escaping the Build Trap: How Effective Product Management Creates Real Value. O’Reilly, 2018.
• Marty Cagan — INSPIRED: How to Create Tech Products Customers Love, 2nd ed.. Wiley, 2017.
• Marty Cagan, Chris Jones — EMPOWERED: Ordinary People, Extraordinary Products. Wiley, 2020.
• Marty Cagan, Lea Hickman, Christian Idiodi, Chris Jones, Jon Moore — TRANSFORMED: Moving to the Product Operating Model. Wiley, 2024.
• U.S. Digital Service — Digital Services Playbook. 2014 — free online.
• Steven Brill — Obama’s Trauma Team. Time, 2014 — free online.
• Frederick P. Brooks Jr. — No Silver Bullet: Essence and Accidents of Software Engineering. Information Processing 86, IFIP, 1986; reprinted in The Mythical Man-Month, anniversary ed., Addison-Wesley, 1995 — free online.
• Josh Seiden — Outcomes Over Output: Why Customer Behavior Is the Key Metric for Business Success. Sense & Respond Press, 2019.
• Marty Cagan — The Four Big Risks. SVPG, 2017 — free online.
• Teresa Torres — Continuous Discovery Habits. Product Talk, 2021.
• Jeff Patton — Dual Track Development is not Duel Track. Jeff Patton & Associates, 2017; and Desirée Sy, Adapting Usability Investigations for Agile User-Centered Design, Journal of Usability Studies 2(3), 2007, 112–132 — free online.
• Ron Kohavi, Alex Deng, Brian Frasca, Roger Longbotham, Toby Walker, Ya Xu — Trustworthy Online Controlled Experiments: Five Puzzling Outcomes Explained. KDD, 2012 — free online.
• Zheyuan Cui, Mert Demirer, Sonia Jaffe, Leon Musolff, Sida Peng, Tobias Salz — The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers. Management Science, 2026.
• DORA — State of AI-assisted Software Development. Google Cloud, 2025 — free online.
• UK Government Digital Service — Service Standard. GOV.UK, 2019 — free online.