Building Software at Scale in the AI & Cloud Era

← the whole book on one page

Chapter 10: Planning and Flow — Economics, Forecasts, and Funding

A plan earns its keep on the day it tells us what to stop doing.

Forty teams can each be well designed, well led and well tooled, and the organization they form can still fail to deliver anything that matters on time. The failure lives between the teams, in the questions no single team can answer for itself. Which of the many good things we could build should come first? When will a piece of work that crosses a dozen teams actually be done, and how sure are we? Who pays for the work that keeps the lights on, and who decides when a bet has earned more money? Chapter 9 designed the teams. This chapter is about the economics, forecasts and funding that let those teams act together, and about the measures that tell an organization whether it is getting better at delivering or only better at reporting.

These questions have a long history, and much of it has been told as a triumph of control. The founding story of modern project management is the US Navy’s Polaris missile programme, which put a ballistic missile on a submerged submarine years ahead of its original schedule and was credited with inventing the Program Evaluation and Review Technique, PERT, the network-and-estimate method that spread from there into every defense contract and project-management textbook. The story is true in its outline and misleading in its moral. Historians who looked closely found that PERT’s greatest service to Polaris was political, and that the programme succeeded through priority, protection, a motivated organization and a willingness to cut scope. Those four things are the subject of this chapter, and we will meet them again under modern names.

Encore needs them now. At month 4 of the Verified Resale bet, in the late spring of year nine, discovery is done and the decision to build has been made. The first quarterly plan for the build shows 12 teams with work to do, 23 dependencies among them, and a launch date that nobody in the room believes. The Products book’s Chapter 10 tells how the trio decided what to build. Here we follow Encore as it prices the cost of delay against the autumn tour season, makes its dependencies visible and removes the ones it can, forecasts the partner API from the throughput its teams have actually achieved, funds the bet in tranches released against learning, and measures its delivery without turning the measures into targets.

10.1 The Economics of Flow

Polaris and the network on the wall

Late in 1955 the US Navy created a Special Projects Office to build a ballistic missile that could be launched from a submarine, and appointed Rear Admiral William F. Raborn to direct it. The office was set up deliberately outside the Navy’s Bureaus of Ordnance and Aeronautics, whose rivalry might otherwise have strangled the programme. Its original target was an interim capability around 1963. Then, in October 1957, the Soviet Union launched Sputnik, and the schedule was compressed to 1960. On 20 July 1960 USS George Washington made the first Polaris launch from a submerged submarine, and the commander’s message to President Eisenhower read, “Polaris — from out of the deep to target. Perfect.” The submarine left on the Navy’s first deterrent patrol that November, roughly three years ahead of the original plan by the Navy’s own account. Figure 10.1 sets the dates against the plan they overtook.

Figure 10.1 — Polaris from the Special Projects Office’s creation to the first deterrent patrol, drawn as a schematic timeline. The date held because the scope gave way: the A-1 went to sea with less range and a smaller warhead than first wanted.

The technique the programme made famous was described in 1959 in Operations Research by D. G. Malcolm, J. H. Roseboom and C. E. Clark of Booz, Allen & Hamilton, with W. Fazar of the Special Projects Office. PERT drew a research and development programme as a network of events joined by activities, and it asked the engineers responsible for each activity to estimate its duration “in probability terms”: an optimistic time, a most likely time and a pessimistic time, combined into an expected time weighted four to one toward the most likely. The longest chain through the network, the critical path, set the date. Anyone who has watched a touring crew load into an arena has seen the same idea at work. The rigging must be flown before the lighting can hang from it, the lighting must be focused before the sound check, and the sound check must finish before the doors open, so the stage manager watches the one chain of tasks that leaves no slack and lets the others float. That was PERT’s lasting gift: it made dependencies visible, and it treated a duration as a range.

Its reputation outgrew its use, as Harvey Sapolsky showed in The Polaris System Development (1972), a study with a chapter titled “PERT and the Myth of Managerial Effectiveness.” PERT never governed most of the programme. Contractors resented it and set up units to process their data before it reached the Navy, and the Special Projects Office never evaluated whether the technique worked. Its real value, Sapolsky argued, was to give the office a reputation for management excellence, and with it the freedom to work without interference from the Pentagon, Congress and rival services. Sylvain Lenfle and Christoph Loch, drawing on Graham Spinardi’s later history, summarize the point: “The image of managerial efficiency helped the project. It mattered not whether parts of the system functioned or even existed, it mattered only that certain people for a certain period of time believed that they did.” What delivered Polaris was a national priority, an organization protected from its rivals, people who believed in the work, and scope that gave way when the date could not. The missile that went to sea in 1960, the A-1, had less range and a smaller warhead than the Navy had first wanted, and Lenfle and Loch describe a programme that put the schedule ahead of cost and specification and changed the specifications as it went. The full reach and yield came with later models.

We should honour both halves of this history. PERT’s network thinking is real, and so is its idea that an estimate is a distribution. Its three-point estimate was a first attempt at a probabilistic forecast, and its known flaw, which MacCrimmon and Ryavec analyzed in 1964, points at the remedy Section 10.3 teaches: when several uncertain paths merge, the critical-path calculation is systematically too early, because the merge waits for whichever path finishes last. The other half is a warning about every planning artifact we will meet. A plan can be a tool that changes decisions or a performance that reassures the people watching, and the same document can be both.

Cost of delay

The economic question underneath every plan is what it costs to be late. Donald Reinertsen, whose The Principles of Product Development Flow (2009) is the most rigorous treatment of the subject, argues that if an organization quantifies only one thing, it should be this. Cost of delay is the value lost for each unit of time a piece of work is not yet delivered, usually expressed per week. With it, time, scope, risk and money can be traded in one currency, and the question of whether to spend two more weeks on a feature or ship it now becomes arithmetic that anyone in the room can check, where before it was settled by argument and seniority.

Cost of delay also gives a rule for sequencing. When two pieces of work compete for the same people, do first the one with the highest cost of delay divided by its duration. Reinertsen calls this weighted shortest job first; Joshua Arnold, who applied it in practice, later named it CD3, cost of delay divided by duration, and the two names describe one rule. Its logic is that of a box office with one window. If one customer needs a single ticket and another needs to rebook a school trip, serving the quick one first costs the slow one a minute and saves the quick one half an hour. The analogy breaks where the economics begin: customers at a box office suffer equally per minute of waiting, while pieces of work do not, and CD3 weights the queue by that difference.

The best public account of the rule in use comes from Maersk Line, where Arnold and Özlem Yüce introduced it to an IT portfolio of about US$100 million between 2010 and 2013. Their Agile 2013 experience report is the consultants’ own account, and its numbers are instructive all the same. Before the change, a quarter of requirements took more than a year to deliver. One feature needed 82 hours of work and took 46 weeks, 38 of them spent waiting, at an estimated cost of delay of about US$210,000 a week. Once cost of delay was quantified across the portfolio, the most valuable quarter of requirements turned out to be worth roughly 1,000× the least valuable quarter, and after sequencing by CD3 cycle time on the main system roughly halved. The skew is the important finding. When value is distributed that unevenly, rough estimates are good enough to sequence by.

That robustness is the condition under which cost of delay works: it is an estimate used to rank work against other work, with its assumptions written down and its ranges stated, re-estimated as information arrives. Reinertsen himself warns that intuitive estimates of it vary enormously between people looking at the same work, and the remedy is an explicit model, so that disagreement becomes a disagreement about an assumption that can be checked.

Encore wrote its model for Verified Resale on one page. The steady part of the cost had two sources: the fee shares, Encore’s three points and the organizer’s three, on the resales the product would complete each week, estimated from the four-show pilot in Lyon and Antwerp and the grey-market listings around the Vesper Hale tour; and the cost of the grey market it would displace, in support contacts about duplicated barcodes, chargebacks and organizers lost to other channels. Together these came to between €30,000 and €70,000 a week once the product was live. The other part was a step. Organizers sign the next year’s contracts in the autumn, and the Products book’s Section 2.5 made the resale clause in those contracts Encore’s high-integrity commitment; if the partner API was not in Rowmate’s hands by the start of October, a whole season of tours would go on sale without resale. Arnold calls this shape an urgency profile, and David Anderson’s Kanban method turns such profiles into classes of service, standing policies for treating work of each shape: expedite, fixed date, standard and intangible. Verified Resale was fixed-date work until October and standard work after it.

Figure 10.2 — Encore’s cross-team initiatives at month 4, ranked by cost of delay divided by duration. The portfolio holds five; the lowest-ranked initiative waited.

Verified Resale did not come first in Figure 10.2, because its duration was long. CD3 rewards short jobs, so the cheapest way to raise a long initiative’s priority is to make it shorter, and every week cut from the build raised the bet’s ranking against everything else. That is one reason the cuts described in Section 10.3 were made early.

Queues are invisible inventory

A factory floor shows its inventory: half-finished parts stack up between machines where any manager can see them. Product development hides its inventory in tickets, unmerged branches and pull requests awaiting review, none of which occupies space or appears on a balance sheet. Reinertsen’s central claim is that these queues, work that has started and is waiting for someone, are the largest hidden cost in most development organizations, and Maersk’s 38 weeks of waiting in 46 is typical of what people find when they measure.

The mathematics has been taught twice already in this series, for servers, and we need only carry it across to people. The Architecture book’s Section 4.3 introduced Little’s law, proved by John Little in 1961: the average number of items in a stable system equals the arrival rate multiplied by the average time each item spends inside. The Resilience book’s Section 2.2 developed J. F. C. Kingman’s result of the same year: waiting time grows in proportion to ρ/(1−ρ), where ρ is utilization, so the wait at 95% utilization is nearly five times the wait at 80%, rising without limit toward 100%.

Applied to a team, Little’s law says that cycle time is work in progress divided by throughput. A team that finishes six items a week and has 30 started will take five weeks, on average, to finish each one, however hard everyone works. The law holds for long-run averages in a stable system, and a team whose work in progress grows week by week is not stable; its future cycle times will be worse than its past ones. Applied to a reviewer, Kingman’s result says that a person who is busy all the time has a long queue in front of them, and the manager pleased to see every engineer fully utilized is looking at the cause of the delays.

Encore met both laws in one place. In the first weeks of the build, months four and five, the Resale team’s changes to encore-core took a median of a little over two working days from opening to merge, and a change touching the barcode core took a median of 11 days from first commit to production, against two for the rest of Seat Inventory & Ticketing. The Inventory team was not slow; it was full. Its engineers were committed to their own roadmap at close to all of their time, and changes touching the barcode core needed review from Lena, the one remaining author of that code and the team’s busiest person before Verified Resale arrived. The remedies were the ones queueing theory suggests, and Section 4.3 tells them: reserved review time lowered utilization where it mattered, and pairing with Lena from month 5 widened the single server into three. By month 8 the median from opening a change to merging it was about seven working hours. The barcode core’s own queue drained more slowly, because it emptied only as knowledge spread, and Chapter 11 follows it from 11 days to six by launch. Coding assistants can make this problem worse, because they raise the arrival rate into exactly these queues.

Batch size

If queues are the hidden inventory, large batches are one of the main ways we create them. A release of many changes, or a pull request touching 40 files, holds finished work back until the slowest part is ready and makes every downstream step bigger and riskier. The Resilience book’s Chapter 4 showed why small changes are safer to deploy; the economic argument is separate.

Reinertsen frames it as a U-curve. Each batch carries a transaction cost, the fixed effort of building, testing, reviewing and deploying it, and a holding cost, the value delayed and risk accumulated while work waits for the batch to fill. Large batches spread the first thinly and pile up the second, small batches do the opposite, and the total is lowest somewhere between. Exhorting people to ship smaller therefore rarely works, because they already sit near the bottom of the curve their transaction costs allow. What moves the optimum is a lower transaction cost, which is the economic case for the fast continuous integration and trustworthy review of Chapter 4. A 31-minute build in encore-core gives every engineer who waits for it a reason to batch.

Accelerate (2018), by Nicole Forsgren, Jez Humble and Gene Kim, lists working in small batches among the capabilities that predict delivery and organizational performance, and DORA’s 2024 report named small batches and robust testing as basics that AI assistance does not replace.

WIP and the five thieves of time

Dominica DeGrandis, in Making Work Visible (2017), names five thieves of a team’s time: too much work in progress, unknown dependencies, unplanned work, conflicting priorities and neglected work. She calls the first the ringleader, because each of the others leaves another item started and stalled. Work in progress, or WIP, is everything started and not finished, and a WIP limit is an explicit cap on how much of it a team or an organization will carry at once. The idea is easy to state and hard to hold to, because starting something new feels like progress and finishing something old often feels like a chore.

Air traffic control offers a useful model. When weather cuts an airport’s landing capacity, controllers hold aircraft on the ground at their origins, where waiting is cheap and safe, and release them at the rate the destination can absorb, instead of letting every flight take off and circle. That is a WIP limit applied to a whole network, and it works because circling aircraft burn fuel and crowd the airspace, as started work consumes attention. The analogy breaks at measurement. An airport knows its landing rate, while a portfolio’s capacity is uncertain and changes as work is started, so WIP limits in software are set from observed throughput and adjusted as the queues reveal themselves.

WIP limits help when the work is visible, so that the limit is seen to bind, and when someone acts on the queues the limit exposes; a limit quietly exceeded teaches people that limits are decoration. The strongest published case near Encore’s scale is Siemens Health Services, an organization of about 50 teams, reported by Daniel Vacanti and Bennet Vallet in 2014. After a WIP policy was applied across all teams in September 2012, story cycle time stabilized at about 41 days, and its 85th percentile fell from 71 days to 43. Before the policy, cycle times had grown release after release, the signature of a system Little’s law calls unstable.

Encore limits WIP at two levels. Each team sets its own limits on its board. Across teams, Encore caps company-wide cross-team initiatives, those needing planned work from three or more teams, at five. The rule is Encore’s own, dating from the months after Vesper Hale, when the post-incident work ran alongside four other initiatives and every one of them slipped. Verified Resale displaced the rebuild of Organizer Insights’ reporting, the lowest item in Figure 10.2, which waited two quarters. The decision was uncomfortable for the team that had planned the rebuild, and the published ranking, with its assumptions on view, made it bearable.

Cadence and decentralized control

Coordination is expensive when it must be arranged each time and cheap when it happens on a rhythm everyone knows. Reinertsen calls this cadence: a regular rhythm for planning, review and integration that makes coordination predictable. Encore’s quarter, its fortnightly operating reviews and its weekly fan-app release train are cadences of this kind, and their value lies in their reliability more than their length. A team that knows the next planning point is six weeks away can defer a question with confidence.

Cadence answers when to decide; decentralized control answers who decides. Reinertsen draws on the US Marine Corps doctrine set out in Warfighting (1989): decisions belong where the information is freshest, with people who understand the commander’s intent well enough to act on it when the plan no longer fits the ground. A central planner cannot know, week by week, whether a team should spend two more days hardening an endpoint or ship it. The team can, if it knows what delay costs.

Maersk published its cost-of-delay figures to every team so that trade-offs could be made locally, and Encore did the same. The one-page model went to all 12 teams with a decision rule: any choice that saves more than a week of the bet’s critical path, at a cost of less than €50,000 in other work, may be made by the teams involved without asking anyone. Most of the trade-offs in the following months were made under that rule, several by teams the trio never heard from.

Recap. Polaris shows that planning can be a tool or a performance, and that priority, protection, motivation and flexible scope delivered the programme. Cost of delay puts time, scope and value in one currency and, divided by duration, sequences work, robustly because value is so skewed. Queues are inventory nobody sees, and the laws of queues apply to people. The lever on batch size is the cost of each batch, WIP limits work when someone acts on the queues they expose, and a published cost of delay lets teams decide locally.

Exercise 10.1. Estimate the weekly cost of delay of your team’s three largest items, with the assumptions behind each and each item’s urgency profile. Rank them by cost of delay divided by duration and compare that order with the order in which you are actually doing them. Where they disagree, find the assumption that would have to change for your current order to be right.

10.2 Planning Across 40 Teams

Planning in three phases

Most planning pain in large organizations comes from arguing about the wrong thing first. Teams arrive with roadmaps that add up to more people than the company has, and the meeting becomes a negotiation over features that is really about headcount and money. Will Larson, in the fourth chapter of The Engineering Executive’s Primer (2024), proposes an order that dissolves much of this: first the financial plan, how much the company will spend on engineering; then the portfolio allocation, what share of that capacity goes to each area and kind of work; and only then the roadmap, what each area will build with its share. Each phase is settled before the next begins.

This is one experienced executive’s prescription, widely cited and not formally studied. Its logic is that of a building budget: an architect who designs the rooms before knowing the budget designs a house nobody can afford, and the later argument about which rooms to cut is harder than the argument about money would have been. The analogy breaks because a building’s budget is fixed at the start, while an engineering organization’s is revisited yearly, which is why Section 10.4 replaces the annual project budget with funding that moves.

Encore runs its annual plan in that order. In year nine the envelope was roughly flat at about 400 engineers; the allocation of Section 10.4 followed; and a roadmap that asked for more than its allocation was returned with a question about what it would drop.

The quarterly cycle

The Products book’s Section 2.5 explains how strategy becomes objectives and what a high-integrity commitment is. What remains here is the calendar. Encore’s quarter has four fixed points. Three weeks before it begins, each team drafts objectives against the annual intent and its allocation. In the following fortnight, areas reconcile the drafts and surface the dependencies between them. In the quarter’s first week a dependency session settles what crosses team boundaries. At mid-quarter each team holds a short check-in on whether its objectives are still right and its commitments at risk.

Marty Cagan and Chris Jones, in Empowered (2020), insist that high-integrity commitments be rare and tracked apart from ordinary objectives. At Encore a commitment has an owner, a date, the dependencies it rests on and a forecast, and it appears on the operating review’s board every fortnight until it is met or explicitly renegotiated. Missing an objective is information; missing a commitment is an event leadership hears about before the date.

The two books use different commitments for Verified Resale. The Products book’s is the resale clause in the autumn organizer contracts. This book’s is the engineering promise that made the clause honest: the partner API, with reissue behind it, in Rowmate’s sandbox by the start of October. Everything else in the build was an objective.

Making dependencies visible

A dependency, in planning, is a piece of work one team needs from another before its own work can finish. Each is a small contract between teams and a queue in waiting, because the providing team has its own priorities. DeGrandis advises exposing dependencies before managing them: tag them on cards, draw them as a matrix of teams against teams, and look at the picture before scheduling anything. Accelerate explains why: a loosely coupled architecture, in which teams test and deploy without fine-grained coordination, was among the strongest predictors of delivery performance it found.

The better response to a dependency is usually to design it out. Section 9.2 drew Encore’s team boundaries so that most work starts and finishes inside one team, and a dependency that remains is a sign that the boundary does not fit this piece of work. Encore’s triage gives each dependency one of three answers. Remove it by moving the work to the team that needs it, or by changing a boundary. Contract it, by turning it into a stable, self-service interface. Coordinate it, by putting it on the plan with a date, an owner on each side and a place on the operating review.

Verified Resale’s first plan showed 23 dependencies across the 12 teams, and Encore removed nine of them by moving work, leaving the schedule alone. The Resale team took over the resale module in the fan app, which the plan had assigned to Mobile App, and Mobile App’s part shrank to the release train and the version floor. Several of the other removals were of the same kind: small pieces of screen and configuration that the first plan had spread across teams for no better reason than that each team owned the screen beside them. Partner credentials and webhook delivery, two dependencies on Security Engineering and Partner APIs, became platform capabilities through the work Chapter 6 describes, and so left the plan altogether. By month 9 the plan showed 14 dependencies. Eight were contracts the Resale team consumed through documented interfaces, such as Fan Messaging’s pass-replacement API. Six required real coordination, and one of them, reissue inside the barcode core, was the dependency that would block for a quarter. Chapter 11 tells what happened to it.

Cross-team planning events and their alternatives

The best-known way to plan across many teams at once is the Program Increment planning event of the Scaled Agile Framework (SAFe), in which every team on a release train meets for two days to plan the next eight to 12 weeks and map its dependencies on a shared board. It is widely adopted, and its supporters report better transparency and handling of dependencies. The evidence consists mainly of case studies, many published by the framework’s own vendor, and practitioner surveys; a 2018 multivocal literature review by Abheeshta Putta, Maria Paasivaara and Casper Lassenius found the peer-reviewed evidence thin, and there are no controlled comparisons. Big-room planning may well help, and we do not know by how much.

At the other end sits Basecamp’s Shape Up, whose betting table the Products book’s Section 3.4 adapts for Encore. A third approach is rolling-wave planning: the near horizon is planned in detail and the far horizon in outline, with the detail rolled forward as the horizon approaches. It is old practice in engineering projects, and it matches what forecasting can honestly support.

Encore combines the second and third with a piece of the first. Inside teams, work is shaped and bet on in six-week chunks. Across the 12 Verified Resale teams, a one-day dependency session at the start of each quarter, attended by each team’s tech lead and product manager, produces one output: the dependency map, triaged and dated. Beyond the coming quarter the plan shows milestones and forecasts, never task lists. The session borrows big-room planning’s best feature, a picture of dependencies drawn by the people who own them, without asking 12 teams to plan every story together.

Planning for unplanned work

Every team does work nobody planned: incidents, support escalations, urgent patches, a partner’s broken integration. A plan that ignores it will be missed, and the miss will be blamed on the team. The remedy is to treat unplanned work as a planned input: measure each team’s historical share, reserve it, and plan the rest. Google’s SRE book puts a number on the operational form, a goal of keeping toil below 50% of each engineer’s time.

The Operations book’s Section 3.1 described how Encore measured its toil, by having on-call engineers tag each interrupt for a quarter, and found it took about 30% of the on-call week, with the On-Sale team near 45% in the touring season. Encore extended the same tagging to all unplanned work and used it as the planning input. Each team’s reserve is its own trailing two-quarter share, which averages about a fifth of capacity and runs from about one-eighth on the quietest teams to over a third for On-Sale in spring. A team with reserve left over pulls work forward; a team that overruns says so at the check-in, where the overrun is treated as a question about the work.

The four team states of Section 9.2, from Larson’s An Elegant Puzzle (2019), apply here. A plan that gives new work to a team falling behind only deepens the hole. A steadily rising unplanned share is the early sign, and the usual answer is to reduce what flows in, by fixing the source of the interrupts or moving work elsewhere.

The operating review

A plan needs a place where it meets reality on a regular rhythm. DeGrandis, following David Anderson’s Kanban method, calls it the operations review: a short, data-first meeting at which the people doing the work look at flow, commitments and risks across teams. Larson gives the underlying practice a name worth keeping, inspected trust: leaders trust teams to run their own work and inspect the results often enough to notice when the trust is misplaced. The evidence is practitioner experience, and the failure mode is well known. A long, ceremonial review owned by people who do not do the work turns into theatre, which is where Polaris’s PERT charts ended up.

Encore’s Verified Resale review meets every second Tuesday for 20 minutes around one board: the bet’s items by state, the dependencies with dates and owners, and the current forecast for the commitment. It asks three questions in order. What finished, and did it teach us anything that changes the plan? What is waiting, and on whom? Has the forecast moved, and what will we cut, move or ask for? The trio runs it, the tech leads with active dependencies attend, and a director comes once a month and speaks last. When the board shows nothing new, the review ends early.

Recap. Settle money first, then allocation, then roadmaps, so that planning stops being a disguised fight over headcount. A quarterly cycle needs only a few fixed points, and commitments are tracked apart from objectives. Dependencies are designed out where possible, contracted where stable and coordinated only where necessary; Encore removed nine of 23 by moving work. Big-room planning has thin evidence, unplanned work is planned for from measured history, and a short, regular review keeps the plan honest.

Exercise 10.2. List every dependency your team’s current quarter relies on. Mark each one “remove”, “contract” or “coordinate”, and for the first one you would remove, estimate what it would take: which work would move, to whom, and what that team would stop doing to absorb it.

10.3 Estimation and Forecasting

Why estimates disappoint

People are optimistic about their own work in a remarkably consistent way. Daniel Kahneman and Amos Tversky named the pattern the planning fallacy in 1979: forecasts of how long a task will take tend to be too close to the best case, even when the forecaster knows that similar tasks have run late before. Roger Buehler, Dale Griffin and Michael Ross tested it in 1994 with students writing their theses. Asked when they expected to finish, the students predicted 33.9 days on average; asked for a date by which they would finish if everything went as badly as it could, they predicted 48.6 days. They took 55.5 days, longer than their own worst case, and only about 30% finished by the date they had predicted.

Software adds two complications. The first is that estimates are often set by someone other than the person doing the work, and sometimes used as a lever. Tom DeMarco and Timothy Lister, in Peopleware, report a study by Michael Lawrence and Ross Jeffery of 103 Australian projects, studied in the early 1980s and scored on a weighted productivity measure. Projects whose estimates were set by the programmers did better than those whose estimates were set by supervisors, and the projects with no estimate did best of all. The data are old and correlational, and they are at least consistent with a view many practitioners share: an estimate used as pressure does not make people faster.

The second complication is the shape of the overruns. Bent Flyvbjerg and Alexander Budzier studied 1,471 IT projects and found an average cost overrun of 27%, which sounds manageable until we look at the tail. One project in six was what they called a black swan, with an average cost overrun of 200% and a schedule overrun of almost 70%. An average is a poor guide to a distribution like that, because the risk lives in the tail. Encore’s own history had the same shape. Its three previous cross-team launches, Partner Listings in year six, the festival app’s checkout integration in year five and rotating barcodes in year eight, were each more than half as late again as planned: 75%, 60% and 67% respectively.

Throughput instead of story points

Many teams estimate in story points, a relative measure of effort, and convert them to dates by dividing by velocity, the points finished per sprint. Within one stable team this can work tolerably. Across many teams it fails, because points are a private currency: one team’s five is another’s eight, the scale drifts, and once outsiders watch velocity the scale inflates. A cross-team plan built on the sum of points adds numbers in different units.

Daniel Vacanti’s Actionable Agile Metrics for Predictability (2015) proposes forecasting from what the team actually did: cycle time, the elapsed time from starting an item to finishing it, and throughput, the number of items finished per unit of time. Both come from the work tracker and need no conversion. A cycle-time scatterplot, with lines at the 50th and 85th percentiles, tells a team more about its predictability than any velocity chart, and it supports a discipline that replaces estimating each item: split work until each item is no larger than the team’s 85th-percentile cycle time, and then count items. The method needs right-sized items and a reasonably stable process; it does not need items of equal size.

The Siemens Health Services report is again the strongest evidence. In a pilot of 15 teams, Siemens moved from points and velocity to WIP, cycle time and throughput. Velocity had looked healthy while features failed to finish, because teams started stories to claim their points. Afterwards cycle time fell by 42%, the first release came in on schedule and more than 10% under budget, the second completed 33% more stories, and first-pass test yield rose from 75% to 95% over two releases.

Encore stopped converting points to dates. Teams may keep points for their own conversations, but no plan above the team uses them. Each team’s scorecard shows its cycle-time scatterplot and throughput, and the cumulative flow diagram, which shows how many items sit in each state over time, became the standard view at the operating review.

Monte Carlo forecasting

Throughput history answers a question estimates cannot: given how fast this team has actually finished work, how long will this backlog take, and with what probability? Monte Carlo forecasting simulates the future thousands of times by drawing weeks at random from the team’s real throughput, records how long each simulated future takes to finish the backlog, and reads dates off the resulting distribution. Monte Carlo simulation is long practice in engineering and finance; its use for software delivery is documented mainly by practitioners, Vacanti and Troy Magennis among them, and Siemens ran it beside its points-based forecasts to compare the two.

It works under three conditions. The history must come from a broadly similar system, the same team doing a similar mix of work. Backlog growth must be modelled, because items split and new work is discovered, and a forecast that assumes a fixed backlog is optimistic in exactly the way PERT was. And it must be re-run weekly, because its value lies in noticing drift early. Here is the Partner APIs team’s forecast as it ran at month 4:

import random
weekly = [7, 9, 4, 11, 8, 6, 10, 8, 5, 9, 12, 7]   # items finished, last 12 weeks
backlog, growth = 80, (1.3, 1.65)  # growth range from past launches (the outside view)

def weeks_to_finish():
    left, weeks = backlog * random.uniform(*growth), 0
    while left > 0:
        left -= random.choice(weekly)
        weeks += 1
    return weeks

runs = sorted(weeks_to_finish() for _ in range(10_000))
for p in (50, 85, 95):
    print(f"{p}%: {runs[len(runs) * p // 100 - 1]} weeks")

The growth range is the line that matters most. Without it, the simulation said 11 weeks at the 85th percentile, in comfortable agreement with the team’s points-based plan. With growth drawn from Encore’s past launches, it said 17 weeks, which from the start of June meant the last week of September. Figure 10.3 sets these forecasts side by side. Encore committed to the start of October knowing the margin was thin.

Figure 10.3 — Four forecasts of the same milestone, in weeks from the start of June. The first two agree with each other and with the team’s hopes; the third includes the work nobody has found yet, and the fourth, the week-8 re-run, crossed the commitment before the cut brought it back.

The weekly re-runs earned their keep in week 8. Early integration sessions with Rowmate’s engineers had turned up partner-specific retry behaviour for webhooks, the backlog had grown faster than its range, and the 85% date moved three weeks past October. Nobody had to announce a slip, because the forecast announced it at the operating review two months before the date. The Partner APIs team moved bulk listing import and two reporting endpoints out of the sandbox milestone and adopted the platform’s new shared webhook delivery in place of the retries it had been building. The next re-run put the 85% date back in late September, and the sandbox opened to Rowmate on 29 September.

The outside view

A forecast built from the details of the work in hand takes what Kahneman and Dan Lovallo, in 1993, called the inside view. The outside view begins instead from how long similar efforts have taken. Formalized as reference-class forecasting, it chooses a class of comparable past efforts, takes the distribution of their outcomes, places the new effort within it, and only then considers what makes this case different.

Kahneman told the story that made the idea memorable in Thinking, Fast and Slow (2011). In the 1970s he led a team writing a curriculum and textbook for Israeli high schools, and the team’s estimates of how long it would take clustered around two years. He then asked the curriculum expert on the team, Seymour Fox, how long comparable teams had taken. Fox recalled that about 40% had never finished, and that none of those that did had taken less than seven years or more than ten. The team carried on anyway. The book took eight years, and it was never used. Fox held the outside view all along, and the inside view prevailed even after he had stated it.

Flyvbjerg has done most to make the outside view an institutional practice. His database, described with Dan Gardner in How Big Things Get Done (2023), covers about 16,000 projects in more than 20 fields, of which 8.5% came in on budget and on time and 0.5% on budget, on time and with the benefits promised. Governments have turned it into rules. Since 2003, supplementary guidance to the UK Treasury’s Green Book has told appraisers of public projects to add optimism-bias uplifts to their estimates of cost and duration, percentages drawn from the record of comparable past projects.

Encore used its three previous cross-team launches as the reference class for Verified Resale. Three is a small class, and Encore knew it. But in all three the backlog had grown by between 30% and 65% after the plan was made, mostly through work discovered at boundaries between teams, and that finding went into the forecast as the growth range above. It also explained what Section 10.2’s triage was for: the lateness came from cross-team waits, and removing nine dependencies attacked that cause more directly than any adjustment to an estimate could.

Saying no with numbers

A forecast changes the conversation about a date. “Can you do it by June?” invites a yes, followed later by a slip. A forecast turns the question into one about scope, date and risk together: at the current scope there is an 85% chance of finishing by this date, and to reach the earlier one with the same confidence, here is what would have to go. The Products book’s Section 3.4 described appetite as a fixed budget against which a forecast is compared; when the forecast exceeds it, scope gives way, as it did for Polaris.

At month 4 the forecast for the whole launch, built from the throughput of all 12 teams and the dependency map, put the first resold ticket at month 14 at the 85th percentile, two months beyond the 12-month appetite. The Products book’s Section 10.3 tells which cuts the trio made, each tested by whether the bet could still be judged afterwards: Tessera moved to after launch, festival passes left the first release, and the organizer dashboard became a monthly statement. One further cut came from the engineering side. Organizers had asked for tour-wide rule templates, and the forecast showed that templates touched the rules engine, the organizer console and the reissue path at once; in the first version, organizers set rules per event. The cuts were agreed at month 4, and as the weekly re-runs absorbed them the 85th percentile came back to month 12 by month 6, where it stayed.

This works only if leadership accepts probabilistic dates, and the commonest failure is that an 85th-percentile date becomes a promise in the retelling, although one time in seven the work will be later. Encore’s guard is small and effective: every published forecast carries its percentile and the date of its last re-run, and nobody quotes a date without both.

Recap. The planning fallacy is consistent, estimates used as pressure do not help, and IT overruns have fat tails. Throughput and cycle time are measured rather than estimated, add up across teams, and support right-sizing in place of estimation. Monte Carlo forecasting turns throughput history into dates with probabilities, provided backlog growth is modelled and the forecast is re-run weekly. The outside view supplies that growth from comparable past efforts, and a forecast turns a demand for a date into a conversation about scope.

Exercise 10.3. Take your team’s last 20 finished items and compute weekly throughput. Run a Monte Carlo forecast for your current backlog, once with the backlog fixed and once with a growth range drawn from your last two comparable efforts. Compare the 85th percentile of each with the date already promised, and write down what you would cut to close the gap.

10.4 Funding Products, Not Projects

From project to product

Most organizations still fund software the way they fund buildings. A project is proposed with a scope, a budget and an end date, a team is assembled for it, and at the end date the team disperses and the software passes to someone else to run. Mik Kersten, in Project to Product (2018), argues that this fits software badly, because software is never finished and the people who understand it best are the ones who built it; project funding dissolves them just as their understanding becomes most valuable. The alternative is to fund long-lived teams by value stream, the sequence of activities that delivers value to a customer, and let the work flow to them.

The Products book’s Section 1.1 introduced this shift and Kersten’s four flow items, features, defects, risks and debt, and deferred the funding mechanics to this section. The evidence for the shift is mostly case-based, and the Flow Framework comes from Kersten’s own company. The stronger support is indirect: Accelerate’s findings on long-lived, cross-functional teams that own their services, and Arnold’s queueing argument that every project start creates a batch of work and every project end an orphan.

Encore made the change in its fourth year, at about 70 engineers, during the split into seven services the Architecture book’s Chapter 5 describes. The trigger was an orphan: the project that built the first venue embeds ended, its people returned to their teams, and the embeds ran ownerless for most of a year until a venue’s broken checkout exposed the gap. Since then every service has had one owning team, funded as a standing cost of its value stream, and new initiatives arrive as work for those teams.

Capacity allocation

Once funding follows teams, the question becomes what share of capacity goes to each kind of work. Capacity allocation is that decision: dividing engineering capacity among value streams and among kinds of work such as new product, maintenance, platform and debt. It is the second of Larson’s phases and the most consequential, because it decides what the roadmaps may contain. Kersten’s flow distribution, the share of finished work that was features, defects, risks and debt, shows whether the allocation was honoured. There is no empirically correct split; the percentages are management choices, stated with reasons and revisited when the reasons change. Figure 10.4 shows Encore’s for year nine.

Figure 10.4 — Encore’s capacity allocation for year nine. Only the first line is negotiated each quarter; the second is measured, the third is protected, and the platform line is top-sliced before allocation.

The structure matters more than the numbers. Keeping the lights on is measured, so nobody can win capacity by understating it. Debt and risk work has a floor per team that leadership does not raid for a deadline. Only new product work is negotiated, and that is where Verified Resale’s capacity came from: its planned work in 11 teams besides Resale was funded from their product share, which in four of them meant deferring roadmap features by a quarter.

The Inventory team tested the rule, because its debt work was the barcode core, which Verified Resale needed changed. A planner under pressure could have reclassified the debt floor as bet work and called it efficiency. Encore kept the characterization tests and the spreading of knowledge beyond Lena on the debt line, protected, because they were worth doing with or without the bet, and funded reissue itself from the bet. Debt work folded into a feature is usually cancelled the day the feature’s priorities change, and the separation prevented that.

Keeping the lights on and the platform tax

Two costs hide easily in a budget: keeping existing systems running, which unmeasured appears as a vague drag on every roadmap, and the platform, whose cost tends to be invisible or resented. Section 8.5 covers showback and unit economics, and the Architecture book’s Section 15.5 the business case for a platform; this subsection covers only how both enter the budget.

Encore makes both explicit lines. Keeping the lights on is the measured 20% in Figure 10.4, and a rising share is an early warning of accumulated debt or a fragile dependency. The platform group is funded centrally, without chargeback, for the reasons given there. What this subsection adds is how the central budget appears in the plan. It is top-sliced before allocation, the 10% line of Figure 10.4, and each value stream’s plan shows its share of that line as a platform tax: the portion of engineering capacity spent by the platform group on that stream’s behalf, shown in the stream’s plan though no team pays it from its own budget. The tax makes the paved road’s cost visible to the people who use it, and showback, which reports each team’s actual consumption, lets them see what they get for it. Neither becomes a bill a team must defend. The tax is reviewed yearly against the platform’s adoption and its measured effect on delivery.

Funding a bet in tranches

A bet like Verified Resale is too uncertain to fund in full at the start and too large to fund a sprint at a time. The answer, long practised in venture capital, is staged funding: money released in tranches, each tied to a learning milestone written in advance, with kill criteria someone has agreed to act on. Paul Gompers’s 1995 study of venture financing showed how investors use staged rounds to monitor a venture and limit what they lose when it falters. Inside companies the evidence is case-based: Eric Ries’s The Lean Startup (2011) supplied the vocabulary of innovation accounting, and Bjarte Bogsnes’s account of Statoil, now Equinor, replacing its annual budget with rolling forecasts and dynamic allocation shows the finance side at the scale of a large company.

Staged funding works when the milestones are learning milestones, statements about what will be known, written before the money is spent, with kill criteria concrete enough that someone who did not write them could check them. When they are progress milestones instead, such as “integration 60% complete”, every tranche is released, because progress can always be reported. The Products book’s Section 10.1 shows the memo that set Verified Resale’s first tranche and its kill criteria, K1 at month 4 and K2 at month 18. Its revision at month 4, after K1 had passed, set out the tranches in full:

VERIFIED RESALE: funding tranches (memo revision 1, month 4)
Tranche 1  Discovery, months 0–4. The trio; six weeks of two lent teams.
           Released at month 0. Gate: K1 at month 4 (passed).
Tranche 2  First partner live, months 4–12. The Resale team, and planned
           work in 11 other teams from their product share.
           Released on: K1 passed; cuts agreed that fit the forecast to appetite.
           Learning milestone, month 9: a reissued ticket's old barcode refused
           offline at a staging gate; Rowmate certifying in the sandbox.
Tranche 3  All partners, from launch. Part A: Tessera's certification, on the
           first cohort's evidence (no barcode admitted twice; resales counted
           at the door). Part B: further markets, after the month-18 review.
Kill       K1 and K2 as written at month 0. No tranche relaxes them.

The last line carries the discipline. A tranche can add conditions but cannot loosen a kill criterion written earlier, because the temptation to loosen them grows with every euro spent. Tranche 2 was released at month 4 on K1 and on the cuts in Section 10.3, and the re-run forecast confirmed within two months that the cuts had done their work. Part A of tranche 3 was released shortly after launch, when the first cohort had produced resales with no double admission at any gate, and funded Tessera’s certification. Part B waited for the month-18 review, where the Products book’s Section 10.5 records the decision to widen.

The finance conversation

Engineering leaders who want to fund products eventually meet the accountants, and it helps to know their terms. The central one is capitalization. Under IAS 38 Intangible Assets, which applies because Encore reports under International Financial Reporting Standards (IFRS), the costs of internally developed software are expensed in the research phase and may be capitalized in the development phase only when six conditions are all met: technical feasibility, the intention to complete, the ability to use or sell the result, probable future economic benefits, adequate resources to finish, and reliable measurement of the costs. Capitalized costs become an asset amortized over the software’s life, which changes reported profit considerably.

The standard was written with projects in mind, and that has long pulled finance departments toward project-shaped funding, since a project has a start, an end and one asset to attach costs to. The pull is weakening. The IASB is reviewing IAS 38, and in the United States the FASB’s Accounting Standards Update 2025-06, issued in September 2025, replaced the stage-based model for internal-use software with a principles-based threshold that suits incremental development better.

Encore’s controller and engineering leadership agreed a practice under the current standard. Capitalization is assessed per initiative, from the work tracker’s records of which items belong to which initiative, and the six criteria are assessed at the tranche gates, which give documented points at which feasibility and intention are established. Verified Resale’s discovery tranche was expensed in full, and its build was capitalized from the release of tranche 2. It is offered as an illustration. The judgments involved are accounting judgments, and each organization must make them with its own controller.

Recap. Projects dissolve the teams that understand the software at the moment their understanding is most valuable; funding long-lived teams by value stream keeps them. Capacity allocation is a choice, best structured so that maintenance is measured, debt work is protected and only new product work is negotiated. Platforms enter the budget as an explicit tax with showback. Bets are funded in tranches against learning milestones and kill criteria that later tranches cannot loosen, and the finance conversation goes better when engineering knows how IAS 38 treats development costs.

Exercise 10.4. Classify your team’s last quarter of finished work into features, defects, risks and debt, by count and, if you can, by effort. Compare the split with what your plan said it would be. If keeping the lights on took more than the plan allowed, find where the difference came from, and write the allocation you would propose for next quarter with a reason for each line.

10.5 Measuring Delivery

DORA for learning

The Architecture book’s Section 11.2 defined DORA’s delivery metrics, and Chapter 1 summarized them: change lead time, deployment frequency, change failure rate, failed deployment recovery time and, since 2024, rework rate, the share of deployments that were unplanned fixes for problems users had seen. DORA groups them into throughput and stability, and the Resilience book’s Section 4.2 showed why the two do not trade off. The question here is what an organization does with them.

Each team uses them to find its own constraint. A team with a long lead time and a high deployment frequency is waiting somewhere before its deploys, often in review. A team with a high rework rate is shipping problems it then has to chase. Only the team knows which remedy fits, so at Encore each team sees its own five metrics on its scorecard with a trend line and nothing to compare against but its own past. Leadership sees the distribution across teams and uses it to find systemic problems, such as a platform capability that slows everyone, and never to rank teams.

Encore’s median lead time was about 26 hours in year nine, with about 180 production deploys on an ordinary working day. Those numbers interest the board and tell leadership almost nothing about where to act. The distribution does. Most teams in the slowest fifth by lead time turned out to be teams whose changes passed through encore-core’s 31-minute build, and that is how a delivery metric became the argument for Chapter 4’s investment in CI.

Flow metrics for the value stream

DORA’s metrics begin when a change is committed, which for most features is late in the story; the weeks of shaping and waiting before the first commit are invisible to them. Kersten’s flow metrics measure the whole value stream, from acceptance of the work to its arrival with a customer: flow velocity, the items finished; flow time, the elapsed time from start to finish; flow load, the items in progress; flow efficiency, the share of flow time in which someone was actively working on an item; and flow distribution, the mix of features, defects, risks and debt. Chapter 1 described value-stream mapping as the way to see one stream once; flow metrics watch it continuously.

They are only as good as the states recorded in the work tracker, and flow efficiency is noisy, because few teams record waiting honestly. The Flow Framework also has less independent validation than DORA’s research. The published cases point the same way all the same: Maersk’s 38 weeks of waiting in 46, and Siemens’s flow efficiency, computed as touch time over cycle time, both show waiting dominating.

Encore measured its Verified Resale items in the first quarter of the build by adding explicit waiting states, “waiting on another team” and “waiting on review”, to the work tracker of the 12 teams. Flow efficiency came to about 30%: the items spent about 70% of their elapsed time waiting, most of it in the Seat Inventory review queue and the dependencies still marked “coordinate”. The number was imprecise, and Encore said so whenever it showed it, but its direction was not in doubt.

Flow time has a blind spot: it is known only when an item finishes, so the item stuck longest is the last to show up in it. Work item age, the time elapsed since an item started and not yet finished, sees it while it is still stuck. The Kanban Guide that Vacanti and John Coleman published in 2020 names it as one of four basic flow measures, with work in progress, throughput and cycle time. Encore applied it where its forecasts were weakest. It charts the age of every open cross-team dependency, counted from the day it was requested, against the ages at which past dependencies of the same kind closed, and an item past the 85th percentile of that history appears at the next operating review. The chart shows only that something is stuck. Somebody still has to be asked to act, and Section 11.4 tells what happened when, for three weeks, nobody was.

Goodhart’s law

In 1975 Charles Goodhart, an economist advising the Bank of England, wrote about the difficulty of controlling the money supply through statistical relationships the Bank had observed. His formulation was precise and narrow: “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.” Relationships that held while nobody steered by them broke down once policy targeted them, because the people being measured changed their behaviour. The popular version is the anthropologist Marilyn Strathern’s phrasing, from a 1997 essay on audit in British universities: “When a measure becomes a target, it ceases to be a good measure.” In this book, Goodhart’s law means the general principle in Strathern’s words, with Goodhart’s sentence as its origin: a measure that people are rewarded for moving stops telling us what it used to tell us. The social psychologist Donald Campbell reached a similar conclusion in the 1970s, arguing that the more a quantitative indicator is used for social decision-making, the more it is corrupted and the more it distorts the processes it was meant to monitor.

The best-documented modern case is Wells Fargo. The bank’s community banking business pressed its employees for years to sell more products to each customer, under targets that were hard to meet honestly. On 8 September 2016 two federal regulators and the Los Angeles city attorney announced penalties totalling US$185 million, over about 1.5 million deposit accounts and about 565,000 credit-card accounts that employees might have opened without authorization; about 5,300 employees had been dismissed over five years. In August 2017 the bank raised its estimate of potentially unauthorized accounts to about 3.5 million. The independent directors’ report of April 2017 named the sales culture and performance-management system, with a decentralized structure that let them go unchecked, as root causes, and in February 2020 the bank agreed to pay US$3 billion to settle investigations by the US Department of Justice and the Securities and Exchange Commission into conduct between 2002 and 2016. The cross-sell figure it had reported to investors as a sign of deep customer relationships had come to measure something else.

Software measures are corrupted the same way, less dramatically. In year eight Encore’s quarterly-objectives template began to suggest deployment frequency as a goal for teams wanting to improve delivery. The Embeds & Channels team adopted it in good faith and nearly tripled its deploys within a quarter, from about four a day to about 11, while its lead time and the size of the changes reaching venues stayed the same. It had begun deploying configuration, flag changes and code for one change separately, which counted three times and delivered the same thing once. Nobody had cheated; the team had done what the objective asked. Encore removed deployment frequency from team goals. It remains on each scorecard as a signal the team can read.

What not to measure

Some measures do harm from the moment they are collected. Lines of code reward verbosity, story points compared across teams reward inflation, and utilization, which Accelerate singled out as a poor measure alongside velocity, rewards the full calendars that Kingman’s result says lengthen every queue. Individual productivity is the most contested. In August 2023 McKinsey published “Yes, you can measure software developer productivity”, which built on DORA and SPACE and proposed adding individual-level and effort measures. Gergely Orosz and Kent Beck replied in The Pragmatic Engineer, in a first part published on 29 August 2023, urging engineering leaders “to look to outcome and impact measurements, to identify what to measure.” Both sides agree that delivery outcomes should be measured, the dispute is about individuals and effort, and on that question the history of Goodhart’s law favours caution.

Encore settled its own position in writing. The engineering leadership group keeps a short list of measures it will not collect, reviewed yearly with two team representatives present:

Encore engineering: measures we do not collect (revision 3, year nine)
- Lines of code, commits or pull requests per person
- Story points or velocity, compared across teams
- DORA or flow metrics computed per individual
- AI usage, or accepted AI suggestions, as a goal for anyone
- Hours online, keystrokes, meeting load as a proxy for effort
- Any ranking of teams on a continuous measure
A team may track anything it finds useful for itself. Nothing on this list
enters a performance review, a promotion case or a report above the team.

The list exists because measures accumulate by default. A dashboard that can compute something will eventually be asked to show it, by someone with a reasonable question, and a refusal agreed in advance saves relitigating each request under pressure.

Measuring for the teams and for the board

Larson, in the Primer’s chapter on measuring engineering organizations, distinguishes measures that help leaders steer from those stakeholders need, and argues for introducing them in sequence, starting with those that help the people doing the work. That practitioner judgment matches Encore’s scorecard rule: each team sees its own measures in full, leadership sees trends and distributions, and nobody sees a ranking. The board needs a short account of whether engineering is delivering what the company needs, at what cost and risk, with enough detail to know where to ask questions. Figure 10.5 shows Encore’s one page.

Figure 10.5 — Encore’s quarterly delivery page for the board. It reports distributions, bets and constraints, and names no team.

When the page was introduced, Encore’s directors asked for one ranking teams by delivery. The head of engineering declined and offered instead to bring the owner of any worrying number to the next meeting. Over the following year the directors asked for that twice, and both times the explanation was a constraint the team had already found.

The same discipline applies to AI assistance. The evidence, gathered in Section 5.1, is mixed and moving: DORA’s 2024 report associated rising AI adoption with slightly lower delivery throughput and markedly lower stability, and its 2025 report found adoption associated with higher throughput and still with lower stability. Encore keeps the same delivery and flow measures and substitutes nothing for them. Its rollout of coding assistants is judged on rework rate and cycle time in the early-adopting teams against their own history, never on usage counts.

Recap. DORA’s metrics help each team find its own constraint, and their distribution helps leadership find systemic ones; flow metrics extend the view to the whole value stream, imprecisely but usefully, and work item age shows a stuck item while it is still stuck. Goodhart’s law, in Strathern’s popular wording, explains why measures turned into targets stop measuring, and Wells Fargo shows the extreme. A written list of what not to collect saves an organization from its dashboards, and the board needs distributions, bets and constraints, never a ranking of teams.

Exercise 10.5. Write down every delivery metric your organization reports upward. For each, name one way a team could improve the number while making the work worse, and one decision the number has actually informed in the last year. Delete any metric for which you can find the first and not the second.

Kata: The Dashboard Deadline

Your brief: “Halyard.” Halyard sells pension-administration software as a service to 40 occupational pension funds in two EU countries, with 180 engineers in 18 teams. The pensions regulator in its larger market requires every administrator to connect to a new pensions dashboard, through which members see all their pensions in one place, and Halyard must pass the regulator’s conformance tests within nine months; any fund whose administrator misses the date is fined, and the contracts pass the fines to Halyard. Meanwhile the funds expect the roadmap promised at renewal: a contributions portal for employers and annual statements in members’ own languages. Planning runs on story points converted to dates, and the last three cross-team projects came in 40%, 85% and 70% late. The dashboard work touches seven teams, and the first plan shows 19 dependencies, six of them on the Member Records team, whose median review wait is 11 days. The Connectivity team finished 6, 3, 8, 5, 7, 4, 9, 6, 2, 7, 5 and 8 items in its last 12 weeks and has 150 items in its backlog. Finance funds the work as one fixed-budget project, and the board pack ranks teams by velocity.

Deliverables:

1. Cost-of-delay ranking — Estimate the weekly cost of delay of the dashboard connection, the contributions portal and the multilingual statements, with assumptions, ranges and urgency profiles, and rank them by cost of delay divided by duration.
2. Dependency map — Draw the 19 dependencies, mark each remove, contract or coordinate, and show how many you can remove by moving work. Say what you would do about the review queue on Member Records, using the queueing results of Section 10.1.
3. Monte Carlo forecast — Forecast the Connectivity team’s backlog from the throughput supplied, once with a fixed backlog and once with a growth range taken from the three past projects. Give the 50th and 85th percentiles of each and compare them with the regulator’s date.
4. Capacity allocation — Propose an allocation for the next two quarters across new product, keeping the lights on, debt and risk, and platform, saying which lines are measured, protected or negotiated, and what the funds will be told about their roadmap.
5. Funding and reporting — Replace the single project budget with tranches, each with a learning milestone and a kill or escalation criterion. Then design the board’s one-page delivery report, and write the list of measures Halyard will stop collecting.

Rubric:

Where you now stand

You can now price delay and sequence by it, see and shrink the queues between people, plan across many teams by settling money and allocation before roadmaps, and design out the dependencies that make cross-team plans fragile. You can forecast from what teams actually finish, with the probability stated and the undiscovered work modelled, and cut scope before a slip has to be announced. You can fund long-lived teams, protect the work that keeps systems healthy, release money to a bet as it learns, and measure delivery without inviting teams to game the measure. Chapter 11 follows a single change, the reissue of a ticket under a new barcode, from the first characterization test in Encore’s oldest code to the first resold ticket scanned at a gate, and shows these practices working together with everything else this book has covered.

References

• Donald G. Reinertsen — The Principles of Product Development Flow: Second Generation Lean Product Development. Celeritas Publishing, 2009.
• Dominica DeGrandis — Making Work Visible: Exposing Time Theft to Optimize Work & Flow. IT Revolution, 2017.
• Daniel S. Vacanti — Actionable Agile Metrics for Predictability. ActionableAgile Press, 2015.
• Daniel Vacanti, Bennet Vallet — Actionable Metrics at Siemens Health Services. Agile Alliance Experience Report, 2014 — free online.
• Joshua J. Arnold, Özlem Yüce — Black Swan Farming using Cost of Delay. Agile 2013 experience report, IEEE, 2013 — free online.
• Sylvain Lenfle, Christoph Loch — Lost Roots: How Project Management Came to Emphasize Control over Flexibility and Novelty. California Management Review 53(1), 2010 — free online.
• Harvey M. Sapolsky — The Polaris System Development: Bureaucratic and Programmatic Success in Government. Harvard University Press, 1972.
• D. G. Malcolm, J. H. Roseboom, C. E. Clark, W. Fazar — Application of a Technique for Research and Development Program Evaluation. Operations Research 7(5), 1959.
• Bent Flyvbjerg, Dan Gardner — How Big Things Get Done. Currency, 2023.
• Bent Flyvbjerg, Alexander Budzier — Why Your IT Project May Be Riskier Than You Think. Harvard Business Review 89(9), 2011 — free online.
• Mik Kersten — Project to Product. IT Revolution, 2018.
• Will Larson — The Engineering Executive’s Primer. O’Reilly, 2024. And An Elegant Puzzle. Stripe Press, 2019.
• Nicole Forsgren, Jez Humble, Gene Kim — Accelerate. IT Revolution, 2018.
• DORA — Accelerate State of DevOps Report 2024. Google Cloud, 2024. And State of AI-assisted Software Development. Google Cloud, 2025 — free online.
• Daniel Vacanti, John Coleman — The Kanban Guide. Kanban Guides, 2020 — free online.
• Marilyn Strathern — “Improving ratings”: audit in the British University system. European Review 5(3), 1997.