Ninety-two percent accuracy. A green arrow on the slide. And users quietly abandoning the product underneath it.

An insurer's claims-intake agent reported beautiful model metrics for two months while adjusters drifted back to reading files themselves — the errors hid in the complex claims, nobody could tell which fields were wrong, and "good" was never defined in the adjuster's language. The root cause wasn't measurement. It was definition, inherited from the vendor dashboard that shipped in the box.

This episode lays out the fix: four numbers to write before the first prompt. Task success defined in the user's units and verified in the system of record. The escalation band, which fails in both directions. Cost per successful outcome, not per call. And the harm line, the wrong action that must essentially never happen. Plus who signs the bar (the business owner, before the build), why the four numbers work as tripwires that police each other, the escalation band as the leading indicator, and the floor you write down while you're still clear-headed.

With a worked example (the four numbers for the bank payments agent from The Not Yet List) and the win: a retailer's returns agent whose operations director wrote the definition of good into the kickoff document, above the architecture diagram.

This is Track B, Episode 3 — the Product track: what to build, and why. New here? Start with Episode 1: Beyond the Demo.

Monday move: finish the sentence "this agent is good when…" in the user's units, write the four numbers and the floor, and get the business owner to sign it. If nobody will sign, you don't have a metrics problem. You have an ownership problem.

Episode 8: Orchestration — Layer two of the nine-layer architecture: orchestration. Who decides what happens next.

We open with a five-agent insurance claims crew that did everything right and still went sideways, then work through what the major labs actually agree on: start with one agent, bound the loop, let code own the known path, and spend the model's judgment only where the decision is genuinely uncertain.

Inside: the ReAct loop and its guardrails, the supervisor pattern, agents-as-tools versus handoffs, the four orchestration shapes, what should travel on a handoff, Anthropic's hard numbers on multi-agent cost versus value, and why delegation quality is the number one failure mode. Plus a second war story: the Tuesday a support triage chain fell over.

Monday move: mark every decision point in your most complex agent flow as model-decides or code-decides, then encode one deterministic decision as code.

A refund agent with a human approval on every action sounds safe.

Until reviewers are clicking approve half-asleep, and a $4,800 decimal slip sails through in under two seconds.

In this episode: the handoff layer. Where the agent meets the user, and who's actually in charge at the moment something irreversible happens.

• When should the agent ask vs. act? (One agent guessed wrong and ordered noodles instead of milk tea.)
• How to gate without alert fatigue: stop where money moves or something can't be undone. Never on read-only work.
• Why transparency is what protects user autonomy.

Plus a beat for PMs: the approval dialog, the escalation message, the activity log. That's product surface you can own and prototype yourself, starting now.

Track A, Episode 2 — the System track: how you build it.
New here? Start with Episode 1: Beyond the Demo, or catch the Two Tracks interlude for the full map.

Monday move: take your highest-stakes agent action and list every approval it triggers. Cross out each gate that doesn't map to something irreversible or expensive. For every gate that survives, rewrite the user-facing sentence until a reviewer can make a real decision from it.