The Handoff Problem: Why Multi-Agent Systems Fail at the Seams

Here is a scene that is now playing out in hundreds of enterprises. A team builds a multi-agent workflow to answer a revenue question. A planner agent decomposes the request. A retrieval agent pulls the relevant tables and documents. A tool agent runs a query and calls a forecasting service. A writer agent turns the result into a paragraph a vice president can read. The team tests each agent in isolation. Every one of them passes. Then they run the whole thing end to end, and the answer is confidently, precisely wrong.

The instinct is to blame the model. The second instinct is to blame the data. Both are usually wrong. When a multi-agent system produces a bad answer while every component behaves correctly, the failure almost always happened at a seam — the moment one agent handed work to the next and something that mattered did not survive the trip.

This is the handoff problem, and it is the defining architectural challenge of agentic systems right now. It is not exotic. It is the same class of failure that has haunted distributed systems, microservices, and human organizations for decades: the interface is where the truth gets lost. What is new is that in an agentic system the interface is not a schema. It is natural language, compressed, reordered, and rewritten at every step by a probabilistic system that has no obligation to preserve what it was given.

The rest of this essay is about naming that failure precisely, measuring it, and closing it. If you build or govern agentic systems, this is the highest-leverage thing you can work on this quarter, and it costs nothing but discipline.

In a traditional pipeline, a handoff is a contract. Stage one writes a table with a known schema. Stage two reads it. If the schema changes, something breaks loudly and immediately. The interface is typed, versioned, and testable, and the whole discipline of data engineering exists to keep it honest.

In an agentic pipeline, the handoff is a paragraph. The planner writes a natural-language instruction for the retriever. The retriever writes a natural-language summary of what it found for the tool agent. The tool agent writes a natural-language description of what happened for the writer. At every step the representation is lossy, and the loss is not random. It is systematically biased toward whatever the intermediate model found salient, which is not always what the downstream agent needs.

Four things fall out at the seams, and they fall out in this order of frequency. The first is constraints. The user asked for last quarter, excluding intercompany transactions, in reported currency. The planner understood all three. By the time the request reaches the tool agent, the currency constraint has quietly evaporated, because it was not repeated in the summary. The second is provenance. The retriever found the number in a certified table, but the summary it passed forward said only that revenue was forty-one million. The writer has no idea whether that number came from a governed source or a stale spreadsheet, so it cannot cite, and the reader cannot verify.

The third is uncertainty. The retriever was not confident. Two documents disagreed. It picked one. That hedge is the single most useful signal in the entire chain, and it is almost always the first thing a summarizer discards, because uncertainty reads as noise. The fourth is negative information — the things that were checked and ruled out. Downstream agents re-explore paths that upstream agents already eliminated, which is both expensive and a common source of contradictory answers.

Notice that none of these are model failures. Each agent did something reasonable with what it received. The system failed because nobody defined what a handoff must carry. This is exactly the argument Chapter 3 makes about smart models needing smart systems, and it is why Chapter 34 treats multi-agent design as an engineering discipline rather than a prompting exercise.

There is a number every team building multi-agent systems should compute and almost none of them do. Call it per-hop fidelity: the probability that a handoff preserves everything the next agent needs. Then raise it to the power of the number of hops.

Suppose each handoff preserves the necessary context ninety-five percent of the time. That feels excellent. Nobody would complain about a component with a ninety-five percent success rate. Now chain four agents together. You have three handoffs, and the system-level fidelity is roughly eighty-six percent. Chain six agents and you are at seventy-seven percent. Chain ten, which is not unusual in a research or coding agent, and you are below seventy percent — meaning nearly a third of runs lose something that mattered, even though every individual component is performing at ninety-five percent.

This arithmetic explains the single most common experience teams report in 2026: the demo works and the product does not. Demos are short. They have two hops. Products are long. They have nine. The compounding is invisible until you scale the chain, and by then the architecture is committed.

It also explains why adding more specialized agents often makes systems worse rather than better. Every new agent you insert adds a seam. If the marginal agent improves its local task by five percent but introduces a handoff with ninety-five percent fidelity, you have traded a certain small gain for a certain small loss and added latency and cost on top. Chapter 18 covers the hidden cost of agentic AI, and orchestration overhead is the line item that surprises finance most. Chapter 21 frames the same tradeoff as a quality, speed, and cost decision rather than an architectural preference.

The practical rule that falls out of this: do not add an agent unless you can state what it does that the previous agent could not, and unless you have defined the contract on both of its seams. Fewer agents with defended handoffs beat more agents with implicit ones, every time.

The fix is not clever prompting. It is a contract — an explicit, structured object that every handoff must carry, regardless of what the agent chooses to say in prose. Think of it as the schema that agentic systems have been missing.

A workable context contract has four required fields. The goal is the original user intent, carried forward verbatim and never paraphrased. Paraphrasing intent is how systems drift; by hop four the question has become a different question. The evidence is the set of facts retrieved so far, each with a source identifier and a confidence marker. Not a summary of the facts. The facts, with pointers, so the writer can cite and the auditor can trace.

The constraints are every filter, exclusion, time bound, currency, entitlement, and policy the request is subject to. Constraints are append-only across the chain. An agent may add a constraint. No agent may silently drop one. If an agent believes a constraint is unsatisfiable, it must say so explicitly rather than quietly relaxing it — this single rule eliminates a large share of confidently wrong answers.

The state is what has already been tried, what was ruled out, and what remains open. This is the field teams skip, and it is the one that saves the most money, because it stops downstream agents from re-running work that upstream agents already completed or already eliminated.

The contract lives alongside the prose, not inside it. The agent may write whatever natural-language message it likes for the next model to read, but the structured object travels with it and is validated at every seam. If a required field is missing or malformed, the handoff fails loudly — the way a schema violation fails in a data pipeline — instead of degrading silently. This is the essence of the governance argument in Chapter 13 and the action-control model in Chapter 14: the boundary is where control has to be enforced, because the boundary is where authority changes hands.

You cannot fix what you cannot see, and most agentic observability today shows the wrong thing. Traces record which agent ran, how long it took, and how many tokens it consumed. Those are process metrics. They tell you the system executed. They do not tell you whether the meaning survived.

What you want instead is a handoff ledger: for every seam, a record of what was carried and what was dropped. The mechanics are simple. Before the handoff, snapshot the contract. After the handoff, snapshot it again. Diff the two. Any constraint present before and absent after is a dropped constraint. Any evidence item that lost its source identifier is a broken citation. Any confidence marker that got flattened into an assertion is a lost hedge. Log all three.

Once the ledger exists, something useful happens. Failures stop being mysterious and become attributable. Instead of a support ticket saying the agent gave a wrong number, you get a row saying the currency constraint was dropped at hop three, between the planner and the tool agent. That is a bug with an owner and a fix, not a vague complaint about model quality.

The ledger also changes your evaluation strategy. Most teams evaluate agentic systems end to end: did the final answer match the expected answer. End-to-end evaluation is necessary and insufficient, because it tells you the system failed without telling you where. Seam-level evaluation gives you per-hop fidelity scores, and per-hop scores tell you exactly which handoff to harden first. This is the correctness discipline described in Chapter 30, applied at the interface rather than at the output, and it is why Chapter 15 treats audit trails as an engineering feature rather than a compliance chore.

One more benefit worth naming. A handoff ledger is the artifact that makes an agentic system explainable to a regulator, an auditor, or a nervous executive. It answers the question they actually ask, which is not how the model works but what the system knew and when it knew it.

Beyond the contract and the ledger, four architectural patterns consistently reduce handoff loss in production systems. None of them require a new platform.

The first is the shared context store. Instead of passing context along the chain, agents read from and write to a common store scoped to the task. The chain becomes a set of participants around a shared document rather than a relay race. This eliminates transitive loss entirely, because hop three reads the same original constraints hop one wrote. The tradeoff is attention: a shared store grows, and a growing store must be compacted, which is its own discipline — the subject of Chapter 36.

The second is verbatim intent. The original user request travels unmodified through every hop, always. Agents may add interpretation, but the raw text is never replaced by its own summary. This is a one-line change in most orchestration frameworks and it removes an entire category of drift.

The third is the constraint check at the boundary. Before an agent acts, it validates its planned action against the accumulated constraint list. Not the model checking itself in prose — a deterministic check in code, outside the model. If the action violates a constraint, it does not execute. This is the difference between asking a model to be careful and building a system that cannot be careless, which is the argument running through Chapter 16.

The fourth is the escalation seam. At least one handoff in any consequential workflow should be capable of stopping and asking a human. Not every handoff — that defeats the point of automation. But the seam immediately before an irreversible action should be able to pause on low confidence or on a dropped constraint. Chapter 17 makes the case that human-in-the-loop is not a transitional crutch but a permanent architectural component, placed deliberately rather than everywhere.

There is a strategic implication hiding inside this engineering problem, and it is the reason the handoff belongs in a book about Context, Control, Cost, and Choice rather than in a framework tutorial.

Handoff quality is not something you can buy. Every vendor will sell you agents. No vendor can sell you the contract that describes what your organization's agents must carry, because that contract encodes your definitions, your entitlements, your policies, and your notion of sufficient evidence. It is written from your semantic layer, which is why Chapter 7 and Chapter 8 matter more than any orchestration library you choose.

That also makes it portable in the way that matters. A context contract defined in open, structured terms survives a change of model, a change of framework, and a change of platform. An implicit handoff encoded in a vendor's prompt templates does not. This is the practical version of the portability argument in Chapter 12 and the lock-in warning in Chapter 22: the asset you are building is the interface definition, and you should own it in a form you can carry out the door.

And it compounds. Every dropped constraint you find and fix is a permanent improvement to a system that will run thousands of times. Every seam you instrument becomes a place where the organization learns something it did not know about its own definitions. Teams that do this work end up with something more valuable than a working agent: they end up with a written, tested account of what their business actually means by its own words. That is the institutional memory argument in Chapter 10, arrived at from the engineering side.

Pick the multi-agent workflow in your environment that produces the most complaints. Do not redesign it. Draw it as a line of boxes with the seams marked, and count the hops. Most teams are surprised — the workflow they thought had three agents has seven, once you count the summarizers and the routers.

Then take one real request that produced a bad answer and walk it backward, seam by seam, asking a single question at each one: what did the next agent receive, and what did the previous agent know that it did not? You will find the drop. In our experience it is usually at hop two or three, and it is usually a constraint or a confidence marker.

Fix that one seam with a contract. Four fields: goal, evidence, constraints, state. Validate it in code. Log the diff. Re-run the failing request and ten more like it. You will have measurably improved a production system in a day, and you will have built the first row of a ledger that will keep paying you back for as long as the system runs.

The agents are not the product. The seams are where the product is either trustworthy or not. Build the interfaces, and the intelligence has somewhere reliable to stand.