The Context Degradation Curve

Watch any agent in production long enough and you will see the same arc. The first few turns are excellent. The agent reads the request, picks the right tool, respects the constraint you gave it, and produces a clean answer. By turn eight it starts re-running a query it already ran. By turn twelve it has forgotten that it must never touch the finance schema, it is quoting a number from a draft it discarded, and it is apologising for a mistake it did not make.

Nothing about the model changed between turn one and turn twelve. What changed is the window. Every tool call left its raw output behind. Every correction added another paragraph. Every retry added another copy of nearly the same thing. The instructions that made turn one excellent are still technically present, but they are now a small island in a large sea of the conversation's own exhaust.

Call this the context degradation curve: answer quality plotted against conversation length. Without deliberate management, the curve stays flat for a while and then falls off a cliff. With management, it stays flat. The difference is not the model you rent. It is the discipline you apply, which is the theme of Chapter 36 on compaction and the token budget.

The instinctive response is to buy more room. If the agent forgets at forty thousand tokens, surely a million-token window solves it. It does not, for the same reason a bigger desk does not make a messy worker organised. The Context Window Is Not Context made the general argument; degradation is the specific, measurable form of it.

Models do not read their window evenly. Research on long-context recall has repeatedly found that information at the start and end of the window is used more reliably than information buried in the middle. In a long conversation, your most important rules sit near the start, the newest noise sits at the end, and the facts the agent actually needs drift into the middle where recall is weakest.

A larger window also changes the bill. Every turn re-sends the history, so cost grows with the square of the conversation if nothing is ever removed. Chapter 18 calls this the hidden cost of agentic AI, and Cheaper Tokens Will Not Save Your AI Budget shows why lower prices per token do not rescue a design that keeps sending more of them. A bigger window lets you degrade more slowly and more expensively. It does not stop the degradation.

Degradation is not one failure. It is three, and each needs a different fix.

The first is stale truth. A query ran at turn three and returned a number. At turn six the agent ran a corrected query and got a different number. Both results are still in the window, with nothing marking which one replaced which. At turn ten the model picks the wrong one, confidently. This is the conversation-scale version of the problem Chapter 11 describes for organisations: context is a living layer, and anything that is not versioned eventually lies.

The second is fading rules. 'Never write to production tables' was clear at turn one, when it was a fifth of the window. By turn twelve it is a few dozen tokens among tens of thousands, competing with recent instructions that feel more urgent. Negative constraints fade first, because nothing in the recent conversation keeps reinforcing them. This is how a careful agent becomes a dangerous one without anyone changing a line of its prompt, and why The Blast Radius insists that hard limits live in code, not only in words.

The third is conversational drag. Long threads accumulate apologies, restatements, partial plans, and abandoned attempts. The model starts reasoning about its own history instead of the task: reconciling what it said earlier, defending past choices, repeating plans it already made. The window is full, but very little of it is about the work.

The second instinct, after a bigger window, is a summary. Every ten turns, ask the model to summarise the conversation so far and replace the history with the summary. This helps the bill and often hurts the answers.

Free-form summaries are lossy in exactly the wrong places. They keep the gist and drop the specifics: the exact table name, the customer identifier, the threshold the user set, the reason an approach was rejected. They also flatten rules into suggestions. 'The user prefers not to touch finance data' is a much weaker sentence than 'Do not query the finance schema,' and a summariser will happily write the first when it reads the second.

Worse, summaries compound. A summary of a summary of a summary drifts from the original in ways nobody reviews. By the third compaction, the agent is working from a story about the conversation, not from the conversation. Summarisation is a tool, but it is not a design.

The design that holds up in production treats the window as three separate areas with separate rules, the way a good analyst separates the policy binder, the working notes, and the scratch paper.

The pinned tier holds what must never change during the task: the role, the hard rules, the canonical business definitions, the permissions in force, and the output contract. It sits at the very start of every call, identical every time. That stability is also what makes it cheap, because an unchanging prefix stays cached, as The KV Cache Discipline explains. Definitions come from governed sources, as described in Chapter 8 on moving from data catalog to business memory.

The ledger tier holds the current state of the task as structured facts, not prose: the goal, the decisions made, the values confirmed, the open questions, the approaches already rejected and why. When a new fact replaces an old one, the old one is overwritten, not appended. This is the single most important change most teams can make, because it removes stale truth by construction.

The scratch tier holds raw tool output and intermediate reasoning, and it is cleared on purpose. When a query returns ten thousand rows, the agent extracts the three facts it needs into the ledger and the raw result leaves the window. Retrieval follows the same rule: filter and rerank before anything enters, as covered in Chapter 35. The scratch tier is where most tokens live, and it is the only tier you should ever expect to throw away.

With three tiers, compaction stops being a vague summary and becomes a set of specific moves. Promote: copy a confirmed fact from scratch into the ledger. Replace: overwrite a ledger entry when a newer fact supersedes it, and keep the reason. Evict: remove raw output once its facts are promoted. Re-pin: if a rule was restated or changed by an authorised human, update the pinned tier, never the scratch.

Each move can be logged. That matters for two reasons. First, when an answer is wrong, you can replay what the agent knew at that moment and see whether the fact was missing, stale, or ignored. Second, the log is an evaluation surface. The Evaluation Gap argues that agents pass demos and fail production because demos are short; a compaction log lets you test long conversations on purpose.

Set triggers rather than schedules. Compact when the scratch tier crosses a token threshold, when a tool returns more than a set size, or when the same tool is called twice with near-identical inputs. A fixed every-ten-turns rule compacts too early in quiet conversations and too late in busy ones.

The strongest defence against degradation is to avoid long windows altogether. Instead of one agent carrying a forty-turn thread, a lead agent holds the ledger and hands each sub-task to a worker with a fresh window: the pinned tier, the slice of the ledger that worker needs, and nothing else. The worker does its job in a few turns, returns a structured result, and its window is discarded.

This is how good teams of people already work. A manager does not forward a sixty-message email thread to a specialist; they write a short brief with the goal, the facts, and the constraints. Humans Never Work Without Context made the broader point. The brief is the context packet, and it is the same contract described in The Handoff Problem: goal, evidence, constraints, state, validated at the seam.

Fresh windows also cap cost and contain mistakes. A worker that goes wrong goes wrong inside a short, disposable thread, and the lead agent can retry it with a corrected packet instead of untangling a long history. The lead agent's own window stays small because it holds a ledger, not a transcript.

You cannot manage a curve you have never drawn. Take twenty real tasks that run long in production. Replay each one, and at turns one, five, ten, and fifteen insert a probe question whose correct answer depends on a rule or fact from early in the conversation. Score the answers. That gives you your own degradation curve, for your agents, on your data.

Track three numbers alongside it. Rule retention: the share of probes where the agent still honours a hard constraint. Fact freshness: the share where it uses the latest value rather than a superseded one. Window composition: the percentage of tokens in each tier at each turn. If scratch is more than half the window by turn ten, you have found your problem before your users do.

Make the curve part of release checks. When you change a prompt, a model, or a tool, re-run the long-conversation set, not only the short one. Chapter 17 is a reminder that a named human should review the result before the change ships, and Chapter 10 explains why the facts worth keeping beyond a single conversation belong in institutional memory, not in an ever-growing chat log.

Context degradation is a context engineering problem in its purest form. It is not fixed by a better model or a longer window. It is fixed by deciding what the model sees, in what order, for how long, and with what proof. That is the job Chapter 9 describes, and it is a large part of what the Databricks Context Engineer Associate practice exam tests under token budgets and compaction.

Start small. Pick the one agent whose long conversations generate the most complaints. Draw its curve. Split its window into pinned, ledger, and scratch. Measure again. Most teams find that the cliff moves from turn twelve to beyond anything their users actually reach, and that the bill goes down at the same time.