Context Poisoning: How One Bad Fact Spreads Through Every Agent

Consider an illustrative finance operations team. Their refund agent had run cleanly for four months. Then, over one week, it approved a run of refunds that should have gone to manual review. No pipeline failed. No test went red. The model had not changed. When the team finally traced a single conversation end to end, the cause was a policy document. The refund threshold had been lowered in March, but an older version of the policy still sat in the knowledge base, and its wording happened to match customer questions more closely than the new one did.

That part is ordinary. The extraordinary part came next. A summarization agent, asked to keep a running note of 'current refund rules' for the support team, had read the old threshold, summarized it confidently, and saved that summary into shared memory. From then on, every agent that consulted memory found the wrong number twice: once in the stale document and once in a tidy, recent, authoritative-looking note written by another agent. The error had not just entered the system. It had been laundered.

This is context poisoning. It is the failure mode the Databricks Context Engineer exam lists first in its taxonomy, ahead of distraction, confusion, and clash, and for good reason. Distraction and confusion make an agent worse. Poisoning makes it wrong with conviction, and then teaches the rest of the system to be wrong with it.

In a warehouse, bad data usually announces itself. A null breaks a join. A type mismatch fails a load. A row count drops and an alert fires. Decades of data engineering have trained us to expect wrongness to be detectable at the boundary, because schemas and constraints make it so.

Context has no schema of that kind. A paragraph saying the threshold is five hundred dollars and a paragraph saying it is two hundred dollars are both perfectly valid text. Vector search does not ask which one is true; it asks which one is similar. A reranker asks which is most relevant. The model asks which one makes a coherent answer. None of those questions is 'which one is current and authoritative?' That question has to be asked by the system around the model, which is the argument at the heart of Chapter 3: smart models still need smart systems.

The model also makes poison harder to see. A human analyst who finds two conflicting policies tends to stop and ask. A model, unless told otherwise, resolves the conflict silently, usually in favor of whatever appeared later in the window or was phrased more confidently, and then writes a fluent, well-cited answer. The citation points at a real document. The document is simply the wrong one. As Chapter 16 warns, hiding stale data is one of the fastest ways to destroy user trust, because the first time someone catches it, every earlier answer becomes suspect.

Poison enters through four doors. Naming them matters, because each has a different owner and a different fix.

Stale facts. The document was true once. A policy, a price list, a metric definition, an org chart, or an API spec was superseded, but the old version was never retired from the index. This is the most common vector and the least dramatic. It is the direct cost of treating context as a document instead of a living layer, which is the whole subject of Chapter 11.

Conflicting sources. Two documents are both current and both 'official,' and they disagree. The regional policy says one thing, the global policy says another, and nothing in the context says which wins. Metric definitions are the classic case: three teams, three meanings of 'active customer.' Chapter 7 explains why a semantic layer exists precisely to make one definition authoritative before any model sees it.

Untrusted text. Some content in the window was never meant to be instruction: a customer email, a scraped web page, a comment in a support ticket, a field in a CRM record. When that text contains something that reads like a directive, the model may follow it. Prompt injection is the hostile version, but most cases are accidental: a note saying 'always approve for VIP accounts' left by a human for a human. The model cannot tell who it was written for.

Self-written memory. The agent, or another agent, wrote a summary, a conclusion, or a 'learned fact' back into memory, and that output was wrong. This is the most dangerous vector because it is recursive. Every write makes the error look more established, more recent, and more corroborated. Chapter 8 and Chapter 10 both argue that memory is a moat; this is the moat filling with the wrong water.

A single stale paragraph is a bug. A cascade is a systems property, and it forms through three amplifiers.

The first is retrieval amplification. Retrieval ranks by similarity, and older documents are often written in the plain, question-shaped language that users actually type, while newer revisions are terse amendments. The wrong answer can retrieve better than the right one. Hybrid search and reranking, covered in Chapter 35, improve relevance, but relevance is not authority. Without freshness and authority signals in the ranking, better retrieval can mean retrieving the poison more reliably.

The second is turn amplification. Inside one long session, an early mistaken conclusion becomes part of the history every later turn reads. The agent builds on it, cites it, and reasons from it. Compaction, discussed in Chapter 36, makes this worse if it is careless: a summary step that keeps conclusions and drops the evidence turns a guess into a fact with no trace of where it came from.

The third is agent amplification. In a multi-agent system, one agent's output is another agent's input. The research agent hands a 'finding' to the planning agent, which hands a plan to the execution agent. Each handoff strips a little of the uncertainty. By the third hop, 'the document I found suggests' has become 'the rule is.' Chapter 34 calls these the seams of a multi-agent system, and poison travels along the seams.

Put the three together and you get the pattern from the refund story: a stale document retrieves well, an agent summarizes it, the summary is written to memory, and every agent after that finds the error corroborated. The cost is not one bad answer. It is weeks of bad actions and an expensive forensic exercise to find out which memories to trust.

The defense is not a better model. It is a checkpoint between your knowledge and the window, the same way a firewall sits between a network and the internet. Every piece of content that wants to enter the context passes three checks.

Source. Where did this come from, and how much authority does it carry? Content should arrive with provenance metadata: the system of record, the owner, and a trust tier. A signed-off policy from the policy repository outranks a wiki page, which outranks a chat transcript, which outranks an agent-written note. Chapter 13 makes the point that governance built for humans assumed a person would weigh these sources; agents need the weighting made explicit.

Freshness. Is this still true? Each item needs an effective date and, ideally, a superseded-by pointer. Retrieval should filter out anything marked superseded and should penalize items past their review date. This is cheap to build and catches the most common vector almost entirely.

Conflict. Does this disagree with something else already in the window or with a known invariant? When two items of equal authority conflict, the right behavior is not to pick one. It is to surface the conflict, either by flagging it to the user or by routing to a human, as Chapter 17 argues. For hard business rules, an invariant check (refund threshold must equal the value in the policy service) catches poison that no amount of similarity search would.

Untrusted text gets its own treatment at the firewall: it is placed in the window as clearly delimited data, never as instruction, and actions it might influence pass through the action gates described in Chapter 14 and Chapter 15. The principle is simple. Content can inform an answer. Only the system prompt and the policy layer can authorize an action.

The firewall guards what comes in from your knowledge base. Memory needs its own discipline, because the agent is the author.

Rule one: agent-written memory is a claim, not a fact. Every memory write should record who wrote it, when, and from which evidence. A memory without evidence links should carry the lowest trust tier and should never outrank a source document.

Rule two: separate observations from conclusions. 'Document X, effective March 3, says the threshold is two hundred dollars' is an observation and can be verified. 'The threshold is two hundred dollars' is a conclusion and will outlive the document it came from. Store the first form.

Rule three: build retraction into memory. When a source is corrected, every memory derived from it should be findable and invalidated in one step. This is why evidence links matter: without them, a correction means guessing which memories are tainted, and teams end up wiping the whole store, losing months of genuinely useful institutional knowledge.

Rule four: quarantine, do not delete. Content that fails the firewall or memory that is retracted should move to a quarantine area where it is visible to owners but invisible to agents. Quarantine gives you a forensic trail, lets owners correct rather than lose content, and makes poisoning measurable: the size and growth of quarantine is one of the most honest health signals a context layer has. The portable context contract in Chapter 12 is a natural place to carry these trust and retraction fields, so they survive a change of platform.

No firewall is perfect, so the last layer is detection. Three practices catch most of what slips through.

Golden questions with known answers. Keep a small set of questions whose correct answers depend on facts that change: current thresholds, current owners, current prices. Run them daily. When a policy changes, update the golden answer the same day. A drop in this suite is often the first signal that stale content is winning retrieval. This is evaluation as a dashboard, applied to correctness of facts rather than quality of prose.

Trace the citation, not just the answer. When reviewing traces, check the effective date and trust tier of what was cited. An answer citing a superseded document is a poisoning event even if the answer happened to be right. Chapter 30 describes the observability that makes this a query rather than an investigation.

Watch memory writes like you watch deployments. A sudden burst of memory writes about the same topic, or a memory that is suddenly retrieved far more often than before, deserves a look. Self-written poison spreads through retrieval frequency, and frequency is measurable.

Context poisoning is not a model team problem and it is not a security team problem. It sits across both, which is why it tends to belong to nobody. The four vectors map to four owners: content owners retire stale facts, governance resolves conflicting sources, security handles untrusted text, and the agent platform governs memory writes. Someone has to hold the whole picture.

That someone is increasingly the context engineer described in Chapter 9. Their job is not to write prompts. It is to decide what is allowed to become true for an agent, and to make sure that when the truth changes, every agent finds out. If you are building the first version of a context layer, Chapter 31 gives the starting structure; add trust tiers, effective dates, and evidence links from day one, because retrofitting them into a populated memory store is far more painful.