Context Building Is the Foundation — RAG, Citations, and Grounding Before Tool Use

Every current AI conversation starts one step too late. It starts with tools, agents, orchestration, protocols, model-vs-model benchmarks — the exciting layer at the top of the stack. It rarely starts where the actual wins are: with the quiet, unfashionable, engineering-heavy work of building the context an AI system consumes before it does anything else.

This is not a stylistic complaint. It is the single most consequential misordering in enterprise AI right now. A team that has skipped the context layer and jumped straight to agents will always be debugging behavior it cannot explain, correcting hallucinations it cannot trace, and losing user trust it cannot rebuild. A team that has built the context layer properly will find that a surprising share of its 'agent problems' were context problems in disguise, and that many of them dissolve when the foundation is right.

The whole of Part 2 of The Context Advantage — Chapters 4 through 7 — is an argument for putting the context layer first. This essay is the field-guide compression of that argument, with the specific technical patterns that separate the systems users trust from the systems they quietly stop using.

Context building is the discipline of assembling, at inference time, the exact set of facts, definitions, examples, and constraints a model needs to produce a trustworthy answer for a specific request. That is a much narrower job than 'give the model everything'. It is a much wider job than 'do RAG'.

A useful working definition has four moving parts. Retrieval — finding candidate material relevant to the request. Grounding — deciding which of that material the model is allowed to rely on, and how strongly. Citation — surfacing the specific sources that justify each claim in the output. Governance — enforcing what may or may not be retrieved, exposed, or acted on for a given user in a given context.

The systems that feel magical to users are the ones where all four are strong. The systems that feel unreliable are almost always weak on two of the four, and the failure mode is confident, articulate, and wrong. That is not a model problem. It is a context-layer problem, and Chapter 4 argues — hard — that this is where most enterprise AI budgets should be spent for the next three years.

There is a specific reason context has to come before tool use, and it is not aesthetic. It is causal. An agent that reaches for a tool is making a decision based on what it currently believes to be true. If those beliefs are shallow, wrong, or missing critical constraints, the tool call amplifies the error instead of correcting it. Every tool an unclear agent touches gets touched wrongly.

Consider the pattern. An assistant is asked to update a customer record. Without proper context, it does not know which customer the user means, which version of the record is current, whether the requested change conflicts with a policy, or whether the user has authority to make it. It picks a tool call, executes it, and ships an error into a system of record — where errors are expensive to reverse and easy to compound. The tool did its job. The context layer failed. The user blames the AI.

Now run the same request against a system with a real context layer. Retrieval brings back the current record, the last three related events, the policy that applies, and the user's authorization scope. Grounding constrains the model to reason only over that material. Citation forces the reasoning to point at specific fields. The tool call is either correct or refused with an explanation. Same model. Same tool. Radically different outcome. This is the argument of Chapter 5 — the context layer is not a preprocessing step. It is the safety layer that makes tool use responsible.

Most production RAG systems are the tutorial version: chunk documents, embed, vector search, top-k into prompt, generate. That pipeline is fine for a demo and dangerous for a business. It ignores almost every dimension that decides whether retrieval is trustworthy at scale.

Serious RAG has at least six layers where the tutorial has one. First, source curation — which corpora are eligible for a given user in a given task, with a permission model attached. Second, chunking that respects the semantics of the document — sections, tables, code blocks, headings — not a naive character count that severs a sentence from its qualifier. Third, hybrid retrieval — lexical plus semantic — because vector search alone misses exact-match terms that users care about (product SKUs, error codes, legal clause numbers) and lexical alone misses paraphrases.

Fourth, reranking — a second-stage model that reorders the candidate set with better judgment than the first-pass similarity score. Reranking is one of the highest-leverage additions to any RAG stack and is chronically skipped. Fifth, freshness and recency policies — because most enterprise questions are implicitly time-scoped, and returning the correct answer from six quarters ago is often the wrong answer. Sixth, negative retrieval — knowing when to return nothing and force an honest 'I do not have the source to answer this' rather than manufacturing an answer from adjacent context.

That is not glamorous engineering. It is exactly the engineering that separates a RAG system users check twice from a RAG system users stop checking. Chapter 6 of the book is the long-form treatment, with the specific patterns for each of the six layers.

Retrieval decides what the model can see. Grounding decides what the model is allowed to rely on. They are not the same, and most stacks confuse them.

A model given a set of retrieved passages will, by default, treat those passages as suggestions rather than boundaries. It will happily blend retrieved content with parametric knowledge from its training data, and it will do so invisibly. When the retrieved material is correct and the parametric material is correct, this looks like intelligence. When they conflict, it looks like hallucination — but the more accurate word is drift, and it is systematic.

Strong grounding pushes back on drift with three specific techniques. Structural prompts that separate 'here is what you may use' from 'here is the question' and instruct the model, in unambiguous language, to answer only from the provided material or to say it cannot. Attribution constraints that require the model to tag each claim with the specific chunk it relied on, so a claim without an anchor is caught mechanically. Verifier passes that re-read the output against the retrieved material and flag any sentence that is not clearly supported. A grounded system does not eliminate the model's fluency. It disciplines it. The best-written answer that cannot be attributed to a source is worth less, in an enterprise, than a rougher answer that can.

Citations are the interface layer of grounding, and they are the single most under-valued design decision in enterprise AI. A system that shows its sources is not merely nicer to use. It is a fundamentally different product, because it converts every user into a partial verifier and every verification into training signal for the platform.

Good citation design has a few durable rules. Citations should point at the smallest useful unit — a paragraph, a table row, a chart label — not at 'the document'. Every load-bearing claim should have at least one citation, and the interface should make an uncited claim visually distinct so the reader notices. Citations should link to the source in its native context, not to a copy, so the reader can see what surrounds it. And citations should carry a freshness stamp, because 'as of' is a fact about the answer, not a footnote to it.

Underneath the interface, citations are also the honest audit trail your compliance team will eventually ask for. A system that generated an answer without recording which sources it consulted has no defensible story about how the answer came to be. A system that recorded the sources — and can replay them — is defensible, improvable, and, crucially, correctable. Chapter 7 is our long-form case that citation is not a UX flourish. It is a governance primitive.

The context layer is the natural home for the hardest question in enterprise AI: what is any given user, in any given task, allowed to see, use, and act on? That question does not belong to the model. It does not belong to the agent. It belongs, cleanly, to the retrieval and grounding layer, where it can be enforced once and inherited by everything above.

In practice this means the retriever is a policy engine as much as it is a search engine. It knows the user, the tenancy, the sensitivity of each source, the role of the user in the workflow, and the purpose of the task, and it filters the candidate set accordingly. A well-governed context layer never lets the model see material it should not use, which is a much stronger guarantee than telling the model to be careful with it.

This is the argument that ties Part 2 of the book to Part 3 (Control). Governance in the context layer is what makes control in the agent layer meaningful. Skip the first and the second becomes theatre — you cannot control the actions of a system that is reasoning on material it was never supposed to have.

There is a short diagnostic every team can run this week. Take your top five production AI answers over the last seven days. For each one, can you produce, from logs alone, the exact retrieved chunks the model relied on, the sources those chunks came from, the freshness stamps, the policy decisions that allowed retrieval, and the specific attribution between each output claim and its source? If the answer is yes for all five, you have a real context layer. If the answer is no, you do not — you have a demo that is aging in production.

Second diagnostic. Ask the system a question you know is out of scope of its authorized sources. Does it say so, clearly, and refuse? Or does it produce a plausible answer from parametric knowledge? A grounded system refuses cleanly. An ungrounded one improvises. Improvisation is charming in a chatbot and disqualifying in a business system.

Third diagnostic. Change a source — update a policy document, correct a customer record. Does the next relevant answer reflect the change, and does the citation link to the new version? If yes, your freshness pipeline is real. If not, you are shipping stale ground truth with confident language on top, and users will find out before you do.

For teams that have skipped this layer and want to catch up without pausing the roadmap, the practical sequence is boring and effective.

Start with source curation and permissions. Decide which corpora are eligible, for which users, for which tasks. Write it down. Enforce it in the retriever. This is unglamorous and pays back forever.

Add hybrid retrieval and reranking to your single most-used AI feature. Measure the difference. Do not proceed to the second feature until the first one is meaningfully better.

Turn on citation in the UI, even in a rough form. The user-facing behavior change is immediate, and the internal telemetry it unlocks — which citations users click, which they ignore, which they dispute — is worth more than any offline evaluation you will run this quarter.

Add a verifier pass on the output. Even a lightweight one that flags unsupported sentences will change how the team thinks about quality within a week.

Finally, wire the context layer into governance early. It is much cheaper to build the policy engine into retrieval when there are three sources than when there are three hundred.

Where this lives in the book — direct links:

→ Chapter 4 — What Context Actually Means in AI Systems: /context-advantage/book/chapter-4

→ Chapter 5 — Why Context Comes Before Tools: /context-advantage/book/chapter-5

→ Chapter 6 — Retrieval, Grounding, and the Semantic Layer: /context-advantage/book/chapter-6

→ Chapter 7 — Citations, Attribution, and Auditable Answers: /context-advantage/book/chapter-7