Design for the Bad Day

Ask a team how their agent behaves when the primary model provider returns errors for forty minutes, and you will usually get one of two answers. The first is a shrug. The second is a retry policy, described with more confidence than a retry policy deserves. Very few teams can describe what a user sees during those forty minutes, because very few teams have decided.

That is the whole problem in one sentence. Most agentic systems in production have two states, working and broken, and the transition between them is a coin toss owned by somebody else. Every mature discipline in computing eventually learns to reject that binary. Payment systems fall back to offline authorisation. Content platforms serve stale cache rather than an error. Aircraft avionics have laws that reduce automation rather than surrender it. Agentic AI is roughly where web infrastructure was twenty years ago: fast, impressive, and one dependency away from a blank page.

Graceful degradation is the discipline of deciding, in advance and in writing, what your system gives up first. It is not pessimism and it is not over-engineering. It is the recognition that an agent depends on more moving parts than any application you have shipped before — a model you rent, an index you built, a set of tools that each have their own uptime, a memory store, and a policy layer — and that the probability of all of them being healthy at once is lower than any single number on any vendor status page.

This is a Choice question before it is a reliability question. A system that cannot run in a reduced mode cannot substitute anything, and a system that cannot substitute anything has no leverage over the people it depends on. Chapter 34 makes the argument that optionality is architecture rather than procurement. Degradation is where that argument becomes code.

Start with an inventory, because you cannot degrade gracefully across a surface you have never enumerated. A typical enterprise agent depends on six classes of thing, and each fails differently.

The model is the obvious one, and it fails in three distinct ways that people wrongly treat as one: hard errors, latency inflation, and silent quality drift after a version change. The retrieval layer fails when the index is stale, when embeddings were rebuilt with a new model, or when the vector store is healthy but returning irrelevant material — the worst case, because nothing alerts. Tools fail on their own schedule, and the tool you depend on most is usually the one owned by the team least aware that an agent is calling it.

Memory fails quietly. If the store that holds prior interactions is unavailable, the agent does not error; it simply forgets, and a forgetful agent produces answers that are individually plausible and collectively inconsistent. Policy fails when the governance service that decides what a caller may see is unreachable, and the honest default there is refusal rather than a guess. Finally, your own context sources fail: a definition changes, an upstream table lands late, and the agent answers correctly against yesterday.

Write those six down with a column for what the user should experience when each one is unavailable. That single table is more valuable than most resilience tooling, because it forces a product decision that is currently being made accidentally by whichever exception propagates first. Chapter 13 covers the dependency surface of agentic systems, and Chapter 15 covers the governance layer that has to keep its word even when everything above it is impaired.

The pattern that works in practice is a small ladder. Four tiers is enough; more than four never gets tested.

Tier 0 is the full answer. Best model, full retrieval, tools live, memory warm, a cited response and an action taken. This is the mode your demo runs in and the mode your architecture diagram describes.

Tier 1 is the narrowed answer. Something is impaired, so the system reduces scope rather than quality. It answers from a smaller, higher-confidence slice of context. It declines the ambiguous variants of the request. It uses a secondary or smaller model with a tighter prompt and a shorter reasoning budget. Crucially, it says so — quietly, in one line, without theatre.

Tier 2 is evidence without conclusion. The agent no longer asserts an answer. It returns the material a competent human would have used: the three most relevant passages, the query it would have run, the numbers it could verify, the constraint it could not satisfy. This tier is wildly undervalued. For a large share of enterprise questions, retrieval plus citation is eighty percent of the value, and it survives the loss of the expensive part of the stack.

Tier 3 is the human handoff, and it is a feature rather than an apology. A good handoff carries the full request, the constraints already parsed, what was attempted, what was ruled out, and a named queue with an expected response time. A bad handoff says that something went wrong and please try again later, which converts a system failure into a customer failure.

Notice what makes this ladder work: each tier is defined by what it keeps, not by what broke. Users cannot reason about your provider outage. They can reason about being told that the system can show them the source documents but will not draw a conclusion today. Chapter 14 treats the human boundary as part of the control surface, and Chapter 29 covers the interaction patterns that make reduced modes legible instead of alarming.

A ladder is only useful if something decides when to step down it, and that decision must not be made by a human reading a dashboard at two in the morning. It must be a number that was agreed while everyone was calm.

Borrow the error budget idea and apply it to answers rather than requests. For each workflow, state the service level in terms the business recognises: the share of requests that must receive a Tier 0 answer, the acceptable latency at the ninety-fifth percentile, and the maximum share that may reach Tier 3 in a day. Then define the automatic triggers. If grounded-citation rate falls below a threshold, drop to Tier 2, because an agent that cannot cite should not conclude. If p95 latency doubles, drop to Tier 1 and shorten the reasoning budget. If the policy service is unreachable, refuse and log, always.

The reason to write the triggers down is not elegance. It is that impaired systems produce their most expensive failures during the interval between degradation and detection. A quality trigger tied to citation rate or retrieval score catches the silent case — the model version that changed under you, the index that rebuilt with mismatched embeddings — which no availability monitor will ever see.

There is a cost dimension here too, and it cuts in a friendly direction. A tier ladder is also a spend ladder. Reduced modes are cheaper, which means the same machinery that protects you on the bad day gives you a lever on the expensive day: when a campaign triples traffic, you can hold Tier 0 for the workflows that earn money and put the rest on Tier 1 deliberately rather than letting your rate limits choose for you. Chapter 18 covers the cost curve of agentic work, and Chapter 21 covers budget-aware design, which is degradation viewed from the finance side of the table.

Every reduced tier assumes you can put something else in the socket, and that assumption is where most programmes quietly fail. Teams believe they are multi-model because they have accounts with two providers. Accounts are not substitution. Substitution means a request can be routed to a different model, in production, today, without a code change, and that you know what the answer quality will be when it happens.

That requires four things, and only the first is technical in the boring sense. First, a routing layer that owns model selection so that no application code names a provider. Second, prompts and tool schemas kept in a portable form rather than tuned into one vendor's proprietary features, because the deeper you go into vendor-specific orchestration the less of your logic travels. Third, the same evaluation suite run against every candidate model on a schedule, so the fallback has a known score rather than a hopeful one. Fourth, a cache of recent grounded answers, which is the cheapest and most overlooked fallback in the entire stack — for many read-heavy workflows, a yesterday answer clearly labelled as yesterday is far better than nothing.

The honest test is a drill: disable your primary model in production for fifteen minutes on a Tuesday afternoon and watch what users see. Teams that have never done this always discover the same three things. A hardcoded model name in one service that predates the router. A tool timeout that was tuned to the primary model's latency and now fires constantly. And an error string reaching users somewhere, because one path never got a tier.

This is the practical face of the argument in Chapter 19 about routing as a first-class layer and Chapter 12 about keeping context portable. Portability that has never been exercised is a belief, not a capability.

The failure mode that damages trust is not the reduced answer. It is the reduced answer delivered in the confident voice of the full one. A system that silently drops from certified sources to a general model, and keeps the same tone, has not degraded gracefully. It has started lying politely.

So reduced modes need three properties. They must be labelled, in one plain line that says what the system did not do: that it answered from cached material, that it could not verify against the certified source, that it narrowed the question. They must be logged with the tier recorded on the trace, so that next week you can answer how many of last month's answers were Tier 1 and whether anybody noticed. And they must be constrained: some actions are not permitted below Tier 0 at all. Nothing that moves money, changes an entitlement, or communicates with a customer should ever execute in a reduced mode. Read degrades. Write refuses.

That last rule is worth defending in writing, because it will be argued against during an incident by somebody reasonable who wants the queue to clear. The asymmetry justifies it: a slightly worse read costs a little accuracy, while a write executed on impoverished context costs a remediation project. Chapter 16 covers permissioning and the boundary around consequential actions, and Chapter 17 covers the audit trail that makes a degraded decision reviewable later.

There is a cultural benefit as well. Once tiers are visible in the product, quality stops being a vague complaint and becomes a measurement. Nobody argues about whether the agent felt worse last Thursday when the trace says nineteen percent of answers were served at Tier 1.

Consider a support agent handling entitlement questions. The primary model is degraded. In a two-state system, the widget spins and then shows a generic error, and the customer opens a second ticket about the first ticket. In a tiered system, the same forty minutes look different.

The router detects elevated error rates and moves to the secondary model. Latency rises slightly. Most answers still land at Tier 0 with citations, and nobody outside the on-call channel notices. Retrieval quality then dips because a dependent service is also affected, citation rate crosses the threshold, and the workflow drops to Tier 2. Customers now see the relevant policy passages, the specific clause that governs their case, and a clear line stating that the assistant is not drawing a conclusion right now. A subset of requests — the ones involving refunds, which are writes — route directly to Tier 3, arriving in a human queue with the account, the entitlement already resolved, and the customer's question quoted verbatim.

Total business impact: slower resolution on a minority of cases and a busier afternoon for the support team. Total trust impact: near zero, because at no point did the system pretend, and at no point did a customer face a blank apology. That difference is not a model capability. It is four decisions made in advance by people who assumed the bad day would come.

The reason this matters more each quarter is that agents keep moving closer to consequential work. When the system was a chatbot, an outage was an inconvenience. When the system approves, schedules, refunds, and writes, an undefined failure mode is a business risk with no owner. Chapter 3 makes the case that capable models need capable systems around them; degradation is the least glamorous and most load-bearing part of that system.

This work does not need a programme. It needs five focused days on your single most important workflow, then repetition.

On day one, write the dependency table: six rows, one column for the intended user experience when each dependency is unavailable. On day two, define your four tiers for that workflow in concrete terms — which model, how much context, which tools, what the user is told, what is forbidden. On day three, implement the labels and the trace field before implementing anything clever, because you cannot manage tiers you cannot count. On day four, set the triggers: two availability triggers, one latency trigger, one quality trigger tied to grounding or citation rate, and one hard refusal rule for the policy service. On day five, run the drill in production for fifteen minutes and record everything you see.

Then do the unglamorous part: fix what the drill found, and put the drill on a monthly calendar with a named owner. A degradation plan that is not exercised decays faster than the code it protects, because every new tool and every new prompt adds a path that was never assigned a tier.

The organisations that will look competent during the next platform incident are not the ones with the best provider. They are the ones who decided, on a calm Tuesday, exactly what they would give up first.