One Model Is a Single Point of Failure

Walk through your company's risk register. The database has replicas. The payment provider has a backup. The message queue has a dead-letter path. Now find the line for the model your agents run on. In most organisations it is not there, even though that model now drafts customer replies, triages tickets, writes queries, and approves routine work.

The omission is understandable. The model feels like a utility — always on, always improving, someone else's problem. But a utility with one supplier is not a utility. It is a single point of failure wearing a service-level agreement.

Providers deprecate the version you tuned your prompts against. They adjust safety behavior in ways that change your outputs. They reprice. They have regional outages. They retire endpoints on a schedule set by their roadmap, not yours. None of this is malpractice. It is what suppliers do. The failure is on the buyer's side: building a production capability on one model and calling the result an architecture.

This is the argument of Chapter 22Chapter 22 · 5 min LockedThe Danger of AI Lock-InHow yesterday's choices limit tomorrow's options.: model choice is not a benchmark ranking. It is a resilience decision.

The first failure is the obvious one: an outage. The endpoint stops answering, and every agent that depends on it stops working at the same time. Because agents are chained into workflows, the blast radius is wider than a failed API call — it is the process the agent was carrying.

The second failure is quieter: deprecation. The version your prompts, few-shot examples, and evaluations were tuned against is retired. The replacement is better on the announcement blog post and different on your workload. You discover the difference in production, one confused customer at a time.

The third failure is behavioral drift. Providers update models for safety, cost, and capability. Your agent's outputs shift by a few percent — tone, refusal rates, formatting, reasoning depth. Nothing is down. Everything is slightly different. Your evaluations, if you have them, are the only thing that notices.

The fourth failure is commercial. When renewal arrives and the price doubles, the team with one model has no negotiating position. The team with a tested fallback has a conversation. Chapter 29Chapter 29 · 6 min LockedCareer Roadmap for the Agentic EraLearning paths for beginners, intermediates, and senior professionals. covers the exit test; the fallback model is the exit test made operational, rehearsed before you need it rather than theorised during a contract dispute.

Some teams resist a second model out of loyalty to the first, as if maintaining a fallback questions the provider's quality. No engineer thinks a database replica questions the database. Redundancy is what you build for components you depend on, precisely because you depend on them.

The resistance is usually practical, not emotional. A second model means a second prompt-tuning effort, a second evaluation pass, a second cost profile. That is real work. But it is front-loaded work that buys three things: continuity when the primary fails, leverage when the primary reprices, and information — a constant, honest comparison of what your workload actually costs and how well it actually performs across providers.

That comparison is worth more than any benchmark. Chapter 23Chapter 23 · 5 min LockedOpen Formats, Open Interfaces, Open ThinkingWhy openness is a strategic moat. argues that model selection should come from your evaluation set, not a leaderboard. Running two models against your real tasks turns that from an annual procurement exercise into a standing measurement.

A resilient agent stack has three tiers, and each tier has a defined trigger. The primary model handles the workload under normal conditions. The fallback model takes over when the primary is down, degraded, deprecated, or priced out — and it is warmed, tested, and evaluated continuously, not installed during an incident. The manual path is the human procedure for work that cannot wait for either model: the queue that pauses gracefully, the approval that routes to a person, the customer message that says when service resumes.

The triggers matter as much as the tiers. Failover on a hard error is easy. Failover on quality degradation is harder and more valuable: if the primary's evaluation score on your canary set drops below a threshold, traffic shifts before customers notice. That requires the evaluation harness from Chapter 21Chapter 21 · 6 min LockedQuality, Speed, and Cost TradeoffsHow to balance accuracy, latency, and spend. to run continuously, not just at launch.

Notice what the manual path implies: the agent was never the process. The process is the outcome — the ticket resolved, the order approved, the report delivered. The agent is one way to get it done. Organisations that forget this discover, during an outage, that they no longer employ anyone who knows how to do the work by hand.

A fallback only works if your agent can actually run on it. That is a property you build, not one you discover. Prompts written against one model's quirks — its formatting habits, its system-prompt behavior, its tool-calling dialect — do not transfer cleanly. Tool schemas, output parsers, and retry logic accumulate provider-specific assumptions the way old code accumulates comments that say do not touch this.

The portable design keeps the model behind an internal interface: your agent speaks in tasks and structured outputs, and a thin adapter translates to each provider's API. Prompts live in version control with the evaluation results they earned. Tool definitions are expressed once, in your schema, and compiled to each provider's format.

Context is the hardest part to move, and the most important. Your retrieval pipeline, memory objects, and compaction policy should be provider-neutral by construction — they are yours, after all. Chapter 12Chapter 12 · 7 min LockedPortable Context: The Open Contract for AgentsContext is your IP — it must move across agents, models, and clouds through an open contract, not sit locked inside one runtime. makes this the core argument for portable context: the organisation's knowledge layer must outlive any model. A team that can carry its context, prompts, and evaluations to a new model in a week has real choice. A team that cannot has a logo on its dependency.

Every organisation has a backup plan. Few have a backup. The difference is testing. A fallback model that has never processed production-shaped traffic is a hypothesis. Its latency under your load, its behavior on your hardest ten percent of tasks, its cost at your real volumes — all unknown.

The practical pattern is a standing shadow: a small, fixed share of real traffic — even one percent — runs through the fallback continuously, with outputs scored by the same evaluations as the primary. You learn the quality gap, the cost gap, and the latency gap while stakes are low. When the gap is too wide, that is a finding, not a failure: it tells you to invest in the fallback's prompts or pick a different fallback before you need it.

This is the eval-set discipline of Chapter 21Chapter 21 · 6 min LockedQuality, Speed, and Cost TradeoffsHow to balance accuracy, latency, and spend. applied to resilience. The evaluation harness is the leash for every model you run, including the one you hope never to need. And as Chapter 35Chapter 35 · 11 min LockedRetrieval Mechanics: Chunking, Hybrid Search, and RerankingThe engineering layer under every context strategy — chunking, hybrid search, reranking, and how to prove it works. notes for retrieval, the quality of a component you cannot measure is a rumor.

Fire drills exist because the worst time to learn the evacuation route is during the fire. The same holds for model failover. Once a quarter, on a calendar rather than in a crisis, run the drill: switch a real workload to the fallback, measure quality and cost and latency, compare against the primary's baseline, then return traffic and write down what changed.

The drill answers the questions that matter. How long does the switch take — minutes or days? Who is allowed to make it? Do the prompts, tools, and context pipeline actually work on the fallback, or has drift quietly broken the path since last quarter? What did the work cost on the second model, and what would a month of it do to the budget from Chapter 18Chapter 18 · 6 min LockedThe Hidden Cost of Agentic AIWhere the dollars actually go.?

Record the results like an incident review. The drill's output is not reassurance; it is a current, honest number for how dependent you are and how much that dependence would cost to end. That number belongs in front of whoever signs the next renewal.

Procurement teams often negotiate exit clauses and call the result optionality. But a contractual right to leave is not the ability to leave. Choice, in the sense this book uses the word, is an operational capability: the demonstrated, rehearsed ability to move a workload to a different model, at acceptable quality and cost, inside a window the business can tolerate.

That capability changes every conversation. With it, a price increase is a negotiation. Without it, a price increase is an invoice. With it, a deprecation notice is a routine migration. Without it, a deprecation notice is an emergency project with a deadline someone else set.

The fourth C exists for exactly this reason. Chapter 34Chapter 34 · 7 min LockedDesigning Multi-Agent Systems That Actually WorkWhen one agent is not enough — and how to make many cooperate without chaos. frames choice as the discipline that keeps the other three — context, control, cost — from being renegotiated upward at every renewal. The fallback model is where that discipline becomes physical: a running system, not a paragraph in a contract.

Week one: inventory. List every production agent, the model and version it runs on, the tasks it carries, and what breaks if that endpoint disappears for a day. Most teams have never written this down. The list alone changes the conversation.

Week two: select and wire the fallback. Pick the strongest alternative for your two or three highest-volume workloads — a different provider where possible, since same-provider fallbacks share the provider's bad days. Put it behind your internal interface, port the prompts, and connect it to your evaluation harness.

Week three: shadow and measure. Run one percent of real traffic through the fallback. Score it with the same evaluations as the primary. Record the quality, cost, and latency gaps, and close the worst of them.

Week four: run the first drill. Switch one workload fully, measure, compare, return, and write the review. Then put the drill on the calendar. You now own something most organisations only claim: a tested answer to the question of what happens when the model fails. Related reading: The Exit Test, Open Weights Are Not Freedom, and Design for the Bad Day.