Why Tool Calling Multiplies Your Agent Token Bill and How to Control It

Here is an illustrative composite, not a single real team. A data platform team builds an agent that answers questions about orders. Before launch they estimate cost the usual way: an average question is about a hundred tokens, an average answer about three hundred, multiply by expected volume, add a margin. The number looks small enough to approve without a meeting.

A month after launch, the bill is roughly eight times the estimate. Nobody made a mistake in the arithmetic. The model price did not change. Usage was close to forecast. What the estimate missed was everything the agent sent to the model that the user never saw: the descriptions of twenty tools on every single call, the full JSON of every query result, and a conversation history that got longer with each step.

This is the tool calling tax. It is the most common reason agent budgets break, and it is almost invisible in a demo, because demos are short. It appears only when real users ask real questions that take five, eight, or twelve tool calls to answer.

To see the tax, look at what one model call in an agent loop actually contains. There is a system prompt with instructions and policies. There is a list of tool definitions, each with a name, a description, and a parameter schema. There is the conversation so far, including every earlier tool call and every result that came back. Then, finally, there is the new user question, which is often the smallest part.

Now remember that an agent does not make one model call. It makes one per step. A question that needs a lookup, a query, a calculation, and a final answer is four or five calls, and each call resends the instructions, the tools, and the whole history. The history is the part that grows. By the last step, the model is rereading everything it already read, plus everything the tools returned.

Chapter 18 calls this the hidden cost of agentic AI, and it is hidden precisely because it is structural. No single call looks expensive. The cost lives in the shape of the loop.

If each step added a fixed number of tokens, cost would grow in a straight line. It does not, because each step also resends all the steps before it. A loop with ten steps does not cost ten times one step. Because the context is reread at every turn, total input tokens grow roughly with the square of the number of steps when results are large. Long tasks are disproportionately expensive.

Tool definitions make it worse in a quieter way. Twenty tools with careful descriptions and detailed schemas can easily run to several thousand tokens. That block is paid on every call, whether the agent uses one tool or none. Teams that added tools generously, often for good reasons, discover that the toolbox itself is one of their largest line items.

The practical lesson is simple. Agent cost is not mainly a function of model price. It is a function of how much context you carry, how many times you carry it, and how much of it the task actually needed. That is a context engineering problem, which is why Chapter 20 treats budget-aware design as part of architecture rather than finance.

The most expensive habit in agent design is returning raw tool output straight into the context window. A query returns five hundred rows, and all five hundred go back to the model. An API returns a full customer record with forty fields, and all forty go back, even though the agent needed two. A document search returns ten full pages when one paragraph held the answer.

Raw payloads hurt three ways at once. They cost tokens on the step where they arrive. They cost tokens again on every later step, because they now live in the history. And they often lower quality, because the model has to find the relevant fact among a great deal of noise, the same failure we described in The Context Window Is Not Context.

The fix is to treat every tool result as something to be shaped before the model sees it. Return the columns the task needs, the rows that matter, and a short summary of the rest. Give the agent a way to ask for more detail if it needs it, rather than handing it everything in advance. This is the same principle behind Chapter 36 on compaction: keep what the next step needs and move the rest out of the window.

Prompt caching is one of the best tools for agent cost, because the instructions and tool definitions at the start of every call are usually identical. When the provider can reuse that repeated prefix, the cost of carrying it drops sharply, and so does latency. We covered the discipline in detail in The KV Cache Discipline.

Tool calling quietly breaks caching in ways that are easy to miss. Tools listed in a different order on each request. A timestamp or user name inserted near the top of the system prompt. Tool descriptions generated dynamically per session. A tool list that changes depending on the user. Each of these changes the prefix, and a changed prefix usually means no cache reuse at all.

The rule is to put everything stable first and in a fixed order: instructions, then tool definitions, then anything that changes per user or per turn. Treat the stable prefix as a versioned artefact, and change it deliberately rather than by accident. A team that does only this often sees a meaningful drop in cost without touching the model or the tools.

First, scope the toolbox per task. Do not send every tool on every call. Route the request first, then load only the tools that kind of request needs. An order question does not need the tools for refunds, email, and calendar. This mirrors the idea in Chapter 19 that not every task needs the most capable model: not every task needs the whole toolbox either.

Second, write tool definitions like expensive real estate. Clear, short descriptions and tight schemas reduce tokens and improve tool selection at the same time. Our essay on tool contracts shows how a five-part contract can be both precise and compact.

Third, add result adapters. Put a thin layer between each tool and the model that trims fields, caps rows, and summarises long output. The adapter is where you decide what the model is allowed to see, which also makes it a natural place for the permission checks described in Chapter 14.

Fourth, compact the history. After a step has been used, replace its full result with a short note of what was learned. Long-running agents should carry conclusions forward, not transcripts, as The Context Degradation Curve explains.

Fifth, set a budget per task, not only per month. Give each run a maximum number of steps and a maximum token spend, and decide in advance what happens when it is reached: return a partial answer, ask the user, or hand off to a person. A monthly budget tells you that you overspent. A per-task budget stops it from happening.

You cannot reduce what you cannot see. Log, for every agent run, the number of steps, the input and output tokens per step, how much of the input was tool definitions, how much was tool results, and how much of the prefix was served from cache. These five numbers explain most agent bills.

Then look at the distribution, not the average. A small share of long, looping runs usually drives a large share of cost. Those runs are often also the ones that fail, because an agent that keeps calling tools is frequently an agent that is lost. Cost and quality problems tend to live in the same traces, which is why Chapter 21 treats quality, speed, and cost as one tradeoff rather than three separate dashboards.

Finally, tie the numbers to outcomes. Cost per resolved question, not cost per token, is the figure that matters to the business. Every Agent Needs a P&L shows how to frame that for leaders, and Cheaper Tokens Will Not Save Your AI Budget explains why waiting for prices to fall is not a plan.

It is tempting to hand agent cost to finance or to the platform team. Neither can fix it alone. The decisions that drive the tool calling tax are decisions about context: which tools exist, how they are described, what their results look like, what the agent remembers, and when it should stop. Those are design decisions, and they belong to whoever designs the agent's context.

That is the core argument of Chapter 30 on the trusted agent architecture. Context, Control, and Cost are not separate concerns. A result adapter that trims fields saves tokens, reduces exposure of sensitive data, and improves answers, all at once. A per-task budget limits spend and also limits how far a confused agent can wander.

If you are preparing for the Databricks Context Engineer Associate practice exam, this is exactly the kind of reasoning the token budget and compaction questions test: not memorising prices, but understanding what fills the window and why.

Use this list to review any agent before it goes to production.

1. You know the average and the worst-case number of steps per run.

2. You log tokens per step, split into instructions, tool definitions, history, and results.

3. Only the tools relevant to the task are sent on each call.

4. Tool descriptions and schemas are short, precise, and reviewed like code.

5. Every tool result passes through an adapter that trims fields and caps rows.

6. Large results are summarised, with a way for the agent to ask for detail.

7. Completed steps are compacted into short notes rather than kept as full transcripts.

8. The stable prefix comes first, in a fixed order, and cache hit rate is monitored.

9. Every run has a maximum step count and token budget, with a defined fallback.

10. Cost is reported per resolved task, not only per token or per month.

If most of these are true, your agent bill will track the value it delivers rather than the length of its loops. The full framework for Cost, and how it connects to Context, Control, and Choice, is in the book.