How it works
A tool budget is a counter the harness owns, not the model. Before a run starts, you assign it an allowance — a number of tool calls, a token ceiling, a wall-clock limit, or a currency figure — and the runner decrements it every time the agent acts. The model does not police itself; the loop does.
The control loop is simple:
- Set an allowance per run (and often a tighter sub-allowance per subagent or per tool).
- Decrement on every tool call, token block, or priced unit.
- Check the remaining budget before dispatching the next action.
- When the budget is exhausted, stop the loop and hand back whatever state exists — a partial result, a checkpoint, or an escalation.
The important design choice is what "stop" means. A budget is not a correctness signal, so exhausting it should not be treated as a wrong answer. It should trigger a defined terminal transition: return best-effort, park for review, or escalate to a human. The budget is a spend meter running alongside the eval, never fused with it. That separation is the whole point — it lets you cut off a run that is burning money without pretending the work was incorrect.
Why it matters in an agent harness
Agents fail expensively in ways a static program cannot. A retrieval loop that keeps re-querying, a planner that re-plans on every step, or two subagents that ping-pong can all produce a technically correct answer after a hundred tool calls that should have taken five. Without a budget, correctness hides cost: the run passes the eval and you only discover the spend in the bill.
A tool budget converts that open-ended risk into a bounded one. It caps blast radius in the dimension that unbounded loops actually threaten — spend and latency — and it gives you a clean, observable termination condition instead of a runaway. Because the counter lives in the harness, it also becomes a first-class metric: cost-per-run, budget-exhaustion rate, and spend distribution across domains become things you can chart, alert on, and regression-test. A run that finishes far under budget and one that slams into the ceiling are different signals even when both return the right answer, and only the harness can tell them apart.
Tool budget vs iteration budget
These are often conflated, but they cap different things and fail differently.
| Tool budget | Iteration budget | |
|---|---|---|
| Caps | Spend: tool calls, tokens, cost | Loop steps: reasoning cycles |
| Answers | "How much did this cost?" | "How long did this think?" |
| Typical failure caught | Expensive-but-correct runs | Non-terminating or oscillating loops |
| Natural unit | Dollars, tokens, call count | Turns |
An iteration budget bounds how many times the agent loops; a tool budget bounds what those loops cost. A cheap agent can blow an iteration budget with dozens of near-free reasoning turns, while a lean loop can blow a tool budget in three calls if each one is a large model invocation or an expensive external query. Instrument both. They protect against different runaways, and one is not a proxy for the other.
The Rifty take
We treat cost as a signal with the same standing as correctness, and we refuse to collapse them. A right answer that cost ten times its worth is a failure of the harness, not a success of the agent, and we want the loop to say so out loud. So we keep the budget meter separate from the eval, we prefer a graceful terminal transition over a hard crash, and we accept the tradeoff that a budget will occasionally halt a run that was one call away from finishing. Bounded and legible beats unbounded and lucky.
Common failure modes
- Fusing budget with correctness. Marking a budget-exhausted run as "wrong" corrupts your eval data. Keep spend and correctness as separate axes.
- Budgeting only the top level. A parent run with no per-subagent cap lets a single delegated worker consume the whole allowance. Sub-allocate.
- Counting the wrong unit. A call-count cap misses runs dominated by a few huge token payloads; a token cap misses runs dominated by many cheap-but-slow external calls. Track the unit that actually drives your cost.
- Silent exhaustion. If hitting the ceiling returns an empty or truncated result with no marker, downstream stages treat it as complete. Emit an explicit budget-exhausted state.
- Static budgets across uneven work. One flat number for every task over-funds trivial runs and starves hard ones. Scope the allowance to the task class.
- No observability on the meter. A budget you cannot chart is a limit you cannot tune. Log remaining budget per step so exhaustion rate and cost distribution are visible.
Frequently asked questions
How is a tool budget different from correctness?
A tool budget measures spend — tool calls, tokens, or cost — while correctness measures whether the output is right. They are orthogonal axes. A run can pass its eval and still be a budget failure because it reached the right answer far too expensively, which is exactly the case a budget is meant to catch.
What should happen when the budget is exhausted?
Trigger a defined terminal transition, not a crash. Return the best-effort partial result, save a checkpoint, or escalate to a human, and emit an explicit budget-exhausted marker. Never silently truncate, and never record exhaustion as an incorrect answer — that corrupts both your cost metrics and your eval data.
Should I budget tool calls or tokens?
Budget the unit that drives your cost, and often both. Call-count caps miss runs dominated by a few huge token payloads; token caps miss runs dominated by many cheap-but-slow external calls. In multi-tool loops, track calls, tokens, and wall-clock together so no single expensive dimension escapes the meter.
How do tool budgets work with subagents?
Give each subagent its own sub-allowance drawn from the parent's budget. Without per-subagent caps, a single delegated worker can consume the entire allowance and starve its siblings. Sub-allocation keeps spend legible per worker and prevents one runaway branch from exhausting the whole run undetected.
Isn't a tool budget the same as an iteration budget?
No. An iteration budget caps how many times the loop runs; a tool budget caps what those runs cost. A cheap agent can blow an iteration cap with many near-free turns, while a lean loop can blow a tool budget in a few expensive calls. Instrument both, since neither proxies the other.