Why Your AI Agent Bill Is So High, and How to Cut It

AI agent bills tend to surprise teams for one reason: a demo costs cents, then the same agent in production costs dollars per run. Usually nothing is broken. Agents simply resend their entire history on every step, and that cost compounds.

Here is where the money actually goes and which fixes work, based on current vendor documentation. Savings claims in this space often come from companies selling routing tools, so the worked example below shows its math rather than quoting a percentage.

Why agents cost more than chat

Most model APIs are stateless, so each step of an agent loop sends the system prompt, tool definitions, and everything that has happened so far. Take a 10-step run with a 20K-token prefix, where each step adds 3K tokens of tool results and 1K of output.

• Input across the run: 380K tokens
• Output across the run: 10K tokens
• Cost on Claude Sonnet 5.5 with no caching: $0.86

Almost 90% of that bill is resent context. Double the steps and input grows far faster than output, because every new step carries all the previous ones.

Fix 1: prompt caching

This is usually the biggest and cheapest win. Applying Anthropic prompt caching to the same run, assuming every step hits the cache, gives 56K tokens of cache writes ($0.14), 324K of cache reads ($0.03) and the same output ($0.10). The total falls from $0.86 to $0.27, about 68% less, without changing the model.

The three vendors work differently, and advice written for one can be wrong for another:

Vendor	Setup	Write cost	Read cost	Min tokens
Anthropic	Add cache_control	1.25x (5 min), 2x (1 hr)	0.05x on Opus and Sonnet 5.5	512
OpenAI	On by default	1.25x (30 min), 2.5x (12 hr)	0.05x on GPT-6.1 Sol and Terra	1,024
Google	Automatic	None listed	About 0.05x on Argon	4,096 on Argon

Write costs and read rates above apply to current models; older ones differ. Google also has a rule worth knowing for chatty agents: Gemini 4 caches the full previous request only if the new request adds at least 2,050 tokens. Below that, the last 2,050 tokens are reprocessed at the standard input rate.

What silently breaks your cache

A cache hit needs an exact prefix match, so small changes near the top of a prompt can erase your savings without any error:

• Adding, removing or reordering tools. On Anthropic, changing tool definitions invalidates the entire cache.
• Changing reasoning effort or thinking settings mid-run.
• Putting a timestamp or user ID early in the system prompt.
• Non-deterministic JSON key order in tool calls.
• Compaction or summarizing that rewrites earlier messages.

OpenAI's guidance is a good general rule: append messages rather than rewriting earlier turns, and keep tool definitions stable. Then measure it. OpenAI reports usage.input_tokens_details.cached_tokens, and Google reports usage.total_cached_tokens. If nobody on the team is watching that number, assume the cache is not working.

Fix 2: route work by difficulty

Using a frontier model for every step is the classic expensive default. Classifying a message, extracting a field, or summarizing a tool result rarely needs one.

Tier	Example	Input per 1M	Output per 1M
Small	Claude Haiku 5.5, gpt-6-luna	$0.10	$0.50
Mid	GPT-6.1 Terra	$1	$5
Strong	Claude Sonnet 5.5, GPT-6.1 Sol	$2	$10
Frontier	GPT-6 Astra, Claude Fable 5.1	$10	$50

That is a 100x spread between the ends. Route by task type first, since a fixed rule is easier to debug than a smart router, and only escalate when a cheaper model fails a validation check. My November model comparison has the current data for choosing tiers.

Fix 3: control thinking

OpenAI and Google both state that reasoning or thinking tokens are billed as output, the most expensive token type. Effort is therefore a cost setting. Use the lowest level that passes your evals, and raise it only for the steps that need it.

Output caps need care too. OpenAI warns that if max_output_tokens is reached during reasoning, you can pay for input and reasoning and still get no visible response. It recommends reserving at least 25,000 tokens.

Fix 4: stop running everything in real time

Anything without a user waiting should not pay realtime prices. Batch costs 50% less at Anthropic, OpenAI and Google, which makes it the obvious choice for evals, backfills and nightly enrichment. OpenAI and Google quote a 24-hour window, while Anthropic says most batches finish within an hour.

Flex tiers are the middle ground. Google's Flex is 50% off and synchronous, with a 1 to 15 minute target, but requests can be shed. Its docs recommend it for multi-step agent workflows, so handle retries properly if you use it.

Price traps to check right now

• Long-context surcharges: on GPT-6.1 Sol, going over 272K input tokens raises the price for the whole request.
• Haiku 5.5 over 100K tokens: the price rises from $0.10 / $0.50 to $0.50 / $2.50, and cache reads count toward that length.
• Introductory pricing: Gemini 4 Argon doubles on December 1, 2026, and Gemini 3.8 Flash doubles on January 1, 2027.
• Automatic rerouting: Gemini 2.5 Pro and 2.5 Flash requests move to newer models from December 1 and are billed at those rates.

Add a hard stop

A runaway agent can loop until something stops it, so every run needs limits of its own: a maximum step count, a token budget per run, a spend alert per customer, and rate limits on any API key an agent can reach. These cost almost nothing to add.

Where to start

1. Log tokens, cached tokens and cost per run, per feature.
2. Turn on caching and stabilize your prompt prefix.
3. Add step and budget limits.
4. Move non-interactive work to Batch.
5. Route the simplest step types to a smaller model.
6. Lower effort wherever evals still pass.

The metric to manage is cost per successful outcome, not cost per token. A cheap model that fails and retries can easily cost more than a strong one that finishes first time.

If your AI feature works but the bill does not, I help startups fix this through remote AI product work.