Agent Caching Changes the Coefficient, Not the Exponent

For short agents, what you cache decides the bill. Same model, same tokens — change only the caching setup and a 20-turn session goes from $4.05 to $0.97.

For long agents, recording only the preface stops helping. Putting a breakpoint only on the stable prefix, which is what most setups do, saves less as the agent gets longer. Caching the history reverses the direction.

But even caching the history caps out at 90% — fast-forwarding is not free. And cost still grows with the square of the turn count. A 200-turn session costs $36 when cached perfectly. Caching changes the coefficient, not the exponent. For long agents the dominant lever is not caching the history — it is shrinking it.

Three inputs

Claude Opus 5 throughout.

Input	Value	Basis
Stable prefix S	10,000 tokens	browser toolset 6,610 + bash 325 (documented) + system prompt 3,065 (assumed)
Per-turn increment δ	3,000 tokens	output 400 + tool result 2,600 (both assumed)
Turn count N	20	assumed

The 6,935 inside the stable prefix is documented: browser_toolset_20260801 with its default members is 6,610 tokens and the bash tool is 325, and those tokens are re-sent on every request, not once per conversation. The remaining 3,065 and both δ and N are assumptions.

One more assumption matters most. Every savings figure below assumes a 100% cache hit rate — the prefix never breaks and the entry is always alive within its TTL. I could not find public statistics on how often prefixes break in production, and as the invalidation section shows there are several ways for it to happen. These numbers are a reachable ceiling.

Cache pricing is already public

Both providers publish cache rates in their pricing docs. This is not something you negotiate.

Model	Input	5-min cache write (1.25×)	1-hour cache write (2×)	Cache read (0.1×)	Output
Claude Opus 5	$5	$6.25	$10	$0.50	$25
Claude Sonnet 5	$2	$2.50	$4	$0.20	$10
Claude Haiku 4.5	$1	$1.25	$2	$0.10	$5

Per 1M tokens. (Anthropic pricing docs, retrieved 2026-08-23)

The 10% read rate is not one vendor’s policy. All three Claude models plus OpenAI’s gpt-5.6-sol ($4.00 → $0.40) and gpt-5.4-mini ($0.75 → $0.075) land on exactly 10%. Only the older, smaller gpt-4o-mini sits at 50% ($0.15 → $0.075), because its cached read matches gpt-5.4-mini’s while its input price is one-fifth. (OpenAI pricing docs, retrieved 2026-08-23)

The docs answer the cache break-even directly: a 5-minute cache pays for its write after one read, a 1-hour cache after two. That holds for reuse within the TTL. If the gap between uses exceeds five minutes you get a rewrite instead of a read, which is a loss, not a recovery.

74% of an agent’s input is history

The API is stateless, so every turn resends the entire history — the narrator starting from page one each time. That makes input tokens grow with the square of the turn count.

Component	Calculation	Tokens	Share
Stable prefix, repeated	20 × 10,000	200,000	26%
Accumulated history	3,000 × 190	570,000	74%
Total input	200,000 + 570,000	770,000	100%
Total output	20 × 400	8,000	—

74% of input is history.

Prefix-only saves 21%, history saves 76%

Setup	Cache read	Cache write	Uncached input	Total	Multiple
No caching	—	—	$3.85	$4.05	1.00×
Stable prefix only	$0.10	$0.06	$2.85	$3.21	1.26×
Rolling breakpoint	$0.35	$0.42	$0.00	$0.97	4.17×

Output cost is $0.20 in all three, so it is out of the columns and inside the totals.

A single request shows the structure more sharply than the session total. The 20th request carries 67,000 input tokens in every setup — what differs is which rate each slice of those 67,000 pays.

One 20th request	Cache read	Cache write	List-price input	Request cost
No caching	—	—	67,000	$0.3350
Stable prefix only	10,000	—	57,000	$0.2900
Rolling	64,000	3,000	—	$0.0508

The $0.01 of output is identical across the three, so it is out. Prefix-only makes that 20th request just 13% cheaper than no caching at all, because 57,000 history tokens stay at list.

Two prefixes need distinguishing here. The stable prefix is what precedes every request identically (system prompt plus tool definitions), 10,000 tokens. The cumulative prefix is the whole span eligible for caching at a given moment (stable prefix plus history so far) — at turn 20, 67,000 tokens.

A rolling breakpoint moves the breakpoint to the tail of the history every turn. The cumulative prefix up to the previous turn reads at 0.1×, and only the 3,000 new tokens write at 1.25×. Uncached input goes to zero.

Breakpoint count is not the constraint. Rolling needs one at the history tail, plus optionally one on the stable prefix — two. The documented ceiling is four per request, so there is room, and if moving them by hand is awkward the automatic caching option manages breakpoints as the conversation grows. Rolling’s real risk is the TTL, covered below.

The two savings curves diverge and stop at 90%

Turns	History share	No caching	Prefix only	Rolling	Prefix-only savings	Rolling savings
5	37.5%	$0.45	$0.28	$0.22	37%	52%
10	57.4%	$1.28	$0.88	$0.43	31%	66%
20	74.0%	$4.05	$3.21	$0.97	21%	76%
50	88.0%	$21.38	$19.18	$3.49	10%	84%
100	93.7%	$80.25	$75.81	$10.69	6%	87%
200	96.8%	$310.50	$301.56	$36.34	3%	88%
500	98.7%	$1,901	$1,879	$203.29	1%	89%
1,000	99.3%	$7,552	$7,508	$781.54	1%	90%

The two curves run in opposite directions. A team that bolted prefix caching onto short conversations, saw it work, and left it there will keep operating while the effect quietly disappears. At 1,000 turns prefix caching is worth 1%.

Which means the number that matters operationally is the absolute figure, not the percentage. A 200-turn session costs $36 cached perfectly, and a hundred of those a day is $109,000 over 30 days. You cannot cache your way out of that.

The point where history overtakes the stable prefix is computable.

crossover turn = 2S ÷ δ + 1

History is larger from the first integer turn above that value. With S=10,000 and δ=3,000 it is 7.7, so from turn 8. A 30,000-token prefix pushes it to 22 turns; a small δ of 500 pushes it to 42.

Sensitivity to the increment runs the same way.

Per-turn increment δ	History share	Prefix-only savings	Rolling savings
500	32.2%	50%	73%
1,000	48.7%	39%	74%
3,000	74.0%	21%	76%
6,000	85.1%	12%	77%

Bigger tool results neutralize prefix caching while rolling moves within 4.5 points. Agents that read web pages or codebases sit exactly here — the docs put an average 10 kB page at roughly 2,500 tokens, making a page-reading turn’s increment 2,900. This table holds output fixed at 20 × 400 tokens and varies only δ.

So the real lever is shrinking the history

To touch the exponent you have to reduce N or δ. Context editing that deletes old tool results, compaction that replaces the conversation with a summary, and starting fresh with only a summary at segment boundaries all reset N.

Here is 100 turns split into segments. At each boundary the cached context is read to produce a 2,000-token summary, and the next segment starts on a 12,000-token prefix carrying that summary. Summary generation cost is included in the table.

Setup	100-turn cost	Savings vs. straight rolling
Straight rolling	$10.69	—
10 turns × 10 segments	$5.13	52.0%
25 turns × 4 segments	$5.57	47.9%
50 turns × 2 segments	$7.17	32.9%

Even paying for every summary, roughly half of the rolling cost is gone. The three summaries account for 5% of the total. Shorter segments win because shrinking N cuts the quadratic term faster than the extra summaries add cost.

What the table does not include is quality. Summaries lose information, and when the lost information turns out to matter the agent redoes the work. That rework is not priced here. On the cost axis shorter segments are better; on the quality axis they are worse.

Downshifting the model is the second lever, and it multiplies with the caching axis.

Model (20 turns)	No caching	Prefix only	Rolling	vs. Opus rolling
Claude Opus 5	$4.05	$3.21	$0.97	1.00×
Claude Sonnet 5	$1.62	$1.28	$0.39	0.40×
Claude Haiku 4.5	$0.81	$0.64	$0.19	0.20×

Caching’s 4.17× times the model’s 5× puts 20.9× between Opus uncached and Haiku rolling. Model choice is a quality decision that cost alone cannot settle, but the fact that the two axes multiply rather than stand apart changes the order in which you evaluate them.

There is a trap on the way down. The minimum cacheable prefix differs by model, and among these three the cheaper model has the higher floor: 512 tokens for Opus 5, 1,024 for Sonnet 5, and 4,096 for Haiku 4.5. Move a short-prompt workload down to Haiku and caching may not engage at all, in which case the table’s $0.19 never materializes.

One hour wins past 0.65 expirations per session

The default cache lifetime is five minutes. At turn 20 the cumulative prefix is 67,000 tokens, and if the cache expires there the next request is a rewrite rather than a read.

Item	Tokens	Rate	Cost
Cache read (0.1×)	67,000	$0.50	$0.0335
Rewrite (1.25×)	67,000	$6.25	$0.4188
Added cost per expiration	—	—	$0.3852
Against the $0.97 session	—	—	40%

The docs’ “pays off after two reads” is measured against prefix caching, where the write happens once. It does not carry over to rolling, which writes every turn — so the 1-hour premium also applies every turn. On the 67,000 tokens a 20-turn session writes, that premium is $0.2512. The threshold is that premium divided by the cost of one expiration.

Expiration point	Cumulative prefix	Cost per expiration	Threshold
Turn 20 (largest)	67,000	$0.3852	0.65
Turn 10 (mid)	40,000	$0.2300	1.09
Turn 5 (early)	25,000	$0.1437	1.75

Roughly one expiration per session makes the hour worthwhile; below that it depends on when the expiration lands. Workflows with several human approvals, or loops that fire sparsely overnight, clear this bar. Fully automated loops firing seconds apart are better off at five minutes.

The cache breaks without an error

Caching is an exact prefix match. The prefix is assembled in the order tools → system → messages, so a change up front invalidates everything behind it. The ways it breaks:

• Putting the current time, a request ID, or a session ID in the system prompt
• Non-deterministic JSON key ordering
• A tool list that varies per request — tools comes first, so the whole thing goes
• Injecting per-user profile text into the system prompt, giving every user a separate cache

Three usage fields tell you what happened: cache_read_input_tokens for tokens read from cache, cache_creation_input_tokens for tokens written, and input_tokens for everything after the last breakpoint, billed at list. The three sum to total input.

The metric to diagnose with is the uncached share, input_tokens ÷ total input tokens.

Turns	No caching	Prefix only	Rolling
10	100%	57.4%	0%
20	100%	74.0%	0%
50	100%	88.0%	0%
100	100%	93.7%	0%

Rolling working correctly puts this near zero. Anything in the 20–90% range means prefix-only.

The read-to-write ratio diagnoses something else. It is tempting to read a high ratio as healthy caching, but a prefix-only 100-turn session runs 99:1 while rolling runs 50.6:1 — prefix-only is higher, because it writes once and never again. The ratio cannot distinguish the two setups. What it does detect is invalidation: below 1:1 the cache is breaking every turn and you are paying for rewrites with no reads.

Converting measurements into cost looks like this. The three terms are shares of total input tokens, so they sum to one.

input cost multiple = read share × 0.1 + write share × 1.25 + uncached share × 1.0

Watch the minimum cacheable prefix too. Below it a breakpoint passes silently: 512 tokens for Opus 5, 1,024 for Sonnet 5, 4,096 for Haiku 4.5.

Batch discounts do not multiply with rolling

The docs state that cache multipliers stack with other price modifiers, including the Batch API discount and data residency.

Item	Effect	Condition
Batch API	50% off input and output	Asynchronous. Unusable for interactive agents, and incompatible with rolling
Prompt caching	Reads at 0.1×	Prefix match plus TTL survival
US-only inference (inference_geo: "us")	1.1× on everything	When data residency requires it
Dedicated endpoints, volume contracts	Not published	Sales conversation

It is tempting to multiply batch by rolling and write $0.49, but that number does not exist. Rolling requires the cache to survive its TTL between turns, and batch is asynchronous with a long completion window. Every turn expires and rewrites. Batch belongs to high-volume one-shot work with no turns at all.

Add-on charges belong in the model too. Web search is $10 per 1,000 searches, so five per session is $0.05 — 5% of the rolling setup’s token cost, and not ignorable for a research agent that searches heavily. Web fetch adds no charge beyond the tokens of what it retrieves.

Dedicated endpoint and volume contract rates are a sales conversation at both providers. That is where public sources run out. It does not move this post’s conclusion, though: a flat percentage discount applies equally to all three setups, so the savings percentages hold and only the absolute figures shift.

How to decide

Find your range first.

Sessions shorter than the crossover turn. Stable prefix caching is the whole answer. At five turns prefix-only gives 37% and rolling gives 52%. Not worth the engineering.

Between the crossover and 100 turns. Without history caching the savings evaporate. Add rolling. Against the prefix-only setup it saves $2.24 per session at 20 turns and $65.12 at 100.

Above 100 turns. Rolling alone is not enough. Look at compaction or segmentation to reset N first — splitting 100 turns into segments pays for the summaries and still delivers close to half again over rolling. On cost alone 10-turn segments are optimal, but summary loss grows, so around 25 turns is the compromise.

Human involvement that can exceed five minutes. Use the 1-hour TTL in any of the above.

Two organizations run through this model. The inputs (S=10,000, δ=3,000, 400 output tokens per turn) are the ones above, on Opus 5.

Both examples say the same thing. Below 100 turns, adding rolling is the whole job; above it, what happens after rolling is the real problem.

Then work in order.

1. Log the usage fields and check the uncached share against the table above. Not near zero means rolling isn’t engaged; a read-to-write ratio below 1:1 means the cache is breaking every turn
2. Compute the crossover turn, 2S ÷ δ + 1. Any session longer than that needs more than prefix caching
3. Compute the payback threshold. Pricing three engineer-days at $2,000, rolling pays back after roughly 894 sessions at 20 turns and 31 at 100. Fewer sessions per month than that and this is not today’s work. Note that this threshold counts token savings only — it leaves out the time-to-first-token gain from cache hits
4. Strip clocks, IDs, and non-deterministic ordering out of the prefix. One invalidation makes that span 10× more expensive, and unlike a contract negotiation this takes days
5. Look at whether N and δ can come down. Once caching is fully in place and cost still grows quadratically, this is the only lever left

Every number here sits on example values. The conclusions are less sensitive to them.

Caching collapses part one’s savings multiple

Part one used the $1.04 serverless list price for the API side and said outright that, lacking real rates, the comparison tilted toward self-hosting. Now that cache and batch rates are in hand, that model can be revisited.

At cache hit rate h the effective input price is $1.04 × (1 - 0.9h). Part one’s workload is 1:1 input to output, so the blended rate is the average of effective input and output. Batch multiplies both by 50%.

Setup	Required utilization	Savings multiple at saturation	Break-even (billed tokens/month)
Part one’s figure (list)	25.5%	3.92×	5.60B
Cache hit 50%	32.9%	3.04×	7.23B
Cache hit 70%	37.2%	2.69×	8.18B
Cache hit 90%	42.8%	2.33×	9.41B
Batch 50% alone	51.0%	1.96×	11.20B
Cache 70% + batch 50%	74.4%	1.34×	16.35B

Part one’s “3.92× cheaper at 100% utilization” drops to 1.34× once cache and batch are both in. Batch alone (1.96×) outweighs a 90% cache hit rate (2.33×). This closes the asymmetry part one flagged as its top item to report on.

The first row needs a caveat. Part one’s 5.60B and 25.5% come from its zero operator cost, 100% effective throughput row, which part one explicitly called the floor of its own calculation — at $4,000 of operator cost its threshold rises 69%. This table applies API-side discounts on top of that floor.

Three asymmetries remain. First, self-hosted stacks commonly run a prefix cache too. But API-side caching cuts the billed rate to 0.1× while a self-hosted prefix cache cuts prefill compute and raises throughput, and there is no guarantee those two paths cancel at the same multiple. I could not obtain hit-rate data for the self-hosted side.

Second, the 0.9 and 0.5 coefficients come from Anthropic’s and OpenAI’s multipliers. The $1.04 part one used is Together AI’s Llama 3.3 70B rate, and Together’s cache and batch policies went unverified. Third, the formula counts reads only and drops cache writes at 1.25×, understating the API’s real rate. Both push the break-even increase upward.

So read this table for direction and rough magnitude, not precision. The direction is clear: any comparison that prices the API side at list tilts toward self-hosting, and it tilts harder than part one expected.