Agentic AI
-
A 31.6% Hit Rate Is Both a Win and a Loss
The break-even hit rate for prompt caching is not one number. At the same 31.6%, hits bunched into a session's early turns cost 14.4% more than caching only the stable prefix, while hits bunched into the late turns come in 28.3% under. Two axes move the threshold and a third makes it meaningless, so compare measured costs directly instead of converting to one.
-
Agent Caching Changes the Coefficient, Not the Exponent
Putting a breakpoint only on the stable prefix — the common setup — saves less as the agent gets longer. Caching the history reverses that, but it caps out at 90% and cost still grows with the square of the turn count. A 200-turn session costs $36 even when cached perfectly.