Why Does Prompt Caching Save Less Than the Pricing Page Says?
Prompt caching is a real discount on a narrow slice of your bill. What you keep depends on the write premium, the expiry window and a hit rate measured in dollars.
The pricing page says a cached token costs a fraction of a normal one. That is true, and it is still not the number on your invoice. Three other numbers decide the gap: the write premium, the expiry window, and your real hit rate. Many teams never put all three in the same spreadsheet.
Prompt caching saves money only when reuse inside the cache lifetime outweighs the write premium. On a busy, stable prefix that can be most of your input bill. On sparse traffic with a short expiry it can cost more than sending the same tokens uncached. Arithmetic decides which one you have.
My background is data science consulting at Deloitte, Booz Allen Hamilton and Accenture, and the habit that stuck is refusing a unit price until I have seen the denominator. A read multiplier is a unit price. My read is that you should treat it as a ceiling you approach and never reach.
Three numbers sit between that ceiling and your bill.
The write premium is what you pay the first time a prefix enters the cache. It is above the normal input price, and it buys nothing until something reads it.
The expiry window is how long a prefix survives without a hit. Miss the window and you pay the write premium again.
The real hit rate is the one measured in dollars, not tokens. The two are easy to confuse, and they can sit a long way apart.
What the pricing pages promise in November 2026
Every figure in this section comes from provider documentation as I read it on 9 November 2026. These pages change often, so check them before you model anything.
Anthropic's documentation states that cache writes with a five minute lifetime cost 1.25 times the base input price, writes with a one hour lifetime cost 2 times, and reads cost 0.1 times, with footnoted exceptions of 0.05 and 0.025 on some newer models. The default lifetime is five minutes. Minimum cacheable length runs from 512 to 4,096 tokens depending on the model, and shorter prompts are "processed without caching, and no error is returned." That last clause deserves more attention than it gets. A prefix one token too short fails silently.
OpenAI's guide lists a minimum of 1,024 tokens for GPT-5.6 and later, and cached input "discounted up to 95%." On those newer models the default 30 minute retention is written at 1.25 times the uncached input rate and the 12 hour option at 2.5 times. The guide calls these total write rates, not surcharges on top of ordinary input. Reads are 0.1 times on most of those models.
Google's documentation says implicit caching is on by default for Gemini 2.5 and newer, with a minimum of 4,096 tokens on the newer models. One row of its pricing page, Gemini 3.5 Flash on the standard tier, lists $1.50 per million input tokens, $0.15 cached, and a storage price of $1.00 per million tokens per hour. Treat that as one row on one page.
The storage line changes the shape of the problem, because it is rent. On that row each read saves $1.35 per million tokens, and each hour of holding the cache costs $1.00 per million. By my arithmetic you need about 0.74 reads per hour, or one every 81 minutes, just to cover the rent.
No provider is hiding any of this. It is all published. It simply lives in several rows, and the headline only quotes one of them.
Prompt caching break even math
Derive this yourself instead of borrowing a blog's number, because published break even figures use different definitions of a reuse and they do not agree.
Let w be the write multiplier, r the read multiplier, and n the number of reuses after the first request. Normalise the base input price to 1. Without caching, one request plus n reuses costs 1 + n. With caching, the same traffic costs w + n × r. Caching wins when w + n × r is less than 1 + n, which rearranges to n greater than (w minus 1) divided by (1 minus r).
Run it for the three write prices above, each with a 0.1 read:
Write at 1.25: n must exceed 0.28, so one reuse pays. You spend 1.35 instead of 2, which is 32.5% off.
Write at 2: n must exceed 1.11, so you need two reuses. One reuse costs 2.1 against 2, a small loss. Two reuses cost 2.2 against 3, which is 26.7% off.
Write at 2.5: n must exceed 1.67, so again two reuses. They cost 2.7 against 3, which is 10% off.
Now hold the write at 1.25 and let reuse grow. One reuse saves 32.5%. Two save 51.7%. Nine save 78.5%. Ninety nine save 88.9%. The 90% on the pricing page is the limit as n goes to infinity, and no prefix you ship will live that long.
That is the first gap, and it is the small one.
Where the discount leaks
Leak one is expiry. The formula above assumes every reuse arrives in time. Firecrawl put the failure plainly on 18 October 2026: "A 5-minute cache on requests that arrive more than 5 minutes apart is a loss." Each request misses, writes again at 1.25 times, and you pay 25% more than with no caching at all.
Real traffic sits between those extremes, so model it. Let p be the share of requests that arrive inside the lifetime of the previous one. Average cost per request is (1 minus p) × w + p × r, and caching breaks even when p equals (w minus 1) divided by (w minus r). At a 1.25 write and a 0.1 read, that is a p of 21.7%. At a 2 write it is 52.6%.
If arrivals are random with a mean gap g and a lifetime T, then p is 1 minus e to the power of minus T divided by g. That assumption is mine, and bursty traffic will do better inside a burst and worse between them. It assumes a hit resets the clock, which is how Firecrawl describes the five minute cache refreshing itself. Here is what it gives on a 0.1 read:
Mean gap of 2 minutes, five minute lifetime at 1.25: 80.6% saved.
Mean gap of 10 minutes, five minute lifetime at 1.25: 20.3% saved.
Mean gap of 10 minutes, one hour lifetime at 2: 89.5% saved.
Mean gap of 60 minutes, five minute lifetime at 1.25: 15.8% more expensive than no cache.
Mean gap of 60 minutes, one hour lifetime at 2: 20.1% saved.
The pricier write wins by a wide margin once traffic thins out. Under this model the five minute cache stops paying at a mean gap of about 20 minutes, and the one hour cache at about 80.
Leak two is dynamic content inside the prefix. A timestamp, a user name or a shuffled tool list near the top changes the bytes, and everything after the change stops matching. Nothing errors. You just pay to write a new entry on each request.
Get Syed Ali’s stories in your inbox
Join Medium for free to get updates from this writer.
Leak three is routing. A cache belongs to a provider and, in practice, to a model, so a router that spreads traffic thins the reuse on every path. I wrote about measuring that separately in LLM Routing Saves Less Than the Papers Claim. The two savings do not add up, because each one eats the other's denominator.
How to measure your real hit rate
Every major API already returns what you need. Anthropic's usage block has cache_read_input_tokens, cache_creation_input_tokens and input_tokens, and its documentation says total input is the sum of the three. OpenAI's has cached_tokens and cache_write_tokens under input_tokens_details. Google's documentation names usage.total_cached_tokens.
Call those R for reads, W for writes and U for uncached. Then compute two numbers side by side.
Token hit rate is R divided by (R + W + U). Dollar savings is 1 minus (R × r + W × w + U) divided by (R + W + U).
Take an illustrative workload of 90 read tokens and 10 written tokens at a 1.25 write and a 0.1 read. The token hit rate is 90%. The cost is 9 for reads plus 12.5 for writes, so 21.5 against 100, or 78.5% saved. Writes are one tenth of the tokens and 58% of the cache spend.
The published numbers show the same pattern. Firecrawl, in its October piece, cites an August 2026 analysis of 63,880 billed Claude Code requests with an aggregate hit rate of 96.59%, where writes carried 76% of the cache input cost in a typical session. That is a secondary citation of one analysis. ProjectDiscovery reported in April 2026 that its hit rate went from 7% to 84% and that caching saved 59% against the same token volume at full input rates. That is one team's result. My read of the two together is that a token hit rate flatters you, and the distance between it and dollar savings is the number worth watching.
One more slice before you tell finance anything. The discount only touches cached input. If your prefix is 80% of input tokens, input is 40% of the bill, and you save 78.5% on the prefix, then the bill falls by about 25%. Those inputs are illustrative. Put in your own.
Where this thesis breaks: when not to cache, and other caveats
Counter one: reads are 90% to 97.5% off, so caching always wins. On dense traffic with a stable prefix this is close to right, and the numbers above say so. If your gaps are short, turn it on and go back to work. The read price is still only half the formula, and it says nothing about what happens when requests stop arriving in time.
Counter two: providers cache automatically, so do not engineer it. This is the one I take most seriously, because automatic caching removes real work, and where there is no write premium the downside mostly disappears. But automatic means no code change. It does not mean no design. ProjectDiscovery's move from 7% to 84% came from restructuring prompts, and an automatic cache cannot reorder your prefix for you.
Counter three: semantic caching or batch APIs beat this. My read, and I have not researched it for this piece, is that they solve different problems and can stack. A semantic cache skips the model call. A batch API trades latency for price. Prompt caching does neither.
I am not claiming caching is a bad deal, and I am not claiming any provider misstates its prices. The discount is real and the pages say what they say. I am claiming the headline multiplier is the wrong input for a forecast.
There are cases where none of this applies. If your monthly input spend is small, an afternoon of instrumentation costs more than the leak. If your prefix sits under the provider's minimum, the question never arises.
The multipliers will also move, so hold this article's numbers loosely. One scenario is that write premiums spread to more models and more lifetimes, which makes the math above more important. The other is that competition absorbs them, which leaves expiry and prefix design as the remaining levers. The signal is whether the next revision of each pricing page adds a write premium or removes one.
A decision checklist
The thresholds here are my starting points, not benchmarks. Adjust them to your traffic.
Check the minimum. Confirm the stable prefix is above your provider's cacheable length for that exact model.
Clean the prefix. Move system text, tools and examples to the top and in a fixed order. Timestamps and user fields go after the last cache boundary.
Pick the lifetime with your own gaps. Pull a week of request timestamps per prefix, compute p for each lifetime you can buy, and price both.
Count your paths. List each distinct pair of prefix and model. Many pairs with thin traffic means routing is eating your cache.
Log three token classes per request. Reads, writes and uncached, with the multipliers on the same row.
Alert in dollars. I would alert when dollar savings falls ten points below its trailing four week average, or when writes pass half of cache spend.
The spreadsheet needs eight input columns: date, model, input price per million, w, r, R, W and U. It needs four computed ones: token hit rate, dollars paid, dollars at full input price, and savings percent. One row per model per day is enough.
The move for this week
Before next Friday, pull seven days of usage fields and put token hit rate and dollar savings in adjacent columns. If they are close, you are done and can stop reading articles about this. If they are far apart, the writes are your bill, and the lifetime is the first thing to reprice.
The pricing page quotes the best request you will ever send. Your invoice is the average of all the others.
Syed Ali
Founder and CEO, Echonos
Syed Ali is cofounder of Echonos, an AI music video pipeline for indie artists. He was previously COO at Tabler App (1M+ users, $50K+ MRR, successful exit) and a data science consultant at Deloitte, Booz Allen Hamilton, and Accenture. Find him on LinkedIn: https://www.linkedin.com/in/syedali-/
Sources
- Anthropic documentation, prompt caching, read 9 November 2026: cache writes at 1.25 times the base input price for a five minute lifetime and 2 times for a one hour lifetime; reads at 0.1 times with per model exceptions; default lifetime five minutes. https://platform.claude.com/docs/en/build-with-claude/prompt-caching
- 2. OpenAI documentation, prompt caching, read 9 November 2026: minimum cacheable length of 1,024 tokens on the newer GPT models; cached input "discounted up to 95%"; default retention written at 1.25 times the uncached input rate, 12 hour retention at 2.5 times. https://developers.openai.com/api/docs/guides/prompt-caching
- 3. Google Gemini API pricing, read 9 November 2026, Gemini 3.5 Flash standard tier: input $1.50, context caching $0.15, storage "$1.00 / 1,000,000 tokens per hour". https://ai.google.dev/gemini-api/docs/pricing
- 4. Google Gemini API, context caching, read 9 November 2026: "Implicit caching is enabled by default for all Gemini 2.5 and newer models." https://ai.google.dev/gemini-api/docs/caching
- 5. Firecrawl, 18 October 2026: a short cache on requests spaced beyond its lifetime is a loss, per the quote in the body. Also cites an August 2026 analysis of 63,880 billed Claude Code requests: aggregate hit rate 96.59%, writes 76% of cache input cost in a typical session. https://www.firecrawl.dev/blog/prompt-caching
- 6. ProjectDiscovery, 10 April 2026, "How We Cut LLM Costs by 59% With Prompt Caching": "Overall, caching saved 59% on LLM costs compared to what the same token volume would have cost at full input rates"; cache hit rate 7% to 84%. https://projectdiscovery.io/blog/how-we-cut-llm-cost-with-prompt-caching
