A prompt cache does not remember your prompt. It remembers the prefixes that earlier requests wrote, at the boundaries those requests marked, and it hands back the longest one that still matches. Every surprise about token caching follows from that sentence, including the invoice you did not expect.
Here is the smallest version of the surprise. Your system prompt is 512 tokens. Turn one sends those 512. Turn two sends the same 512 plus a 512-token message. The provider reads 512 tokens from cache, charges full price for the other 512, and the second half stays outside the cache until some later request writes it. Turn three sends 1,536 tokens and reads 1,024.
The cache grows one turn behind you. That is not a bug in any provider; it is what a prefix cache is.
Key takeaways
- A cache entry is a hash of a prefix, written at a boundary you marked. It never grows by itself: the longer request only reuses the part an earlier request already wrote.
- The 512-to-1,024 case has one answer. The first 512 tokens come back at the read price, the new 512 are charged in full, and they become reusable only from the next request.
- The minimum length decides whether you get a hit at all. 512 tokens is cacheable on the newest Anthropic models, 1,024 on OpenAI's GPT-5.6 and later, and 2,048 to 6,144 on Gemini depending on the model.
- Where the breakpoint sits is the whole game. On the block that changes every request, every write is wasted; at the end of the stable block, a single write survives the whole conversation.
- Measure the reported counters, not the feature flag. Every provider tells you how many tokens came from cache on that request, and that number is the only evidence that caching is working.
Four rulebooks, one mechanism
Every major provider caches the same three things: the model's key-value state for a sequence of tokens, a hash of the prefix that produced it, and an expiry. They differ in four places, and each difference shows up on the bill.
Implicit caching is on by default and the provider chooses where the reusable prefix ends. OpenAI does this on every supported model, Gemini calls it implicit caching, and DeepSeek runs it on disk with no opt-in at all.
Explicit breakpoints put you in charge of the boundary. Anthropic takes cache_control on a block, up to four per request. OpenAI supports the same idea on GPT-5.6 and later through prompt_cache_options.mode: "explicit", also with four writes per request.
Named cache objects are Gemini's explicit mode: you create a cache, reference it by name, and pay for storage per token-hour on top of the input tokens. TTL runs from one minute to no maximum, 60 minutes by default.
Router stickiness is not a cache. OpenRouter tracks a conversation so the next request lands on the provider that still holds your prefix, and it derives the conversation key from a hash of the first system message and the first user message. A system prompt that changes every minute breaks that key too, and the conversation moves somewhere with an empty cache.
A timestamp in the tool definitions invalidates all three levels. The same timestamp in the last message invalidates only what comes after it.
The numbers below come from the providers' own documentation in November 2026. Multipliers are similar across vendors; minimums and lifetimes are not, and those two are what decide whether a request hits.
| Anthropic | OpenAI (GPT-5.6+) | Gemini | DeepSeek | |
|---|---|---|---|---|
| Write trigger | cache_control block, or automatic at the last cacheable block |
explicit breakpoint, or implicit at the last eligible message | implicit by default, or a named cache | automatic at request boundaries |
| Minimum prefix | 512 tokens on the newest models, 1,024 to 4,096 on older ones | 1,024 visible input tokens | 2,048 on Gemini 2, 4,096 on Gemini 3 and 4, 6,144 on some 3.x previews | none documented |
| Lookup | prefix hash at each breakpoint, walking back 20 blocks | breakpoint hash, plus up to 20 earlier message endings | best effort | must fully match a stored prefix unit |
| Default lifetime | 5 minutes, refreshed on use; 1 hour if you ask | 30 minutes, or 12 hours at a higher write price | up to 24 hours implicit; 60 minutes default explicit | hours to days, best effort |
| Read price | 0.1x input, 0.05x on Sonnet 5.5 and Opus 5.5 | 0.1x input, 0.05x on GPT-6.1 Sol and Terra | 90% off | discounted hit price |
| Write price | 1.25x for 5 minutes, 2x for 1 hour | 1.25x for 30 minutes, 2.5x for 12 hours | standard input price, plus storage if explicit | not a separate price |
A cached token still counts toward your tokens-per-minute limit on OpenAI, so none of this buys throughput. It buys money.
The 512-token question
Take a 512-token prefix, a 512-token message per turn, and the multipliers most models use: input at $2 per million tokens, a write at 1.25x, a read at 0.1x.
Turn one writes 512 tokens at $0.00128 and reads nothing. Turn two reads 512 at $0.00010 and writes the new 512 at $0.00128. Turn three reads 1,024 and writes the new 512. Over three turns that is $0.0041, against $0.0061 with no caching at all.
Two things go wrong here, and both are common enough that the documentation names them.
Extending a message instead of appending one. OpenAI's guide is explicit. If request one ends with a user message containing Content A, and request two extends that same message to Content A + Content B, the old boundary now sits inside a message. The saved prefix is not reused there without an explicit breakpoint. DeepSeek states the same rule with different words. A request of A + B followed by A + B + C hits the unit A + B; a request of A + B followed by A + C hits nothing, though it does teach the system to persist the shared A for a third request.
Falling under the minimum. On GPT-5.6 and later, 512 tokens cannot be cached at all, because the minimum is 1,024 visible input tokens; the first possible hit is turn two, and it covers the whole 1,024. On Gemini 2 the wait is 2,048 tokens, and on Gemini 3 and 4 it is 4,096. The newest Anthropic models take 512, which makes the example above possible there and nowhere else.
Two breakpoints that lie to you
A breakpoint on the block that changes every request writes a fresh entry each time and never reads one. Anthropic's documentation calls this the common mistake in as many words. The lookup walks backward looking for entries that earlier requests wrote, not for stable content. If nobody wrote an entry at the end of your static block, there is nothing to find behind a timestamp.
The second trap is distance. Anthropic looks back 20 block positions from a breakpoint; OpenAI checks up to 20 earlier message endings in implicit mode. A turn that appends more than 20 blocks, or a conversation that grows past them, leaves the last written entry outside the window. That is what the second and third breakpoints are for.
Seven settings that decide the hit rate
- Order by how often each part changes. Tool definitions, then instructions, then reference material, then the conversation. Anything that changes per request belongs after everything that does not.
- Put the breakpoint at the end of the last stable block. Not on the last block. On a growing conversation the final block works, because the turn before it wrote there; on a prompt with a varying suffix it does not.
- Append, never rewrite. A summary is a new message at the end, not a smaller version of the transcript at the start. Compaction and truncation reset the prefix from the first changed token.
- Keep the tools array byte-identical. Disable a tool with
tool_choice: "none"instead of removing its definition, and never put a session id or a snapshot timestamp in a description. - Match the lifetime to the pause. A handbook reused all day wants the one-hour write at 2x. The conversation tail wants the default, because it expires between sessions anyway.
- Send related requests to the same cache. On OpenAI models before GPT-5.6 that means a stable
prompt_cache_key, aimed at roughly 15 requests per minute per key. Through a router it means a prefix that does not change, so the routing key does not change with it. - Read the counter.
cache_read_input_tokens,cached_tokens,prompt_cache_hit_tokens,cachedContentTokenCount. Each provider reports the tokens that came from cache on that request. A hit rate near zero on a prompt that looks static means the prefix is changing somewhere you have not looked.
What caching does not buy
Cached input tokens still count toward rate limits on OpenAI, so caching will not rescue a loop that is throttled. A cache entry on Anthropic only becomes available after the first response starts, so parallel requests that all arrive at once cannot help each other. Hidden system content does not count toward the minimum prefix on either provider. Caches are isolated per organization and, on the Claude API, per workspace, so two copies of the same prompt in two workspaces are two writes. And Gemini charges storage for an explicit cache by the token-hour, which is a second meter that a 24-hour cache can run up.
turns in the loop
None of this changes the shape of the cost curve. Caching divides the bill by five or so and leaves the growth alone, because the transcript still grows and you still resend it every turn. On the forty-turn loop priced in the previous article, caching moved $2.16 to $0.43 and both numbers kept climbing with the square of the turn count.
Order the prompt by how often each part changes, put the breakpoint at the end of the part that does not, and measure the counter on the next request. If the hit rate is near zero and the prompt looks static, the prefix is changing in the tools array, the first line of the system prompt, or somewhere above the block you meant to cache.