The support agent answers every ticket correctly. It has a six-thousand-word policy handbook in its prompt, five tool definitions, and the growing transcript of the conversation. Prompt caching is on, which is the part the team checked when the invoices started climbing.
Twenty-four turns across three tickets send 164,854 tokens and cost $0.4841. The same loop with caching switched off costs $0.4017. The feature is not just failing to help; it is charging a premium of 21% for the privilege.
This is lab 6 of inva-lab, and it runs with python3 alone: no Docker, no database, no key, no network. The provider is a simulation of one vendor's documented rules, published in November 2026, so the numbers are reproducible and free.
Key takeaways
- Caching is on and the hit rate is 0.0%. 164,854 tokens sent across twenty-four turns, $0.4841, which is 21% more than the same loop with caching disabled at $0.4017.
- Three lines break the prefix, and all three look reasonable in review. A registry timestamp inside the tool definitions, a clock inside the system prompt, and a breakpoint parked on the block that changes every turn.
- The mission is
make check: seven checks, none green at the initial state, and the fixed version reaches 90.9% with a bill at 34% of the no-cache run. - The lab runs offline. Python 3, no API key, no network: the provider is a deterministic simulation of one vendor's published caching rules.
What you see when you run it
| command | what appears |
|---|---|
make demo |
hit= 0.0% $0.4841 for all three tickets |
make what-changes |
tools DIFFERENT, block [0] CHANGED, breakpoint moved from block [1] to [3] |
make check |
0 of 7 checks green |
the same loop with cache_enabled=False |
$0.4017 |
Nothing looks broken. Every ticket is answered, the loop exits 0, and the provider reports no error. The bill is the only symptom, and make bench turns it into a per-turn table where write equals sent on every single row: the whole prompt is written again, every turn, and nothing is ever read.
Why a prompt that looks static is not
Three things change between two consecutive requests, and each one is defensible in a code review.
The tool definitions carry a notes field with the registry snapshot time. It is there for debugging, it changes every turn, and the provider hashes the tools before it hashes anything else, so the handbook below it is new text every time.
The system prompt opens with the date and time. Telling the agent what day it is seems obviously useful, and the model does use it; the cache does not.
The breakpoint sits on the last message. That is the block that changes every turn. The provider's documentation names it as the mistake that writes a fresh entry and never reads one: the lookup walks backward looking for entries earlier requests wrote, and nothing was ever written at the end of the handbook.
There is a fourth, quieter one in the loop: every third turn the transcript is replaced by a one-line summary to keep the context small. It does keep the context small. It also guarantees the prefix never survives, because a rewritten transcript is not a prefix.
Your mission
Make make check green. Seven checks, one theme: the cache follows the bytes.
- At least 75% of the tokens you send come back as cache reads, and the run costs at most 45% of the same loop with caching off.
- The model still gets what it needs. The date has to reach the model, the tools array has to be byte-identical on every request, and every request has to be the previous one plus new blocks rather than a rewrite.
- The loop survives a ten-minute pause after turn four, and
python3 agent/loop.pyprints one line with the numbers it actually measured.
The fixed version in the repository reaches 90.9% and $0.1437, which is 34% of the same requests with caching off. The transcript is 6% longer than the broken version, because nothing is summarised away any more. The bill went down while the token count went up, and that is the whole lesson.
Why the provider is simulated
A lab that calls a real API is a lab that costs money, needs a key, and gives different numbers on your machine. This one has a fake clock, so a five-minute cache timeout and a ten-minute customer pause run in under a second, and the ledger is deterministic. The trade is explicit: agent/provider.py implements one vendor's rules, one token is one word, and the numbers are not what your account will be charged.
If you want the rules the other vendors play by, and what happens when the prefix grows from 512 tokens to 1,024 while the cache stays put, that is the companion explainer.
When you are stuck
HINTS.md has three levels, and the first one only asks you to look at two consecutive requests side by side. The answer article has the measured progression, including the two fixes that changed nothing and the one line that changed everything.