This is the answer article. If you have not worked lab 6 and still want to try it yourself, close the page first. Everything below is the answer, with the measured numbers.
The interesting part is not the fix. It is that three of the five edits changed no number at all for two stages, and the one that did change everything is a single line nobody would flag in review.
Key takeaways
- The tools array was the single expensive line. A registry timestamp inside a tool description kept the hit rate at 0.0%; making the definitions constant took it to 78.4%.
- Moving the clock and the breakpoint alone changed nothing. Both were necessary, neither was sufficient, and the check output says so at every stage.
- The transcript rewrite was buying 10,029 tokens and costing four times the bill. The fixed loop sends 6% more tokens and pays 34% of the no-cache run.
- The one-hour write is what survives a ten-minute pause. After the pause, the turn that follows reads 6,135 tokens from cache instead of rewriting them.
Eight states, one loop
Measured with scripts/check.py, on the same machine, without touching data/, scripts/check.py or agent/provider.py:
| stage | change | hit rate | bill | green |
|---|---|---|---|---|
| 0 | as shipped | 0.0% | $0.4841 | 0/7 |
| 1 | the clock leaves the system prompt | 0.0% | $0.4840 | 0/7 |
| 2 | + a breakpoint at the end of the handbook | 0.0% | $0.4840 | 0/7 |
| 3 | + a stable tools array | 78.4% | $0.1865 | 2/7 |
| 4 | + an append-only transcript | 80.4% | $0.1857 | 4/7 |
| 5 | + the date moved into the user turn | 80.4% | $0.1860 | 5/7 |
| 6 | + a one-hour TTL for the handbook | 90.9% | $0.1437 | 6/7 |
| 7 | + the loop reports its own numbers | 90.9% | $0.1437 | 7/7 |
The same requests with caching switched off cost $0.4017 at stage 0 and $0.4218 at stage 7. The second number is bigger because the fixed loop carries its transcripts instead of summarising them.
The tools array, which is where the money was
def tools_for_request() -> list[dict]:
return TOOLSStage 3 is one line, and it takes the hit rate from nothing to most of it. The provider hashes the tool definitions before it hashes the system prompt, so a notes field holding the registry snapshot time invalidates everything below it, on every request, forever. The model never read that field. It was there for a human.
The order matters too. Removing the timestamp from the system prompt at stage 1 was correct and bought nothing, because the tools were still handing the cache a different prefix each turn. A prefix is only as stable as its least stable part, and the tools sit at the top.
The breakpoint, and the date that has to move
A breakpoint does two jobs: it writes an entry at that boundary, and it is the position the next request looks back from. Parking it on the last message means the entry is written at a boundary that the next turn extends, so the next request finds nothing unless the lookback window reaches an older write.
blocks = [Block(role="system", text=system_prompt(), cache=True, ttl=LONG_TTL)]Two breakpoints end up being the right shape: one at the end of the handbook, kept for an hour, and one at the end of the growing transcript, kept for the default five minutes. The first is what every ticket shares; the second is what saves the previous turn.
The date then has to move rather than disappear. Check C fails if the static prefix changes between two days, and fails again if the date never reaches the model, so the only fix that passes both is to stamp the turn:
history.append({"role": "user", "text": f"{text}\n\n[session date {today(provider.clock)}]"})Stage 5 is where check C turns green and the bill does not move at all, which is the point. Correctness and cost are separate gates here.
The rewrite nobody counted
Every third turn the loop replaced the transcript with a one-line summary. It saved 10,029 tokens over the run and it made every prefix after the second one useless. The fixed version sends 174,883 tokens against the original 164,854 and pays $0.1437 against $0.4841.
If you need a summary, append it as a new message at the end. The earlier blocks stay untouched, the prefix still matches, and the summary becomes part of the cached history instead of a hole in it.
The lifetime is a bill multiplier, not a detail
With a five-minute default, the customer's ten-minute silence costs one full rewrite of 6,135 tokens per ticket. With the one-hour write at 1.25x on that block only, the pause costs nothing: the turn after it reads 6,135 tokens from cache. The write premium is paid once per hour per handbook, and the handbook does not change.
Note the shape of that decision. The static block gets the expensive, long-lived write; the transcript gets the cheap, short-lived one. One TTL for the whole prompt would be wrong in both directions.
The counter is part of the fix
Check G is the last one for a reason. It runs the loop as a program and compares the line it prints with the ledger:
$ python3 agent/loop.py
summary turns=24 hit_rate=0.909 cost_usd=0.143682Without that line, stage 0 and stage 7 are indistinguishable from the outside: both answer every ticket, both exit 0, both look fine. A number that the loop reports about itself and a checker re-derives is the difference between a fix and a belief.