Prompt caching · field report

Your cache hit rate is 99%. You're still leaking money.

I audited 30 days of my own Claude Code transcripts. One session hit the cache 99.0% of the time and still burned $82 on rebuilds. Here's where the money goes, and the small tool I built to stop it.

Cache time-to-live draining 44:12 cache ttl
◕ cache 44:12 · 452k ctx · rebuild $9.04 · ⚠ /compact saves ~$84

A 452k-token Fable 5.1 session. Let the ring run out and the next message costs $8.92 extra. One message before it does costs $0.11.

Prompt caching is the best deal in AI right now. Send the same long prefix twice and the second time you pay a tenth of the price, or a fortieth on the newest models. Every agent harness does it, and every dashboard shows a reassuring number: cache hit rate, 92%.

That number counts tokens. Your bill counts something else. A cache hit is cheap, but a miss rebuilds the whole context at the write price, and in a long agent session the context is hundreds of thousands of tokens. A handful of misses can outweigh thousands of hits.

So I wrote a small auditor, kvleak, pointed it at the transcripts Claude Code already keeps on disk, and looked at what a month of my own work actually cost.

97.5%cache hit rate, interactive sessions
27.5%of input spend went to rebuilds
2.2%of calls carried 54% of uncached input
−58%input spend with auto-compact at 200k

1 · The asymmetry

A miss costs 12 to 80 hits

Caching is priced on two sides. Reading a cached prefix costs about 0.1× the input price (0.05× on Opus 5.5, 0.025× on Fable 5.1). Writing it costs 1.25× with a 5-minute lifetime, or 2× with a 1-hour lifetime. Every read also resets the timer for free.

Cheaper reads make each miss hurt more by comparison. Here is one rebuild of a 200k-token context against one read of it, with a 1-hour cache:

one cache hitavoidable cost of one miss
$0$1$2$3$4 Fable 5.1 $3.95 · 80× Opus 5.5 $1.56 · 40× Sonnet 5.5 $0.76 · 20× Haiku 5.5 $0.04 · 20×
Avoidable cost = 200k × (write − read) at list price; 1-hour TTL. Source: kvleak models, kvleak breakeven.
Hit rate is a vanity metric. Leaked dollars are the real one.

2 · Where it went

Mostly, I walked away

kvleak finds every expensive miss and labels why it happened. Over 30 days of interactive sessions, $418 leaked. This is where it went:

idle gap outlived the TTL · 82.4% subagent cold start 6.5% prefix changed 4.8% model switch 4.7% · compaction + cold 1.6%
30 days, 289 interactive transcripts (257 of them subagents), 13,110 calls. Leak = rebuilt context × (write − read).

The surprise was the length of the gaps. Claude Code now writes about 90% of its cache with the 1-hour lifetime, so the 5-minute expiry everyone worries about barely matters. The expensive misses were lunch, meetings, and overnight: the median gap before an expiry was 2.8 hours.

One example: a Fable 5.1 session carrying 452k tokens sat idle for 1 hour 24 minutes. The next message rebuilt it for $8.92. One message at minute 59 would have reset the timer for $0.11.

max worthwhile pings K = floor((write − read) / read) 11 at 0.10× reads · 24 at 0.05× · 49 at 0.025×

That's what the ring at the top of this page is. kvleak statusline --install puts it in Claude Code's status bar, so you can see the cache dying and decide whether to save it.

3 · The bigger surprise

A big context costs you on every call, even when it hits

Then I looked at the hits. A cache read is cheap per token, but a 450k-token context is a lot of tokens. Carrying it costs a read on every call, plus a small chance that the call misses and rebuilds everything. On Opus 5.5 that adds up to about $0.17 per call, before the model writes a single word.

Compacting, or handing off to a fresh session with a short doc, costs money once and makes every later call cheaper. The break-even is simple enough to compute live:

R* = one-time cost / ((C − S) × (read + miss_rate × (write − read)))

For a 450k context on Opus 5.5, compacting costs about $0.31 and pays for itself after 2 calls. kvleak advise does this math for your current session. It estimates how many calls you have left from how long your own sessions usually run.

To check it honestly, I replayed my history with the advisor in the loop. At each call it could only see the past. When it fired, every later call was re-priced with the smaller context, at what that call actually paid:

compact only abovecompactions in 30 daysinput spendsaved
100k tokens334$1,297$832 · 66.7%
200k tokens98 (≈3 per session)$1,297$721 · 57.7%
400k tokens23$1,297$487 · 40.2%
Source: kvleak advise --backtest on 33 main sessions. Money only: it assumes the summary keeps what the rest of the session needs.

You don't have to watch for the hint, either. Claude Code can compact for you at a context size you choose (autoCompactWindow), and it can pick the cache TTL and compact idle sessions before the cache dies. The defaults are tuned for nobody in particular. kvleak tune replays your own history under each value and --apply writes the winners to settings.json. On my 30 days: −$909, 45% of input spend, most of it from auto-compacting at 200k instead of ~967k, the rest from a 5-minute TTL on one-shot claude -p batches whose prompts are written at 2× and never read back.

This counts money only. A summary can drop a detail you needed later, and that cost doesn't show up here. Compact at natural breaks: after a feature lands, before switching tasks. Even the cautious row, ten compactions in a month, cuts the bill by almost two thirds.

Try the numbers

What does your context cost?

450k
one cache hit$0.090
one miss, avoidable part$3.51
carrying it, per call$0.17
compaction pays off after2.0 calls
verdict

Compact at the next natural break.

Assumes a 2.2% per-call miss rate (measured), an 8k-token summary, 25k of cached system prompt, and output priced at 5× input. Same formulas as kvleak advise.

4 · Many agents, many models

Swarms multiply every leak

Everything above gets worse with several agents. Three effects stack:

  • Caches don't cross models. KV state belongs to the weights that made it. Hand a 130k context from one model to another and it's a full-price rebuild, every time.
  • Shared boards cost O(N²). Paste one 500-line build log on a board six agents read, and all six carry it on every later turn.
  • One-shot subagents pay a cold-start tax. My subagents averaged 28k tokens of full-price prefill before doing anything.
naivepaste logs, respawn, switch $71.10 + pointers on the board $27.37 · −62% + warm workers $23.18 · −67% + one model per worker $9.63 · −86%
Simulation, not measurement: 6 workers × 40 rounds, Opus 5.5 prices. Run python examples/swarm_sim.py and change the constants.

The fix for the board is boring and huge: write the log to disk and post a pointer. kvleak pointer build.log --status FAIL prints about 60 tokens of JSON (path, hash, line count, last lines), and only the agent fixing the build opens the file. The playbook has the full six-layer version, plus a contract you can paste straight into your swarm's instructions.

5 · Try it on your own logs

Three seconds, no API key

kvleak reads the transcripts already on your disk. It has no dependencies, makes no network calls, and doesn't ask for config. It covers Claude Code out of the box, including subagents, and any JSONL with token usage: OpenAI-style, Gemini-style and most agent frameworks.

uvx kvleak
$ kvleak --since 30d
  Cache hit rate    97.5%  ████████████████████████
  Input spend       $1,522   caching already saved you $16,563
  Leaked to misses    $418  27.5% of input spend went to cache rebuilds
  Recoverable          $90  with keep-alive pings (idle ≤ 2h)

$ kvleak advise
  Context now 207k · carrying it costs $0.077 per call
  → Write a short hand-off doc and start a fresh session (pays off within 4 calls).
  1. kvleak gives the full audit: hit rate, dollars leaked, causes, worst sessions, and what to do about them.
  2. kvleak statusline --install adds the live TTL ring and the ⚠ /compact hint to Claude Code.
  3. kvleak advise tells you whether to compact or hand off right now, and --backtest checks the advice against your own history.
  4. kvleak --fail-over 15 fails a CI job when more than 15% of its input spend leaked.

About these numbers: this is one heavy user's month at list API prices. That's a single data point, which is why the tool exists. Run it and you'll have your own in three seconds.