Metadata-Version: 2.5
Name: kvleak
Version: 0.1.0
Summary: Find where your AI coding agent leaks prompt-cache money (Claude Code, multi-agent, any JSONL) and how to stop it.
Project-URL: Homepage, https://github.com/shangliy/kvleak
Project-URL: Issues, https://github.com/shangliy/kvleak/issues
Author: kvleak contributors
License-Expression: MIT
License-File: LICENSE
Keywords: claude-code,cost,kv-cache,llm,multi-agent,observability,prompt-caching,tokens
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Utilities
Requires-Python: >=3.9
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Description-Content-Type: text/markdown

# kvleak

**Find out where your AI coding agent leaks prompt-cache money, and how to stop it.**  ·  [中文](README.zh-CN.md)

[![CI](https://github.com/shangliy/kvleak/actions/workflows/ci.yml/badge.svg)](https://github.com/shangliy/kvleak/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/kvleak)](https://pypi.org/project/kvleak/)
![Python](https://img.shields.io/badge/python-3.9%2B-blue)
![Dependencies](https://img.shields.io/badge/dependencies-0-brightgreen)
![License](https://img.shields.io/badge/license-MIT-green)

> **A 99.0% cache hit rate, and the session still leaked $82.** On Claude Fable 5.1 one
> cache miss costs as much as 80 hits. In a 30-day audit of real Claude Code use, 2% of calls
> carried more than half of all uncached input. [See the numbers →](docs/benchmarks.md)

Your agent probably reports a 90%+ cache hit rate, and you are still overpaying.
A cache miss costs about **12× more than a hit**, and the misses are rare but huge.
They come from idle gaps that outlive the 5-minute TTL, from model switches, and
from subagents that start cold. In real audits, **about 5% of calls burn about
half of all uncached input**.

`kvleak` reads the transcripts that are already on your disk, finds every
expensive miss, says *why* it happened, and puts a dollar figure on it.

```bash
uvx kvleak          # or: pipx run kvleak  ·  pip install kvleak
```

No API key, no config, no network calls, no dependencies. It reads
`~/.claude/projects` by default and any JSONL with token usage via `--path`.

```text
 kvleak · prompt-cache leak report
 14 sessions + 18 subagents · 1,594 model calls · 10.0k min miss size

  Cache hit rate    92.1%  ██████████████████████
  Input spend       $145.09   caching already saved you $791.64
  Leaked to misses   $69.54   47.9% of input spend went to cache rebuilds and cold starts
  Recoverable        $27.00   with break-even-capped keep-alive pings alone

 Where the leak comes from
  cause                            misses  uncached    leak  share
  Idle gap outlived the cache TTL      48     10.0M  $38.40  55.2%  ████████▊
  Model switched mid-session           21      5.2M  $18.34  26.4%  ████▏
  Prefix changed within the TTL        16      2.6M  $11.14  16.0%  ██▌
  New session cold start               14      336k   $1.61   2.3%  ▎
  Subagent cold start                  18      468k  $0.054   0.1%

 What to do about it
  1. Idle gaps are your biggest leak class. Reply before the TTL ring empties, ...
  2. Switching models throws away the whole cache (KV state is per model). ...
```

<sub>Output from the bundled synthetic demo: `python examples/make_demo.py demo && kvleak --path demo`</sub>

---

## Real numbers

| | |
|---|---|
| 30 days of interactive Claude Code sessions | **97.5%** hit rate, but **27.5%** of input spend went to cache rebuilds |
| Share of that leak caused by idle gaps | **82%** (median gap 2.8 h; Claude Code now uses the 1 h TTL) |
| Cost of one miss on a 200k context | **$3.95** on Fable 5.1, $1.56 on Opus 5.5 (1 h TTL) |
| 6-worker swarm, pointers + warm workers + pinned models | **−86%** input cost in simulation |

Methodology and reproduction commands: [docs/benchmarks.md](docs/benchmarks.md).

<p align="center"><img src="https://raw.githubusercontent.com/shangliy/kvleak/main/docs/assets/report.png" width="720" alt="kvleak HTML report"></p>

## What you get

| Command | What it does |
|---|---|
| `kvleak` | Audits every session: hit rate, dollars leaked, leak broken down by cause, leakiest sessions, worst single misses, and concrete next steps. |
| `kvleak --since 7d --html report.html` | A self-contained HTML report (light/dark, no JS) you can share with your team. `--markdown` and `--json` also work. |
| `kvleak session <id>` | Call-by-call timeline of one session, with each miss flagged and explained. |
| `kvleak statusline --install` | Adds a live **cache TTL countdown** to Claude Code: `◕ cache 3:41 · 182k ctx · rebuild $0.89`. |
| `kvleak breakeven 150k --idle 20m` | Tells you whether to keep this cache alive or let it die. |
| `kvleak pointer build.log --status FAIL` | Turns a huge file into a ~60-token hand-off message for multi-agent boards. |
| `kvleak --fail-over 15` | Exits with code 2 when more than 15% of input spend leaked, so it can gate CI or nightly jobs. |

### Every miss gets a cause

| Cause | What happened | Typical fix |
|---|---|---|
| **Idle gap outlived the TTL** | You were away for more than 5 min (or 1 h), so the next message rebuilt everything. | Watch the statusline countdown, batch work, or use capped keep-alive pings. |
| **Prefix changed within the TTL** | Something near the top of the prompt changed: a timestamp, unsorted JSON, or a different tool list. | Make the prefix deterministic. |
| **Model switched mid-session** | KV caches are per model, so a switch always means a full-price rebuild. | Pin one model per session or worker and hand off with pointers. |
| **History compacted** | `/compact` or auto-compact rewrote the history. | Compact once at a natural break, not mid-task. |
| **Cold start / subagent cold start** | A fresh agent paid full price for its system prompt and tool schemas. | Reuse warm workers; keep system prompts identical. |

Only the **avoidable** part counts as leak: the context the cache already held,
priced at write cost minus read cost. A new user message or new tool output
would be paid for anyway, so it is not counted. Details:
[docs/how-it-works.md](docs/how-it-works.md).

## Live TTL countdown in Claude Code

```bash
kvleak statusline --install      # backs up settings.json; refuses to overwrite an existing statusLine without --force
```

```text
◕ cache 3:41 · 182k ctx · rebuild $0.89        (green → yellow → red as the TTL drains)
○ cache cold · next msg rebuilds 182k ($0.89)
```

When the ring is almost empty, sending your next message *now* refreshes the
TTL at the cost of a cache read. If you already have a statusline script, call
`kvleak statusline` from it; it reads the same JSON on stdin.

## Should I keep this cache alive?

```text
$ kvleak breakeven 100k --model claude-sonnet-5 --idle 20m

  Cached prefix 100k on claude-sonnet-5 (5m TTL)
  Rebuild if it expires   125k tok-equiv  ($0.250)
  ...of which avoidable   115k tok-equiv  ($0.230)
  One keep-alive ping     10.0k tok-equiv  ($0.020)
  Break-even              11 pings ≈ 54m30s of idle time

  Idle 20m00s: 4 pings cost 40.0k tok-equiv  ($0.080), saving 75.0k tok-equiv  ($0.150). Keep it alive.
```

The rule is `K_max = floor((write − read) / read)`: 11 pings at 0.10× reads,
24 at 0.05×, 49 at 0.025×. Past that, let the cache expire.

## Running multi-agent swarms?

Multi-agent and multi-model setups multiply leaks in three ways:

* caches never cross models
* shared boards make every agent re-read everyone's logs, which costs O(N²)
* one-shot subagents each pay a cold-start tax

**[The multi-agent cache playbook](docs/playbook.md)** describes six layers of
practice: pinned models with pointer hand-offs, a script → small model → large
model funnel, warm workers with TTL batching, a slim board contract,
deterministic prefixes, and deliberate compaction. It includes a contract you
can paste straight into your swarm's instructions, and `kvleak` measures each
layer.

```bash
$ kvleak pointer /tmp/build.log --status FAIL --note "3 tests failing in //pkg"
{"bytes":40211,"lines":512,"note":"3 tests failing in //pkg","path":"/tmp/build.log","sha256":"9f2c1e07a1b4c3d2","status":"FAIL","tail":[...]}
```

## Supported logs

| Source | How |
|---|---|
| **Claude Code** (incl. subagents) | automatic: `~/.claude/projects`, respects `$CLAUDE_CONFIG_DIR` |
| **Any JSONL with usage counters** | `--path <dir-or-file>`. Auto-detects Anthropic (`cache_read_input_tokens`), OpenAI (`prompt_tokens_details.cached_tokens`), Gemini (`cachedContentTokenCount`) and flat agent-framework keys (`input_tokens`, `cache_read_tokens`, `created_at`). |

Prices for current Claude models are built in (`kvleak models`). For other
providers, add a pricing file; see [`examples/pricing.json`](examples/pricing.json).
Models without a price still get full token-level analysis.

Want your agent harness supported natively? A reader is one small function
that yields `Step`s. See `src/kvleak/sources.py` and [CONTRIBUTING.md](CONTRIBUTING.md).

## Use it as a library

```python
from kvleak import analyze, load

report = analyze([load("session.jsonl")])
print(report.hit_rate, report.leak_usd, report.buckets["ttl_expiry"])
```

## FAQ

**Is my data sent anywhere?** No. kvleak only reads local files and makes no
network calls.

**I'm on a subscription, not the API. Does this matter?** Yes. Cache misses
use up your rate-limit headroom the same way they would cost money on the API.
Read the dollar figures as relative weights.

**My hit rate is 95%. Am I fine?** Maybe not. Hit rate counts tokens, not
dollars, and one 800k-token rebuild weighs as much as thousands of cheap hits.
Look at the *Leaked* line instead.

**How accurate is the cause detection?** It follows the rules in
[docs/how-it-works.md](docs/how-it-works.md). The main blind spot is very long
generations: timestamps are written when a response ends, so a few TTL
expiries can show up as prefix busts.

## Credits

Inspired by the cache-countdown idea in
[olddonkey/cache-refresher](https://github.com/olddonkey/cache-refresher) and by
production audits of multi-agent swarms.

MIT licensed. Contributions welcome!
