Metadata-Version: 2.5
Name: promptcachelint
Version: 0.1.0
Summary: Explain why your LLM prompt cache missed: prefix diffs, silent invalidators and hit-rate accounting for Anthropic and OpenAI.
Project-URL: Homepage, https://github.com/OsmnvAslan/promptcachelint
Project-URL: Repository, https://github.com/OsmnvAslan/promptcachelint
Project-URL: Issues, https://github.com/OsmnvAslan/promptcachelint/issues
Project-URL: Changelog, https://github.com/OsmnvAslan/promptcachelint/blob/main/CHANGELOG.md
Author-email: Aslan Osmanov <osmnvaslan@gmail.com>
License-Expression: MIT
License-File: LICENSE
Keywords: anthropic,cache,cost,llm,observability,openai,prompt-caching,tokens
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Typing :: Typed
Requires-Python: >=3.11
Provides-Extra: httpx2
Requires-Dist: httpx2>=2.0; extra == 'httpx2'
Description-Content-Type: text/markdown

# promptcachelint

Explain why your LLM prompt cache missed.

Prompt caching on Anthropic and OpenAI is a prefix match: one changed byte
anywhere before the cache marker and everything after it is billed again at
full price. The providers tell you *that* it happened (`cache_read_input_tokens: 0`)
but not *where*. `promptcachelint` records your requests, diffs each one against the
previous request of the same conversation, and names the block, the offset and
the characters that broke the prefix, plus the usual suspects it can see without
history: timestamps in the system prompt, a missing marker, a prefix below
the model's minimum.

```
promptcachelint: 3 requests in 1 session(s); prompt tokens 4880 (read 0, write 4806, uncached 74); hit ratio 0.0%; 2 prefix break(s), ~3112 tokens lost (estimate)
  (sessions grouped automatically by first message; use promptcachelint.session(id) for exact grouping)

session 600a924e2101 [anthropic] 3 req, hit ratio 0.0%, 2 break(s)
  #0  claude-opus-5  segments=2 cacheable=2  in=24 read=0 write=1601
      ! CL012 iso-datetime inside the cacheable prefix at system[0]
        → Timestamps change every request. Move them after the last breakpoint or drop them.
  #1  claude-opus-5  segments=4 cacheable=4  in=25 read=0 write=1602
      ✗ prefix broken at system[0] +6190 (~1553 tokens lost, estimate)
        was: '…ime: 2026-09-20T10:15'
        now: '…ime: 2026-09-20T10:16'
  #2  claude-opus-5  segments=6 cacheable=6  in=25 read=0 write=1603
      ✗ prefix broken at system[0] +6190 (~1559 tokens lost, estimate)
        was: '…ime: 2026-09-20T10:16'
        now: '…ime: 2026-09-20T10:17'
```

Zero dependencies. Python 3.11+. Anthropic Messages API, OpenAI Chat Completions
and Responses (with the full history in the request; when `previous_response_id`
keeps the conversation server-side, the request body holds only the new turn and
there is no prefix to analyse).

## Install

```bash
pip install promptcachelint            # core: explicit recording, analysis, CLI
pip install "promptcachelint[httpx2]"  # + transport that records SDK traffic automatically
```

## Quick start

The SDKs accept an `http_client`. Keep their own client class (it carries the
SDK's connection limits and timeouts) and wrap only the transport:

```python
import anthropic, httpx2
import promptcachelint
from promptcachelint.transport import wrap_transport

recorder = promptcachelint.Recorder(
    promptcachelint.LogWatcher(),               # warn in the log the moment a prefix breaks
    promptcachelint.JsonlSink("trace.jsonl"),   # keep a trace for the offline report
)
claude = anthropic.Anthropic(
    http_client=anthropic.DefaultHttpxClient(
        transport=wrap_transport(httpx2.HTTPTransport(), recorder)
    )
)
# openai: openai.OpenAI(http_client=openai.DefaultHttpxClient(transport=wrap_transport(...)))
# async:  anthropic.DefaultAsyncHttpxClient(transport=wrap_transport(httpx2.AsyncHTTPTransport(), recorder))

# ... run your agent as usual ...

print(promptcachelint.analyze(recorder.records).to_text())
```

Then, or on a trace from production:

```bash
promptcachelint report trace.jsonl              # human-readable; broken lines are skipped and counted
promptcachelint report trace.jsonl --json       # machine-readable
promptcachelint report trace.jsonl --fail-on CL001,CL002   # non-zero exit for CI
promptcachelint lint request.json --provider anthropic     # static checks on one body
promptcachelint codes
```

No SDK, or a gateway of your own? Record explicitly:

```python
recorder.record("anthropic", body, response=response_json)          # non-streaming
recorder.record("anthropic", body, sse_events=events, stream=True)  # streaming
```

## What it reports

| Code | Meaning |
|---|---|
| `CL001` | Content inside the previously cached prefix changed. Block, offset, before/after excerpt, estimated tokens lost. **Confirmed** by the diff. |
| `CL002` | Tool definitions changed between requests (tools render first: everything is invalidated). |
| `CL003` | A top-level parameter changed: `model`, `thinking`, `output_config.effort`, `tool_choice`, `speed`, `prompt_cache_key`. |
| `CL004` | The last breakpoint moved earlier than in the previous request. |
| `CL005` | Prefix (or a marked tools+system head shared across requests) was identical and within TTL, but the provider read nothing. Suspects outside the payload. Informational on OpenAI, whose cache is best-effort. |
| `CL006` | The gap since the previous request exceeds the cache TTL (Anthropic: the longest marker's, 5 min or 1 h; OpenAI about 10 min). |
| `CL007` | More than 20 positions appended in one turn; Anthropic's breakpoint lookback window is 20 (a run of parallel `tool_use` blocks is one position, as is a run of `tool_result` blocks). |
| `CL010` | No `cache_control` anywhere on a prompt large enough to cache. |
| `CL011` | Cacheable prefix below the model's minimum (512 to 4096 tokens depending on model). Estimate. |
| `CL012` | Timestamp / UUID / random-looking id / "today is" inside the cacheable prefix. A **suspicion** from patterns; `CL001` is the confirmation. |
| `CL013` | More than 4 `cache_control` slots (explicit markers plus the top-level automatic one). |
| `CL014` | Requests with the same, unmarked tools+system write cache and never read it: the only marker sits after per-request content. |

## What is exact and what is estimated

The diff runs on a canonical rendering of the request: each tool, system block
and message block becomes one segment; text blocks keep their text as is,
structured blocks are serialized as JSON with sorted keys, and cache markers are
stripped (the marker moving forward is not an invalidator). Offsets are
character offsets into that block's own text, so they point at the character
you wrote. Key order inside JSON blocks is *not* compared, because the provider
parses the JSON before hashing.

Costs come from the provider's own `usage` fields and are exact. Only "tokens
that could have been read" is an estimate (about 4 characters per token) and is
labelled as such everywhere.

## Sessions

Consecutive requests are diffed within a *session*. Two modes:

* **Explicit**: `with promptcachelint.session("conv-42"):` around the calls, or
  `session_id=` on `recorder.record`. Reliable; use it in production.
* **Automatic** (default): a request continues the session whose **first message
  block** it repeats byte for byte, when its message history is the previous
  history plus new turns (or an exact retry). Growth of the history is the
  signature of a continuation, so a tool reorder or a rewritten system prompt
  mid-conversation stays in the session and is reported. Interleaved
  conversations that share a system prompt stay apart, and so do two users who
  both open with "hi" from their second turn on. Two rewrites that would
  otherwise look like new conversations are recognised and reported as prefix
  breaks with their own hint: a **sliding history window** (oldest turns
  dropped; detected up to 16 blocks deep) and an **edited history** (old tool
  results truncated, a turn summarised; detected when most of the previous
  blocks are still there in order). The report says when grouping was automatic;
  explicit sessions remain the reliable choice in production.

## Privacy and cost

A trace contains full prompts. Before it leaves the machine, pass a redactor to
the sink:

```python
from promptcachelint.redact import strip_media, hash_text
promptcachelint.JsonlSink("trace.jsonl", redact=strip_media)  # drop base64 images/PDFs, keep text
promptcachelint.JsonlSink("trace.jsonl", redact=hash_text)    # keep structure and sizes only
```

With `hash_text` every text becomes its SHA-256 digest repeated or cut to the
same length in characters, so token estimates and size-based findings stay
right, different texts stay different, and the report still says *which block*
changed and by how much, but offsets and excerpts point into placeholders.

**Memory.** `Recorder` keeps the last 10 000 records by default (`keep=`; `None`
for unlimited, `0` for a pure live setup that only feeds sinks). `LogWatcher`
runs the analyzer without history: one request per session is remembered for
the diff, and sessions idle for six hours are dropped, so a long-running process
holds memory proportional to its active conversations, not to its lifetime
traffic. The offline report (`analyze`, `promptcachelint report`) keeps everything for
the trace it is given.

**CPU.** Recording runs on the caller's thread right after the response body
is consumed: one pass over the request body plus an O(1) session lookup.
`Recorder` is thread-safe. On a latency-critical async path with very large
prompts, record from a worker instead of the transport.

## In tests

```python
from promptcachelint.testing import assert_cache_stable

def test_agent_keeps_its_cache(recorder):
    run_agent(http_client=...)
    assert_cache_stable(recorder.records)   # fails with the rendered report on CL001/002/010/013
```

## Custom gateways

Register an adapter for a proxy that speaks a known dialect on another host:

```python
from promptcachelint.providers import AnthropicProvider, register_provider

class Gateway(AnthropicProvider):
    name = "gateway"
    def matches(self, url: str) -> bool:
        return "llm.internal" in url

register_provider(Gateway())
```

## Not in scope (yet)

Rewriting requests or placing markers for you, Gemini, semantic caches, a proxy
server. The diagnosis has to be right before the fix can be automatic.

## License

MIT
