Metadata-Version: 2.5
Name: cachemux
Version: 0.1.2
Summary: One wrapper for LLM context caching across providers. Measures whether it pays, and sets it up when it does.
Project-URL: Homepage, https://github.com/soumyadeep423/cachemux
Project-URL: Repository, https://github.com/soumyadeep423/cachemux
Project-URL: Issues, https://github.com/soumyadeep423/cachemux/issues
Author-email: Soumyadeep Das <soumyadeep.das423@gmail.com>
License: MIT
License-File: LICENSE
Keywords: anthropic,cache,cost,gemini,llm,openai,prompt-caching
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.9
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=7.4; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Description-Content-Type: text/markdown

<div align="center">

# cachemux

**Stop paying full price for the same prompt tokens.**
One wrapper makes LLM context caching measurable — and makes it pay — across OpenAI, Anthropic and Gemini.

[![PyPI](https://img.shields.io/pypi/v/cachemux.svg)](https://pypi.org/project/cachemux/)
[![Tests](https://img.shields.io/badge/tests-79%20passing-brightgreen.svg)](tests/)
[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
[![Python 3.9+](https://img.shields.io/badge/python-3.9%2B-blue.svg)](pyproject.toml)
[![Dependencies](https://img.shields.io/badge/dependencies-zero-lightgrey.svg)](pyproject.toml)

</div>

---

Providers sell cached input tokens at **90% off** — and make it your job to collect. Anthropic wants breakpoints placed, OpenAI wants routing pinned, Gemini rents you cache objects that bill by the hour whether or not anyone reads them. Get it wrong and caching **costs more than not caching**: a cache written and never read is a 25% surcharge on nothing.

cachemux is one line:

```bash
pip install cachemux
```

- ⚡ **Drop-in.** Wrap your existing SDK client; change nothing else.
- 💰 **Pays or it doesn't act.** Every caching decision clears a measured break-even (a 1.25× write needs 1.28 reads) before a single request is touched.
- 🔍 **Names the culprit.** Not "hit rate dropped" — *"the timestamp in `system` voids ~7,500 cacheable tokens per request."*
- 🛡️ **Never breaks a call.** cachemux failing means your request goes out exactly as written. There's a test for it.
- 📦 **Zero dependencies.** ~26 KB wheel, nothing dragged in.

```python
from anthropic import Anthropic
from cachemux import cachemux, Recorder

recorder = Recorder()
client = cachemux(Anthropic(), recorder)   # the only change to your code

# ... use the client exactly as before ...

print(recorder.report(base_input_per_mtok=5.0))
```

```text
cachemux report
  calls           : 12 (0 errored)
  hit rate        : 91.7%
  input tokens    : 480
  cache reads     : 23,100
  cache writes    : 2,100
  auto applied    : 9 requests
  cache spend     : 0.0247
  uncached cost   : 0.1260
  net saved       : 0.1013  (saving money)

prefix: anthropic / claude-opus-5
  requests analysed : 12
  stable prefix     : 2 blocks, ~2037 tokens
  volatile blocks   :
    [2] messages[0]: 12 variants, ~12 tokens uncacheable from here

  fix [2] messages[0] first: it alone costs ~12 tokens of cacheable prefix per request.
  verdict: worth it - a write costs 1.25x, so it needs 1.28 reads; 11.0 expected
```

The first call is always the cache *write*, so short sessions read low; the rate climbs toward 100% as reuse accumulates.

Try it with no API key: `python examples/demo.py`

## Why

Every major provider caches your prompt prefix, and every one does it differently:

| Provider | How it works | Who controls it | Cost of a cache write |
|---|---|---|---|
| **Anthropic** | Inline `cache_control` breakpoints, 4 max | You | 1.25x base input (5m), 2x (1h) |
| **OpenAI** | Automatic above 1,024 tokens | Nobody | 1.25x on GPT-5.6+, free before |
| **Gemini** | Automatic, or a rented cache object | You, optionally | Rent per token-hour |

Same headline discount, three different cost models. So **"should I cache this?" has a different answer per provider**, and caching can lose money outright:

- A cache written and never read costs *more* than not caching.
- A rented Gemini cache accrues rent whether or not anyone reads it.
- Anthropic silently skips caching below the model minimum, with no error.

A hit-rate dashboard shows none of that.

## What it does

- **Measures** — normalises every provider's cache token counts into one shape, so hit rate and real cost are comparable across providers.
- **Analyses** — splits each request into hashed blocks and names the exact block stopping your prefix from caching. A timestamp in your system prompt voids everything after it; cachemux points at it.
- **Decides** — computes the break-even (how many reads a cache needs to pay for its own write, or its rent) and compares it against your *measured* reuse.
- **Acts** — the default. Once the math clears, requests get caching set up on the way out: a `cache_control` breakpoint on Anthropic, a pinned `prompt_cache_key` on OpenAI, a rented explicit cache on Gemini — rented only when it beats the implicit caching Gemini already does for free. `auto=False` to only observe.
- **Diagnoses** — when the hit rate drops, `diagnose()` compares two traffic windows and names the block that rotated the cache key.

## How it works

```text
your code ──▶ PREPARE (four gates) ──worth it──▶ ACT ──▶ SEND ──▶ provider API
                  ▲            │                          ▲           │
                  │            └──────── not yet ─────────┘           ▼
                  └─────── next request reads this ─────── OBSERVE + RECORD
```

Every request is fingerprinted; every caching decision is made from what was actually measured, never a guess. If cachemux fails at any step, the call still goes through uncached — **a caching layer must never take down an app**, and there's a test for it.

## The math, in one breath

An Anthropic cache write costs **1.25x** base input; a read costs **0.1x**:

```text
reads needed = (1.25 − 0.1) / (1 − 0.1) = 1.28 reads to break even
```

cachemux acts only when your measured reuse clears that bar. No dollar prices are bundled — the decision runs on multipliers, and you supply your own base rate for money reports. A shipped price list goes stale silently; yours doesn't.

## Design rules

1. **Zero config.** Import it and it works.
2. **If cachemux fails, the call still goes through uncached.**
3. **No new abstraction to learn.** Keep using your provider's SDK exactly as before.
4. **No runtime dependencies.**
5. **No bundled price list.**

## Layout

```text
src/cachemux/
├── providers/     one module per provider, same four functions each
│   ├── base.py        shared Usage / Applied types
│   ├── anthropic.py   breakpoints, usage fields, model minimums
│   ├── openai.py      prompt_cache_key routing
│   └── google.py      rent-or-not planning
├── analysis/      prefix.py (stability) + drift.py (what changed)
├── economics/     pricing.py (cost models, break-even)
└── core/          proxy.py (interception) + recorder.py (aggregation)
```

Adding a provider is one module in `providers/` and one line in its registry. Nothing above that layer branches on which provider it's holding.

## Status

**Working:** measurement, prefix analysis, break-even decisions, drift diagnosis, and automatic application (the default) on all three providers — live-verified against Gemini (rented explicit cache) and OpenAI (accepted `prompt_cache_key`).

**Not measured yet:** streaming. A `create(stream=True)` call returns before usage exists, so such calls are recorded but excluded from the hit rate rather than miscounted as misses.

## Development

```bash
uv venv --python 3.11
uv pip install -e ".[dev]"
pytest -q
ruff check src tests
```

## Licence

[MIT](LICENSE)
