Distil — Compression with a quality contract

How it works — a cost-optimized cache hierarchy with a contract

Most compressors optimize bytes. In an agent loop the money is somewhere else.

Two highest-leverage techniques — and why they win

TECHNIQUE #1

Cache-aware compression

You re-send the growing context every turn. With prompt caching a cache read is ~10× cheaper than fresh input — so the dominant cost is cache misses, not context size. Distil keeps the prefix byte-stable and compresses only the volatile tail. Naive recompression sends fewer tokens yet costs more, because it rewrites the cached prefix every turn.

TECHNIQUE #4

Causal / counterfactual pruning

The eval engine isn't a ruler — it's a discovery engine. Remove a context block, replay, did any decision change? Blocks that never change a decision are provably free to drop: speculative retrievals, stale history. The measurement produces the compression policy.

The reframe that makes “100%” real. Byte-equivalence and high compression are information-theoretically in tension. Decision-equivalence is the right target: the agent takes the same actions and produces the same outputs whether or not the context was compressed. That is measurable and certifiable — “100%” becomes a statistical non-inferiority guarantee on outcomes, not a string diff.

The proof — a quality contract, not an estimate

We didn't just benchmark ourselves. We ran the real LLMLingua-2 and Headroom packages through the same gate, graded live by the real model.

Live head-to-head: Distil vs LLMLingua-2 vs Headroom

Distil is the only method that is simultaneously the most aggressive, fully decision-equivalent, and the lowest-latency. Certified 83.2% token savings at a 0% live decision-change rate (≤5% at 95% confidence) — while LLMLingua-2 cuts 53% but flips 1-in-5 decisions, and Headroom is safe but 2.4× less aggressive. See the full live benchmark →

Underneath it: a strategy ships only if a pre-registered non-inferiority test certifies it. Lossless passes; quality-degrading compression is rejected.

$ distil savings --pricing claude-opus-4-8
strategy                               $ / run   vs baseline  cache hits
------------------------------------------------------------------------
baseline (no cache, no compress)       0.01524          0.0%           0
cache only                             0.01115         26.8%       1,028
naive compress + cache                 0.01691        -11.0%           0   ← busts cache
distil (cache-aware lossless)          0.01019         33.1%       1,028

$ distil certify --strategy distil
decision-equivalence match rate: 100.0%
VERDICT: PASS  (certified non-inferior)

$ distil certify --strategy aggressive
decision-equivalence match rate: 0.0%
VERDICT: FAIL  (would degrade quality — blocked)

Measured across 7 domains — the same gate, not one example

A strategy isn't trustworthy because it works once. distil bench certifies every domain — ops, coding, support, research, data, devops, finance — or it doesn't ship.

Risk-graded tiers

Apply the provably-safe ones everywhere; gate the rest on evidence.

Tier 0

Provably lossless

Reconstructable transforms — JSON minify, reversible run-length encoding. Always on.

Tier 1

Reversible digest

Decision-aware digest + a handle; the full original stays local and re-expands on demand.

Certified

Lossy, but gated

Pruning & summarization allowed only at ratios the non-inferiority gate certifies.

Capabilities — every layer of the cost stack

Capability	What it does	Loss profile
Priced cache-aware engine	Models the multi-turn loop and proves naive recompression busts the cache	—
Schema canonicalization	Recursively key-sorts JSON/tool payloads so the prefix is byte-stable	lossless
Volatile-field extraction	Lifts dates/UUIDs/JWTs out of the cached prefix so it stops churning	lossless · reversible
Reversible digest + handles	Originals stay local and re-expand on demand	reversible
Reject-if-bigger invariant	Never emits a block larger than its original	safety
Causal / counterfactual pruning	Ablation discovers context that never changes a decision	certified
TOST non-inferiority gate	A strategy ships only if it passes the quality contract	the moat
Savings ledger + leaderboard	Local-first, privacy-preserving cumulative savings	opt-in

Works with every SDK — one proxy, no code change

Point any base_url-honoring client at the proxy — Python, TypeScript, any language — and get cache-aware lossless compression. See integrations →

Install — pick a format

The stdlib-only core makes the packaging clean: zero runtime deps, corpus bundled.

ZERO INSTALL

uvx

uvx --from distil-llm distil bench

ISOLATED CLI

pipx

pipx install distil-llm
distil certify

HOMEBREW

brew

brew install dshakes/tap/distil
distil bench

CONTAINER

Docker

docker build -t distil .
docker run distil bench

SINGLE FILE

zipapp

make pyz
python dist/distil.pyz bench

Needs Python 3.11+. Install isolated (pipx/uv/brew/Docker) — system pip install is blocked by PEP 668 on modern macOS/Linux. Node/other stacks: point your SDK baseURL at distil proxy.

Drop-in, no call-site change:

from distil.adapters.anthropic import wrap
client = wrap(anthropic.Anthropic())  # compresses + keeps the cache warm

What we won't pretend

A headline like “87% reduction, 100% accuracy” is unverified marketing until you see the eval. Published quality numbers in this space are typically measured at far lower compression than the headline workload ratios — so the two figures don't describe the same run. Distil's answer isn't a bigger number; it's a gate: the accuracy claim and the compression are measured on the same trajectories across 7 domains, and a strategy that can't pass non-inferiority doesn't ship. The default tokenizer is an offline heuristic (ratios robust, dollars approximate); --tokenizer anthropic and --runner anthropic make it billing-grade against the real model.