# Distil — cut what your coding agent costs

Distil cuts what Claude Code and other coding agents cost, and shows the real
bill, cache included. It trims the tool output, logs and history an agent
re-sends every turn, never rewrites bytes the provider still has cached, and
keeps every byte it trims recoverable through a `distil_expand` tool. On the
maintainer's own Claude Code traffic (13,191 requests, 2026-09-01..24) it saved
an estimated 10.2% of the bill, cache reads and writes priced in
(benchmarks/results/2026-09-24/live_savings_decomposition.json), before its own
expand re-queries and shadow replays were netted out; net of them the corrected
estimate is about 9%, 8.3-9.2% (docs/research/expand-undercount.md); results depend
on how much large, repetitive tool output the agent reads.

Install (one line; the PyPI package is distil-llm, the command is distil):

  uv tool install distil-llm && distil setup

Alternatives: `curl -LsSf https://dshakes.github.io/distil/install.sh | sh`
(installs uv if missing), `brew install dshakes/tap/distil`, or on Windows
`powershell -ExecutionPolicy ByPass -c "irm https://dshakes.github.io/distil/install.ps1 | iex"`.
Then `distil wrap -- claude` to run an agent through it, `distil savings` to
see what it saved, `distil doctor` if something looks off.

Beyond cost, distil also checks itself: it certifies that the agent's
decisions are statistically unchanged (non-inferiority test) rather than
assuming it. Zero runtime dependencies. Four ways to use it, and they do not
all reach the same tier:

  - Agent wrap: `distil wrap -- <agent>` for claude, codex, gemini, aider,
    opencode, qwen, goose, grok, openhands, copilot (GitHub Copilot CLI, BYOK
    mode), kimi (Kimi CLI), vibe (Mistral Vibe), kilo (Kilo Code CLI) — all of
    which route through an environment variable — plus cn (Continue CLI),
    droid (Factory Droid), omp (Oh My Pi), crush and cline (Cline CLI), which
    have no env-var contract at all and route through a config file distil
    manages for that one session and restores byte-for-byte on exit. Zero
    config either way. (openhands additionally needs its own
    --override-with-envs flag before it will read the environment; copilot
    additionally needs COPILOT_MODEL, which only you can choose.) Run
    `distil wrap --list` for every target, its mechanism, its provider wire
    shape, and the primary doc each routing contract was verified against. The
    agents distil can NOT reach are on that same list with the reason, rather
    than given a guessed variable that would route nothing while reporting
    success. One preset is itself unverified: codex speaks the Responses wire
    shape, but codex-rs reads its base URL from the `openai_base_url` config key
    and no mapping from OPENAI_BASE_URL was found, so it may route nothing.
    `distil wrap -- claude` also sets ENABLE_TOOL_SEARCH=true so Claude Code keeps
    deferring MCP tool definitions behind the proxy (unverified on a metered
    session so far); distil never rewrites the request `tools` array.
  - HTTP proxy for Anthropic Messages, OpenAI (Chat Completions + the Responses
    API), Azure OpenAI, and Google Gemini generateContent. Point any base_url
    client at it. The same provider coverage is in all three servers: the sync
    proxy, the async proxy, and the standalone gateway.
  - Python library: `from distil import compress_messages, expand_handle`.
  - TypeScript library: `compress(messages)` from the `distil-llm` npm package,
    plus `distilMiddleware()` for the Vercel AI SDK's wrapLanguageModel.

Framework hooks (in-process, no network hop): LiteLLM, LangChain, LangGraph,
Agno, Strands, AutoGen, LlamaIndex. A generic ASGI middleware (distil.integrations.asgi)
compresses provider-shaped POST bodies for apps that host their own LLM-facing
endpoint. For the LiteLLM Proxy server, distil.integrations.litellm_hook is a
pre-call hook (litellm_settings.callbacks) that compresses each request before
routing, lossless-only by default. An MCP server exposes distil_compress / distil_expand /
distil_savings to any MCP client.

IMPORTANT for anyone integrating: the reversible DIGEST tier is reached by the
proxy, wrap, and MCP paths. The in-process LIBRARIES are LOSSLESS-tier only, on
purpose — the digest mints restore handles that must share one store with the
proxy, and it is the tier the decision-equivalence certificate measures. The
TypeScript port is held byte-identical to the Python engine by a conformance
suite; where it cannot match Python's bytes it declines rather than emitting
output the certificate does not cover.

Task outcome of the shipped path, SWE-bench Lite (n=300, two runs): cost-neutral
and within about 2 points on task success (-2.0 pts, 95% CI -5.5..+1.5; -1.7 pts,
95% CI -5.1..+1.7), not statistically distinguishable; non-inferiority at the
5-point margin not shown; long sessions not yet measured. Research
configurations on SWE-bench Verified (E8-E14) are not the shipped path. On a
synthetic offline corpus: 83.2% savings at 0% decision-change rate on a head-to-head
(2026-07-05, distil 1.10.1 vs llmlingua 0.2.2 and headroom-ai 0.27.0; that run's Headroom figures are 0.27.0-era). Re-run 2026-09-04
against headroom-ai 0.37.0: distil-causal 52.9% tokens / 58.7% dollars at 100%
decision-equivalence (PASS) vs Headroom 1.7% / 2.0% / 81% (FAIL) on the warm
corpus gate; on the read->edit->re-read codebench workload Headroom takes 35.6%
tokens for +4.9% dollars.

Two reporting commands read the same content-free local records and answer
different questions. `distil dissect <session>` is backward-looking and
per-session: what happened to this one. `distil discover` is forward-looking and
cross-session: where savings are still being left on the table across the last N
sessions, as a ranked list of actions, each carrying the tokens and dollars per
week it would recover, the derivation of that number, and the one command to act
on it. `discover` prints the median and p10/p90 of per-session savings beside the
best session so a best case is never read as typical, prefers the rate the
machine itself measured over any published ratio, and stays silent where it
cannot measure.

These properties are checked rather than asserted, and each is separately
runnable:

  - Cache contract: compression is suffix-only and cache-monotonic — it may
    never rewrite bytes the provider has already cached. Enforced as an
    invariant, after shipping the bug once and measuring it at 2x the cost of
    compressing nothing.
  - Exact-quote guarantee: a block whose bytes are a verbatim slice of a file is
    never digested, so a later Edit(old_string=...) still matches. Provenance is
    read from the shell command (cat, head, sed -n), not just the tool name.
    Byte-exact quote loss on real coding traffic: 39.3% -> 16.2%.
  - Adversarial validation: `distil validate --adversarial` runs a COMA-class
    battery through the same public path the proxy uses. No keep budget is
    shared between blocks anywhere, asserted as an equality in CI. Two findings
    are published rather than smoothed over: dedup-baiting does fold the genuine
    error line (reversibility recovers it), and decoy-verdict flooding is a real,
    unmitigated denial of savings.
  - Degradation curve: `distil bench --curve` reports savings, fact recall,
    visible recall, facts lost and reversibility at every rung of the ladder,
    offline and free.
  - Paired live estimator: shadow mode replays a sampled request three times (A
    and A' on the original, B on the compressed context) and reports the paired
    difference 1{A=B} - 1{A=A'} with a bootstrap 95% CI, unclipped, so it can
    report harm. One reporting floor (50 A/B + 30 A/A) gates every surface, and
    below it every surface prints "below reporting floor" rather than a number.
    It reports per compression mode. The maintainer's 2026-09-15 sample: digest
    agreed 102/206 against a self-agreement of 113/206 (-5.3 pp, over the 5%
    budget); lossless-only 109/192 against 109/193 (about 0). Pooled, the two
    read inside the budget, which is why pooling is no longer the headline; the
    drift guard holds digest at lossless-only when its harm bound is over budget.
    Replays run hot (399 of 399, temperature not pinned), so the paired
    difference is the statistic, not the 53.0% raw agreement. Artifact:
    benchmarks/results/shadow-live-2026-09-15.json
  - Adaptive output shaping: the verbosity directive that shortens replies is
    lossy, so it runs only while that same live referee says it is safe and
    paying. `--shape-output auto` is the default on a metered session and turns
    shaping on when the paired verdict clears the reporting floor, its harm
    bound is inside the same +/-2pp budget `distil certify` uses, and the
    measured effect of compression on reply length excludes zero on the saving
    side; otherwise off, with the reason printed and recorded in the session
    manifest. The gate reads only shadow rows measured with shaping off (each
    row is tagged with its active levers), so shaping cannot keep itself on with
    its own shorter replies. Always off on subscription/OAuth — distil does not
    alter a flat-rate prompt.
  - Drift guard: shadow mode's paired verdicts feed one anytime-valid
    e-process per machine, betting against the same budget the certificate is
    written at. A proven breach holds every proxy at lossless-only (Tier-0, no
    digest, no shaping) across restarts until `distil reset --drift-guard`. The
    multi-tenant gateway is exempt (ADR 0016). New since 1.53.0rc1, rc soak
    pending.
  - Receipts: every request appends a content-free, hash-chained receipt; the
    chain is sealed into segments with RFC 6962-style Merkle-root checkpoints,
    so `distil receipts --prove <request-id>` hands an auditor one receipt and
    `--check-proof` verifies it against a pinned checkpoint hash (content,
    segment, position) or a pinned root (membership only).
  - Opaque provider content: Anthropic `compaction`/`thinking` blocks and OpenAI
    `reasoning` (encrypted_content) and `compaction` items are forwarded
    byte-identical, pinned by contract tests, and censused rather than hidden.

Not yet measured live: cold-point recompression (ADR 0014) shrinks older tool
output only on a turn the provider's cache has certainly expired for; its gate
is an rc soak plus a live A/B. `--no-cold-point` opts out.

## Key sources

README (canonical overview, install, head-to-head table, security):
https://github.com/dshakes/distil/blob/main/README.md

CHANGELOG (per-version feature list):
https://github.com/dshakes/distil/blob/main/CHANGELOG.md

Changelog (the same CHANGELOG.md, rendered on the site so it can never drift):
https://dshakes.github.io/distil/changelog.html

THREAT_MODEL.md (what distil protects against and explicitly does not):
https://github.com/dshakes/distil/blob/main/THREAT_MODEL.md

docs/EVALUATION.md (evaluation methodology, what the numbers mean):
https://dshakes.github.io/distil/EVALUATION.md

benchmarks/README.md (reproducible head-to-head harness):
https://github.com/dshakes/distil/blob/main/benchmarks/README.md

## Site pages

Getting started (install, quickstart, wrap presets, proof ledger):
https://dshakes.github.io/distil/getting-started.html

Architecture (module map, tier model, cert loop, OTel, Proof Ledger):
https://dshakes.github.io/distil/architecture.html

CLI reference (every subcommand, flags, sample output):
https://dshakes.github.io/distil/cli.html

Deploy & Security (topologies, encrypt-at-rest, gateway keys, OTel session id):
https://dshakes.github.io/distil/deploy-security.html

Security (where to report a vulnerability, the full data-handling whitepaper,
and the deploy security guide — one page, three links):
https://dshakes.github.io/distil/security.html

Metrics & Observability (Prometheus /distil/metrics exposition and its admin
gate, OpenTelemetry counters + GenAI spans, what to alert on):
https://dshakes.github.io/distil/metrics.html

FAQ (decision-equivalence, nightly gate, OpenAI/Gemini adapters, troubleshooting):
https://dshakes.github.io/distil/faq.html

Benchmark (certified compression frontier, live head-to-head):
https://dshakes.github.io/distil/benchmark.html

Provider Compaction (pre-registered A/B of Anthropic context editing & OpenAI
server-side compaction: does the provider's own context manipulation change
agent decisions? clearing changed 92.5% of agent decisions at keep=0 and 95-100% at the
shipped keep=3 default, where the failure shifts from stalling to acting on the
wrong facts; OpenAI compaction 12.5-20%):
https://dshakes.github.io/distil/provider-compaction.html

Reproduce the numbers (all CI gates, real-model grading, scorecard):
https://dshakes.github.io/distil/benchmarks.html

Compare (distil vs LLMLingua-2 vs Headroom):
https://dshakes.github.io/distil/compare.html

Techniques (tiers, causal pruning, query-aware salience, learned keep-model):
https://dshakes.github.io/distil/techniques.html

Concepts (DERC certificate, conformal risk control, trajectory certificate):
https://dshakes.github.io/distil/concepts.html

Library API (embed distil in your own agent — Python and TypeScript, the tier
boundary, and why the in-process path is lossless-only):
https://dshakes.github.io/distil/library.html

Which mode am I? (proxy vs hook vs library — the one page that answers "which
one applies to me" before any jargon; start here if unsure):
https://dshakes.github.io/distil/which-mode.html

Subscription mode (why the proxy stays lossless-only on a flat-rate plan, what
the PostToolUse hook does instead, and what "savings" means when the bill is
fixed):
https://dshakes.github.io/distil/subscription.html

Post-tool hooks (Claude Code, Cursor MCP output, Gemini CLI, Codex CLI: compress
tool output before it enters the history; lossless-only on a subscription unless
the user opts in with --digest, digest by default on a metered key, every digest
recoverable with `distil expand <handle>`; Windsurf documented as unsupported):
https://dshakes.github.io/distil/hooks.html

Evals (every probe distil runs on itself, including the three bugs these probes
found in distil's OWN harness — published, not hidden):
https://dshakes.github.io/distil/evals.html

Model migration (how the default live certifier, claude-sonnet-5-5 at low effort, was chosen by a paired, pre-registered eval on real traffic; the served-path gap: the certified strategy saves 2.6% of tokens on real SWE-agent trajectories, serving saves 52.6%, and `distil certify --strategy served` grades the real adapter, whose result is an open risk; the SWE-bench outcome eval that would settle it is built and not yet run):
https://dshakes.github.io/distil/model-migration.html

What the exit summary proves (the four verdicts every `distil wrap` session ends
with — an anytime-valid alarm on the certified risk budget, a distribution-free
upper bound on the live decision-change rate, compression's measured effect on
reply length, and the hash-chained receipts — and the method behind each):
https://dshakes.github.io/distil/evals.html#exit-summary

Independent benchmark (75 agent runs by an outside engineer, graded from the
API's own usage fields: the defect that finished 6 of 15 tasks, why it happened,
and what 1.49.0 changed):
https://dshakes.github.io/distil/benchmark-independent.html

Research (the negative results too: where compression did NOT survive real
end-to-end execution, and what that bounds):
https://dshakes.github.io/distil/research.html

Cache contract (the invariant: compression may never rewrite bytes the provider
has already cached, and what happens when it does):
https://dshakes.github.io/distil/cache-contract.html

Threat model (compression as an attack surface: the COMA-class battery, what is
mitigated by construction, what is only detected, and the two cases that do not
come back clean):
https://dshakes.github.io/distil/threat-model.html

Cache (prompt-prefix economics: why a compressor that removes tokens can still
cost more, and how distil stays cache-monotonic):
https://dshakes.github.io/distil/cache.html

Integrations (framework hooks and SDK adapters — LiteLLM, the LiteLLM Proxy
pre-call hook distil.integrations.litellm_hook, LangChain, LangGraph, Agno,
Strands, AutoGen, LlamaIndex, an ASGI middleware, Vercel AI SDK, OpenAI,
Anthropic, Gemini):
https://dshakes.github.io/distil/integrations.html
https://dshakes.github.io/distil/autogen.html
https://dshakes.github.io/distil/llamaindex.html
https://dshakes.github.io/distil/asgi.html

Adoption (install counts and community savings, and why an opt-in census can
never attribute them to individual users):
https://dshakes.github.io/distil/adoption.html

IDE agents and config-file agents (Cursor, Cline, Kilo, Roo, Warp, Cortex Code
via the proxy since `wrap` cannot reach them; the config key each config-file
agent needs pointed at `distil proxy`'s printed URL):
https://github.com/dshakes/distil/blob/main/docs/IDE-AGENTS.md

Enterprise (Helm chart, OIDC + RBAC, audit receipts, and an honest commercial
status: no SOC 2, no SLA, no SAML today):
https://github.com/dshakes/distil/blob/main/docs/ENTERPRISE.md

Security whitepaper (data residency, encrypt-at-rest, supply chain, what is
explicitly out of scope):
https://github.com/dshakes/distil/blob/main/docs/SECURITY-WHITEPAPER.md
