# Distil — compression with a quality contract

Distil is a cache-aware, causally-pruned context compressor for LLM agentic
runtimes. It compresses input and output context, certifies that the agent's
decisions are statistically unchanged (non-inferiority test), and is the only
compressor that proves its guarantees rather than assuming them. Zero runtime
dependencies. Four ways to use it, and they do not all reach the same tier:

  - Agent wrap: `distil wrap -- <agent>` for claude, codex, gemini, aider,
    opencode, qwen, goose, grok, openhands, copilot (GitHub Copilot CLI, BYOK
    mode), kimi (Kimi CLI), vibe (Mistral Vibe), kilo (Kilo Code CLI) — all of
    which route through an environment variable — plus cn (Continue CLI),
    droid (Factory Droid), omp (Oh My Pi), crush and cline (Cline CLI), which
    have no env-var contract at all and route through a config file distil
    manages for that one session and restores byte-for-byte on exit. Zero
    config either way. (openhands additionally needs its own
    --override-with-envs flag before it will read the environment; copilot
    additionally needs COPILOT_MODEL, which only you can choose.) Run
    `distil wrap --list` for every target, its mechanism, its provider wire
    shape, and the primary doc each routing contract was verified against. The
    agents distil can NOT reach are on that same list with the reason, rather
    than given a guessed variable that would route nothing while reporting
    success. One preset is itself unverified: codex speaks the Responses wire
    shape, but codex-rs reads its base URL from the `openai_base_url` config key
    and no mapping from OPENAI_BASE_URL was found, so it may route nothing.
    `distil wrap -- claude` also sets ENABLE_TOOL_SEARCH=true so Claude Code keeps
    deferring MCP tool definitions behind the proxy (unverified on a metered
    session so far); distil never rewrites the request `tools` array.
  - HTTP proxy for Anthropic Messages, OpenAI (Chat Completions + the Responses
    API), Azure OpenAI, and Google Gemini generateContent. Point any base_url
    client at it. The same provider coverage is in all three servers: the sync
    proxy, the async proxy, and the standalone gateway.
  - Python library: `from distil import compress_messages, expand_handle`.
  - TypeScript library: `compress(messages)` from the `distil-llm` npm package,
    plus `distilMiddleware()` for the Vercel AI SDK's wrapLanguageModel.

Framework hooks (in-process, no network hop): LiteLLM, LangChain, LangGraph,
Agno, Strands, AutoGen, LlamaIndex. A generic ASGI middleware (distil.integrations.asgi)
compresses provider-shaped POST bodies for apps that host their own LLM-facing
endpoint. An MCP server exposes distil_compress / distil_expand /
distil_savings to any MCP client.

IMPORTANT for anyone integrating: the reversible DIGEST tier is reached by the
proxy, wrap, and MCP paths. The in-process LIBRARIES are LOSSLESS-tier only, on
purpose — the digest mints restore handles that must share one store with the
proxy, and it is the tier the decision-equivalence certificate measures. The
TypeScript port is held byte-identical to the Python engine by a conformance
suite; where it cannot match Python's bytes it declines rather than emitting
output the certificate does not cover.

Graded on SWE-bench Verified: 42.0% task success
vs 39.2% full context (paired non-inferiority certified; superiority not
significant). 83.2% savings at 0% decision-change rate on a live head-to-head
(2026-07-05, distil 1.10.1 vs llmlingua 0.2.2 and headroom-ai 0.27.0; that run's Headroom figures are 0.27.0-era). Re-run 2026-09-04
against headroom-ai 0.37.0: distil-causal 52.9% tokens / 58.7% dollars at 100%
decision-equivalence (PASS) vs Headroom 1.7% / 2.0% / 81% (FAIL) on the warm
corpus gate; on the read->edit->re-read codebench workload Headroom takes 35.6%
tokens for +4.9% dollars.

Two reporting commands read the same content-free local records and answer
different questions. `distil dissect <session>` is backward-looking and
per-session: what happened to this one. `distil discover` is forward-looking and
cross-session: where savings are still being left on the table across the last N
sessions, as a ranked list of actions, each carrying the tokens and dollars per
week it would recover, the derivation of that number, and the one command to act
on it. `discover` prints the median and p10/p90 of per-session savings beside the
best session so a best case is never read as typical, prefers the rate the
machine itself measured over any published ratio, and stays silent where it
cannot measure.

These properties are checked rather than asserted, and each is separately
runnable:

  - Cache contract: compression is suffix-only and cache-monotonic — it may
    never rewrite bytes the provider has already cached. Enforced as an
    invariant, after shipping the bug once and measuring it at 2x the cost of
    compressing nothing.
  - Exact-quote guarantee: a block whose bytes are a verbatim slice of a file is
    never digested, so a later Edit(old_string=...) still matches. Provenance is
    read from the shell command (cat, head, sed -n), not just the tool name.
    Byte-exact quote loss on real coding traffic: 39.3% -> 16.2%.
  - Adversarial validation: `distil validate --adversarial` runs a COMA-class
    battery through the same public path the proxy uses. No keep budget is
    shared between blocks anywhere, asserted as an equality in CI. Two findings
    are published rather than smoothed over: dedup-baiting does fold the genuine
    error line (reversibility recovers it), and decoy-verdict flooding is a real,
    unmitigated denial of savings.
  - Degradation curve: `distil bench --curve` reports savings, fact recall,
    visible recall, facts lost and reversibility at every rung of the ladder,
    offline and free.
  - Paired live estimator: shadow mode replays a sampled request three times (A
    and A' on the original, B on the compressed context) and reports the paired
    difference 1{A=B} - 1{A=A'} with a bootstrap 95% CI, unclipped, so it can
    report harm. One reporting floor (50 A/B + 30 A/A) gates every surface, and
    below it every surface prints "below reporting floor" rather than a number.
    The maintainer's live sample cleared that floor on 2026-09-15: paired
    equivalence 97.5% with a 95% CI of [95.5, 99.5] over n=398 A/B and 399 A/A,
    paired difference -0.025 [-0.045, -0.005], digest and lossless-only mixed.
    Replays run hot (399 of 399, temperature not pinned), so the paired
    difference is the statistic, not the 53.0% raw agreement. Artifact:
    benchmarks/results/shadow-live-2026-09-15.json
  - Adaptive output shaping: the verbosity directive that shortens replies is
    lossy, so it runs only while that same live referee says it is safe and
    paying. `--shape-output auto` is the default on a metered session and turns
    shaping on when the paired verdict clears the reporting floor, its harm
    bound is inside the same +/-2pp budget `distil certify` uses, and the
    measured effect of compression on reply length excludes zero on the saving
    side; otherwise off, with the reason printed and recorded in the session
    manifest. The gate reads only shadow rows measured with shaping off (each
    row is tagged with its active levers), so shaping cannot keep itself on with
    its own shorter replies. Always off on subscription/OAuth — distil does not
    alter a flat-rate prompt.
  - Drift guard: shadow mode's paired verdicts feed one anytime-valid
    e-process per machine, betting against the same budget the certificate is
    written at. A proven breach holds every proxy at lossless-only (Tier-0, no
    digest, no shaping) across restarts until `distil reset --drift-guard`. The
    multi-tenant gateway is exempt (ADR 0016). New since 1.53.0rc1, rc soak
    pending.
  - Receipts: every request appends a content-free, hash-chained receipt; the
    chain is sealed into segments with RFC 6962-style Merkle-root checkpoints,
    so `distil receipts --prove <request-id>` hands an auditor one receipt and
    `--check-proof` verifies it against a pinned checkpoint hash (content,
    segment, position) or a pinned root (membership only).
  - Opaque provider content: Anthropic `compaction`/`thinking` blocks and OpenAI
    `reasoning` (encrypted_content) and `compaction` items are forwarded
    byte-identical, pinned by contract tests, and censused rather than hidden.

Not yet measured live: cold-point recompression (ADR 0014) shrinks older tool
output only on a turn the provider's cache has certainly expired for; its gate
is an rc soak plus a live A/B. `--no-cold-point` opts out.

## Key sources

README (canonical overview, install, head-to-head table, security):
https://github.com/dshakes/distil/blob/main/README.md

CHANGELOG (per-version feature list):
https://github.com/dshakes/distil/blob/main/CHANGELOG.md

Changelog (the same CHANGELOG.md, rendered on the site so it can never drift):
https://dshakes.github.io/distil/changelog.html

THREAT_MODEL.md (what distil protects against and explicitly does not):
https://github.com/dshakes/distil/blob/main/THREAT_MODEL.md

docs/EVALUATION.md (evaluation methodology, what the numbers mean):
https://dshakes.github.io/distil/EVALUATION.md

benchmarks/README.md (reproducible head-to-head harness):
https://github.com/dshakes/distil/blob/main/benchmarks/README.md

## Site pages

Getting started (install, quickstart, wrap presets, proof ledger):
https://dshakes.github.io/distil/getting-started.html

Architecture (module map, tier model, cert loop, OTel, Proof Ledger):
https://dshakes.github.io/distil/architecture.html

CLI reference (every subcommand, flags, sample output):
https://dshakes.github.io/distil/cli.html

Deploy & Security (topologies, encrypt-at-rest, gateway keys, OTel session id):
https://dshakes.github.io/distil/deploy-security.html

Security (where to report a vulnerability, the full data-handling whitepaper,
and the deploy security guide — one page, three links):
https://dshakes.github.io/distil/security.html

Metrics & Observability (Prometheus /distil/metrics exposition and its admin
gate, OpenTelemetry counters + GenAI spans, what to alert on):
https://dshakes.github.io/distil/metrics.html

FAQ (decision-equivalence, nightly gate, OpenAI/Gemini adapters, troubleshooting):
https://dshakes.github.io/distil/faq.html

Benchmark (certified compression frontier, live head-to-head):
https://dshakes.github.io/distil/benchmark.html

Provider Compaction (pre-registered A/B of Anthropic context editing & OpenAI
server-side compaction: does the provider's own context manipulation change
agent decisions? clearing changed 92.5% of agent decisions at keep=0 and 95-100% at the
shipped keep=3 default, where the failure shifts from stalling to acting on the
wrong facts; OpenAI compaction 12.5-20%):
https://dshakes.github.io/distil/provider-compaction.html

Reproduce the numbers (all CI gates, real-model grading, scorecard):
https://dshakes.github.io/distil/benchmarks.html

Compare (distil vs LLMLingua-2 vs Headroom):
https://dshakes.github.io/distil/compare.html

Techniques (tiers, causal pruning, query-aware salience, learned keep-model):
https://dshakes.github.io/distil/techniques.html

Concepts (DERC certificate, conformal risk control, trajectory certificate):
https://dshakes.github.io/distil/concepts.html

Library API (embed distil in your own agent — Python and TypeScript, the tier
boundary, and why the in-process path is lossless-only):
https://dshakes.github.io/distil/library.html

Which mode am I? (proxy vs hook vs library — the one page that answers "which
one applies to me" before any jargon; start here if unsure):
https://dshakes.github.io/distil/which-mode.html

Subscription mode (why the proxy stays lossless-only on a flat-rate plan, what
the PostToolUse hook does instead, and what "savings" means when the bill is
fixed):
https://dshakes.github.io/distil/subscription.html

Evals (every probe distil runs on itself, including the three bugs these probes
found in distil's OWN harness — published, not hidden):
https://dshakes.github.io/distil/evals.html

What the exit summary proves (the four verdicts every `distil wrap` session ends
with — an anytime-valid alarm on the certified risk budget, a distribution-free
upper bound on the live decision-change rate, compression's measured effect on
reply length, and the hash-chained receipts — and the method behind each):
https://dshakes.github.io/distil/evals.html#exit-summary

Independent benchmark (75 agent runs by an outside engineer, graded from the
API's own usage fields: the defect that finished 6 of 15 tasks, why it happened,
and what 1.49.0 changed):
https://dshakes.github.io/distil/benchmark-independent.html

Research (the negative results too: where compression did NOT survive real
end-to-end execution, and what that bounds):
https://dshakes.github.io/distil/research.html

Cache contract (the invariant: compression may never rewrite bytes the provider
has already cached, and what happens when it does):
https://dshakes.github.io/distil/cache-contract.html

Threat model (compression as an attack surface: the COMA-class battery, what is
mitigated by construction, what is only detected, and the two cases that do not
come back clean):
https://dshakes.github.io/distil/threat-model.html

Cache (prompt-prefix economics: why a compressor that removes tokens can still
cost more, and how distil stays cache-monotonic):
https://dshakes.github.io/distil/cache.html

Integrations (framework hooks and SDK adapters — LiteLLM, LangChain, LangGraph,
Agno, Strands, AutoGen, LlamaIndex, an ASGI middleware, Vercel AI SDK, OpenAI,
Anthropic, Gemini):
https://dshakes.github.io/distil/integrations.html
https://dshakes.github.io/distil/autogen.html
https://dshakes.github.io/distil/llamaindex.html
https://dshakes.github.io/distil/asgi.html

Adoption (install counts and community savings, and why an opt-in census can
never attribute them to individual users):
https://dshakes.github.io/distil/adoption.html

IDE agents and config-file agents (Cursor, Cline, Kilo, Roo, Warp, Cortex Code
via the proxy since `wrap` cannot reach them; the config key each config-file
agent needs pointed at `distil proxy`'s printed URL):
https://github.com/dshakes/distil/blob/main/docs/IDE-AGENTS.md

Enterprise (Helm chart, OIDC + RBAC, audit receipts, and an honest commercial
status: no SOC 2, no SLA, no SAML today):
https://github.com/dshakes/distil/blob/main/docs/ENTERPRISE.md

Security whitepaper (data residency, encrypt-at-rest, supply chain, what is
explicitly out of scope):
https://github.com/dshakes/distil/blob/main/docs/SECURITY-WHITEPAPER.md
