Adapters & Integration

Three paths to production: a one-line Python wrapper that compresses in-process, a standalone HTTP proxy that works with any SDK or framework, or distil wrap — a zero-config launcher that routes any existing command through the proxy automatically.

Path 1 — In-process: wrap(client)

Module: distil/adapters/anthropic.py

The fastest path for Python codebases already using the Anthropic SDK. Wrap your client once at construction time — all downstream messages.create calls are transparently compressed and cache-pinned with no call-site changes.

import anthropic
from distil.adapters.anthropic import wrap

# Before: plain Anthropic client
client = anthropic.Anthropic()

# After: one-line drop-in
client = wrap(anthropic.Anthropic())

# No other code changes — all calls compressed transparently
response = client.messages.create(
    model="claude-opus-4-5",
    max_tokens=1024,
    system="You are a helpful assistant.",
    messages=[{"role": "user", "content": "Analyse this log output…"}],
)

What the wrapper does

On every messages.create(**kwargs) call, the wrapper:

1. Compresses the messages array via compress_messages():
   • User text blocks: Tier-0 lossless transforms (JSON minify + run collapse).
   • Tool result blocks with ≥ 6 lines: Tier-1 reversible digest, original stored in a local RestoreStore.
   • Assistant text, tool_use, and image blocks: passed through unchanged.
2. Pins the cache prefix via place_cache_control(): marks the last stable system block with {"cache_control": {"type": "ephemeral"}}, so the first call pays the cache-write price and every subsequent call pays the ~0.1× cache-read price.
3. Forwards the modified kwargs to the real client.messages.create.

The RestoreStore

Digested blocks embed an 8-hex SHA-256 handle in their compressed text. The RestoreStore maps handles to originals locally — it is never sent to the model and costs zero tokens. To recover an original:

from distil.adapters.anthropic import compress_messages

compressed, store = compress_messages(messages)
# store.handles → frozenset of all active handles
original = store.expand("a3f92b1c")  # retrieve original by handle

Design properties

• Duck-typed. The wrapper imports nothing from the Anthropic SDK — it works with any object that exposes a messages.create(**kwargs) method, including mocks in test environments.
• Non-mutating. Input message lists are never modified in place; new lists are returned.
• Reversible. Every change is either lossless (Tier-0) or stored in the local RestoreStore (Tier-1). Nothing is permanently lost.

Path 2 — Provider proxy: distil proxy

Module: distil/proxy.py

A lightweight HTTP proxy that sits between your client and the real LLM API. It intercepts POST /v1/messages, POST /v1/chat/completions, POST /v1/responses, and POST /v1beta/models/{model}:generateContent (and :streamGenerateContent) requests, compresses the payload, and forwards the modified request to the upstream. All other paths and methods pass through unchanged.

This approach works with any SDK, framework, or language — you do not need to touch your application code at all.

$ distil proxy --port 8788 --upstream https://api.anthropic.com
distil proxy listening on http://127.0.0.1:8788
  → upstream: https://api.anthropic.com

Point your client at the proxy

import anthropic
client = anthropic.Anthropic(
    base_url="http://localhost:8788"
)

import openai
client = openai.OpenAI(
    base_url="http://localhost:8788/v1",
    api_key="…"
)

litellm.completion(
    api_base="http://localhost:8788",
    model="anthropic/claude-…",
    messages=[...]
)

from langchain_anthropic import ChatAnthropic
llm = ChatAnthropic(
    base_url="http://localhost:8788"
)

$ distil proxy --upstream \
    https://generativelanguage.googleapis.com

# then in your code — no other changes
import google.generativeai as genai
genai.configure(
    transport="rest",
    client_options={
        "api_endpoint": "http://localhost:8788"
    },
)

Proxy flags

Response headers

The proxy adds up to 9 response headers to compressed responses (not all appear on every request):

• x-distil-compressed: 1 — compression was applied
• x-distil-tokens-saved: <n> — heuristic estimate of tokens saved (not billing-grade)
• x-distil-expanded: 1 — present when --expand fired and a digest was resolved in the transparent expand loop
• x-distil-cache-prefix-msgs: <n> — leading messages left byte-identical vs the previous turn, i.e. the prompt-cache-read region (only with --session-delta)
• x-distil-cache-refs: <n> — total cache-delta references (only with --session-delta)
• x-distil-cache-delta: <n> — delta-encoded references (only with --session-delta)
• x-distil-cache-tokens-saved: <n> — tokens saved by cache-delta encoding (only with --session-delta)
• x-distil-output-shaping: light|aggressive — present when output shaping fired
• x-distil-shadow: sampled — present when this request was sampled for shadow-mode decision-equivalence

The managed gateway additionally adds x-distil-tenant: <id> for per-tenant accounting.

Threading model

The proxy uses Python's ThreadingHTTPServer — each incoming connection is handled in its own thread, so concurrent requests from multiple clients are supported. It is intentionally simple and easy to audit.

Path 3 — Transparent: distil wrap

Module: distil/proxy.py · wrap_run()

When you don't own the agent's code — it's a CLI, a third-party tool, or any process that reads ANTHROPIC_BASE_URL — distil wrap gives you Path 2's compression with none of the setup. It spawns the proxy on an ephemeral port, injects the env var into the child, runs your command, and tears everything down on exit (flushing genuine savings to your ledger).

$ distil wrap -- claude -p "summarize this repo"
distil wrap → proxy http://127.0.0.1:54xxx (upstream https://api.anthropic.com)
  → ANTHROPIC_BASE_URL=http://127.0.0.1:54xxx
  → recording genuine savings → distil leaderboard

Everything after -- runs verbatim and its exit code is propagated. Use --env-var to point a different variable (e.g. an OpenAI-compatible client) at the proxy. See the CLI reference for all flags.

Google Gemini adapter

Module: distil/adapters/gemini.py

The Gemini adapter compresses Google's generateContent REST request shape. It is wired into the proxy, the async proxy, and the gateway — no extra configuration beyond pointing the proxy at the Gemini API.

$ distil proxy --upstream https://generativelanguage.googleapis.com
distil proxy listening on http://127.0.0.1:8788
  → upstream: https://generativelanguage.googleapis.com

See examples/python_gemini.py for a runnable end-to-end example.

Request shape handled

{
  "contents": [
    {
      "role": "user" | "model",
      "parts": [
        { "text": "…" },
        { "functionCall": {"name": "…", "args": {…}} },
        { "functionResponse": {"name": "…", "response": {…}} }
      ]
    }
  ]
}

What is compressed

Path detection

The adapter activates on:

• /v1beta/models/{model}:generateContent
• /v1beta/models/{model}:streamGenerateContent
• /v1/… equivalents on the Gemini host

Savings reporting

Token savings are reported via the x-distil-tokens-saved response header and recorded to the savings ledger, exactly as for other adapters.

Shadow-mode decision-equivalence

Shadow-mode live decision-equivalence works for Gemini. It reads the chosen action from candidates[0].content.parts[].functionCall in the response.

Not yet wired for Gemini

Verbatim and lossless-only modes

--verbatim is the Tier-0-only mode for all adapters, including Gemini: the Tier-1 digest is skipped entirely, so the model sees message content byte-for-byte (JSON minify + exact-duplicate-run collapse only). Use it for interactive (human-in-the-loop) sessions, out-of-distribution traffic, or anywhere distil_expand recovery is unavailable. Lower savings.

--lossless-only is the policy mode: it restricts the proxy to lossless strategies — no lossy output-shaping, no tool injection (--expand) — but the reversible Tier-1 digest still runs. This is the correct flag for subscription or OAuth-gated deployments.

Choosing your path

Honest note: live provider path

The --runner anthropic flag on distil certify also routes to the live Anthropic API. It is implemented and wired up, but marked UNVERIFIED in the README until you run it with a real key. The offline deterministic runner (the default) is what powers all corpus measurements.