# Integrations — one proxy, every SDK

Start `distil proxy` once and point any SDK's `baseURL` at it. No library changes, no monkey-patching — compression happens at the network layer.

## How it works

The proxy (`distil proxy`, default `http://127.0.0.1:8788`) is a local HTTP server. It intercepts the three compressible paths across all major LLM APIs:

- `/v1/messages` — Anthropic Messages API
- `/v1/chat/completions` — OpenAI Chat Completions (also used by LiteLLM, LangChain, etc.)
- `/v1/responses` — OpenAI Responses API
- `/v1beta/models/{model}:generateContent` and `:streamGenerateContent` — Google Gemini REST API (also the `/v1/…` host variants)

All other paths and HTTP verbs pass through unchanged. Your API key travels in the request headers exactly as normal — the proxy never logs or stores it.

Zero SDK changes beyond baseURL. The proxy is transparent: the SDK sees the same wire format it always expects. Auth headers, streaming, tool use, and all other features work as-is.

## SDK integration matrix

SDK / Framework | Language | Setting | Value | Example
--- | --- | --- | --- | ---
Anthropic Python SDK | Python | `base_url=` | `http://127.0.0.1:8788` | python_anthropic.py
OpenAI Python SDK | Python | `base_url=` | `http://127.0.0.1:8788/v1` | python_openai.py
LiteLLM | Python | `api_base=` | `http://127.0.0.1:8788` | python_litellm.py
Vercel AI SDK (`@ai-sdk/anthropic`) | TypeScript | `baseURL` in `createAnthropic({…})` | `http://127.0.0.1:8788` | js_vercel_ai_sdk.ts
LangChain.js (`@langchain/anthropic`) | TypeScript | `anthropicApiUrl` in `ChatAnthropic({…})` | `http://127.0.0.1:8788` | js_langchain.ts
Google Gemini REST (`google-generativeai`) | Python / curl | `api_endpoint` in `client_options` | `http://127.0.0.1:8788` (upstream: `https://generativelanguage.googleapis.com`) | python_gemini.py

## Snippets

### Anthropic Python SDK

    import anthropic

    client = anthropic.Anthropic(
        api_key="sk-ant-…",
        base_url="http://127.0.0.1:8788",
    )
    response = client.messages.create(
        model="claude-opus-4-5",
        max_tokens=1024,
        messages=[{"role": "user", "content": "Hello!"}],
    )

### OpenAI Python SDK

    import openai

    client = openai.OpenAI(
        api_key="sk-…",
        base_url="http://127.0.0.1:8788/v1",
    )
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Hello!"}],
    )

### LiteLLM

    import litellm

    response = litellm.completion(
        model="claude-opus-4-5",
        api_base="http://127.0.0.1:8788",
        api_key="sk-ant-…",
        messages=[{"role": "user", "content": "Hello!"}],
    )

### Vercel AI SDK (TypeScript)

    import { createAnthropic } from "@ai-sdk/anthropic";
    import { generateText } from "ai";

    const anthropic = createAnthropic({
      baseURL: "http://127.0.0.1:8788",
      apiKey: process.env.ANTHROPIC_API_KEY,
    });

    const { text } = await generateText({
      model: anthropic("claude-opus-4-5"),
      prompt: "Hello!",
    });

### LangChain.js (TypeScript)

    import { ChatAnthropic } from "@langchain/anthropic";

    const model = new ChatAnthropic({
      model: "claude-opus-4-5",
      apiKey: process.env.ANTHROPIC_API_KEY,
      anthropicApiUrl: "http://127.0.0.1:8788",
      // older versions: clientOptions: { baseURL: "http://127.0.0.1:8788" }
    });

    const response = await model.invoke([
      ["human", "Hello!"],
    ]);

### Google Gemini REST

    # Start proxy pointing at the Gemini API
    distil proxy --port 8788 --upstream https://generativelanguage.googleapis.com

    import google.generativeai as genai

    genai.configure(
        transport="rest",
        client_options={"api_endpoint": "http://127.0.0.1:8788"},
    )
    model = genai.GenerativeModel("gemini-1.5-pro")
    response = model.generate_content("Hello!")

See `examples/python_gemini.py` for a full runnable example including tool-use turns. The proxy transparently compresses `text` parts (Tier-0 lossless) and `functionResponse` payloads (Tier-1 reversible digest). `functionCall`, `inlineData`, `fileData`, model-authored text, and `systemInstruction` are always passed through unchanged.

## MCP server

A zero-dependency Model Context Protocol server (stdlib only — no SDK) exposes distil's reversible compression to any MCP client (Claude Desktop, IDEs, agents) over stdio:

    distil mcp        # newline-delimited JSON-RPC 2.0 over stdio

Tools: `distil_compress(text)` → a compact digest + an 8-hex handle (the original is kept in a local store, never returned until asked); `distil_expand(handle)` → the original text; `distil_savings()` → cumulative ledger savings. Wire it into an MCP client's server config as a stdio command (`command: "distil", args: ["mcp"]`).

## In-process hooks (LiteLLM & LangChain)

Prefer not to run a sidecar? Compress the request in-process — the same reversible compression, no proxy. Both helpers lazy-import their framework, so distil stays zero-dep.

    # LiteLLM — drop-in for litellm.completion
    from distil.integrations import litellm as distil_litellm
    resp = distil_litellm.completion(model="claude-opus-4-8", messages=[...],
                                     distil_verbatim=True)  # optional, Tier-0 only

    # LangChain — compress a message list before the model call (duck-typed)
    from distil.integrations.langchain import compress_messages
    msgs = compress_messages(state["messages"], verbatim=True)

Tool/function messages get the reversible Tier-1 digest; human/system messages get Tier-0 lossless; the model's own words are never rewritten. `compress()` (LiteLLM) and `compress_messages()` (LangChain) are framework-free and unit-tested.

## Observability headers

The proxy adds up to 8 response headers per compressed response. The first two appear on every compressed request; the rest are conditional:

Header | Meaning | Condition
--- | --- | ---
`x-distil-compressed: 1` | Compression was applied this turn. | Always (on compressed requests)
`x-distil-tokens-saved: <n>` | Estimated input tokens saved (heuristic tokenizer). | Always (on compressed requests)
`x-distil-expanded: 1` | A digest was resolved via the transparent expand loop. | When `--expand` fired
`x-distil-cache-refs: <n>` | Total cache-delta references this turn. | With `--session-delta`
`x-distil-cache-delta: <n>` | Delta-encoded references this turn. | With `--session-delta`
`x-distil-cache-tokens-saved: <n>` | Tokens saved by cache-delta encoding. | With `--session-delta`
`x-distil-output-shaping: light|aggressive` | Output shaping was injected at this level. | When `--shape-output` fired
`x-distil-shadow: sampled` | This request was sampled for shadow-mode decision-equivalence. | When `--shadow` rate triggered

The managed gateway (`distil gateway`) additionally adds `x-distil-tenant: <id>` for per-tenant accounting.

## Node.js / TypeScript

Distil's universal path is the proxy — it's language-agnostic, so JS/TS stacks need no Distil-specific package. Start the proxy (or wrap your agent) and point your SDK's `baseURL` at it:

    # start the proxy (needs Python 3.11+ — see Install)
    distil proxy --port 8788 --upstream https://api.anthropic.com

    // then in your Node/TS app — no code change beyond baseURL
    const client = new Anthropic({ baseURL: "http://localhost:8788" });

Or wrap an existing CLI/agent in one shot: `distil wrap -- <your-command>` starts the proxy and injects `ANTHROPIC_BASE_URL` automatically.

## Homebrew

    brew tap dshakes/tap
    brew install dshakes/tap/distil
    distil proxy --port 8788