Integrations — one proxy, every SDK

Start distil proxy once and point any SDK's baseURL at it. No library changes, no monkey-patching — compression happens at the network layer.

How it works

The proxy (distil proxy, default http://127.0.0.1:8788) is a local HTTP server. It intercepts the three compressible paths across all major LLM APIs:

/v1/messages — Anthropic Messages API
/v1/chat/completions — OpenAI Chat Completions (also used by LiteLLM, LangChain, etc.)
/v1/responses — OpenAI Responses API
/v1beta/models/{model}:generateContent and :streamGenerateContent — Google Gemini REST API (also the /v1/… host variants)

All other paths and HTTP verbs pass through unchanged. Your API key travels in the request headers exactly as normal — the proxy never logs or stores it.

Zero SDK changes beyond baseURL. The proxy is transparent: the SDK sees the same wire format it always expects. Auth headers, streaming, tool use, and all other features work as-is.

SDK integration matrix

SDK / Framework	Language	Setting	Value	Example
Anthropic Python SDK	Python	`base_url=`	`http://127.0.0.1:8788`	python_anthropic.py
OpenAI Python SDK	Python	`base_url=`	`http://127.0.0.1:8788/v1`	python_openai.py
LiteLLM	Python	`api_base=`	`http://127.0.0.1:8788`	python_litellm.py
Vercel AI SDK (`@ai-sdk/anthropic`)	TypeScript	`baseURL` in `createAnthropic({…})`	`http://127.0.0.1:8788`	js_vercel_ai_sdk.ts
LangChain.js (`@langchain/anthropic`)	TypeScript	`anthropicApiUrl` in `ChatAnthropic({…})`	`http://127.0.0.1:8788`	js_langchain.ts
Google Gemini REST (`google-generativeai`)	Python / curl	`api_endpoint` in `client_options`	`http://127.0.0.1:8788` (upstream: `https://generativelanguage.googleapis.com`)	python_gemini.py

Snippets

Anthropic Python SDK

import anthropic

client = anthropic.Anthropic(
    api_key="sk-ant-…",
    base_url="http://127.0.0.1:8788",
)
response = client.messages.create(
    model="claude-opus-4-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello!"}],
)

OpenAI Python SDK

import openai

client = openai.OpenAI(
    api_key="sk-…",
    base_url="http://127.0.0.1:8788/v1",
)
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
)

LiteLLM

import litellm

response = litellm.completion(
    model="claude-opus-4-5",
    api_base="http://127.0.0.1:8788",
    api_key="sk-ant-…",
    messages=[{"role": "user", "content": "Hello!"}],
)

Vercel AI SDK (TypeScript)

import { createAnthropic } from "@ai-sdk/anthropic";
import { generateText } from "ai";

const anthropic = createAnthropic({
  baseURL: "http://127.0.0.1:8788",
  apiKey: process.env.ANTHROPIC_API_KEY,
});

const { text } = await generateText({
  model: anthropic("claude-opus-4-5"),
  prompt: "Hello!",
});

LangChain.js (TypeScript)

import { ChatAnthropic } from "@langchain/anthropic";

const model = new ChatAnthropic({
  model: "claude-opus-4-5",
  apiKey: process.env.ANTHROPIC_API_KEY,
  anthropicApiUrl: "http://127.0.0.1:8788",
  // older versions: clientOptions: { baseURL: "http://127.0.0.1:8788" }
});

const response = await model.invoke([
  ["human", "Hello!"],
]);

Google Gemini REST

# Start proxy pointing at the Gemini API
distil proxy --port 8788 --upstream https://generativelanguage.googleapis.com

import google.generativeai as genai

genai.configure(
    transport="rest",
    client_options={"api_endpoint": "http://127.0.0.1:8788"},
)
model = genai.GenerativeModel("gemini-1.5-pro")
response = model.generate_content("Hello!")

See examples/python_gemini.py for a full runnable example including tool-use turns. The proxy transparently compresses text parts (Tier-0 lossless) and functionResponse payloads (Tier-1 reversible digest). functionCall, inlineData, fileData, model-authored text, and systemInstruction are always passed through unchanged.

Observability headers

The proxy adds two informational headers to every compressed response:

Header	Meaning
`x-distil-compressed: 1`	Compression was applied this turn.
`x-distil-tokens-saved: <n>`	Estimated input tokens saved (heuristic tokenizer).

Node.js / TypeScript

Distil's universal path is the proxy — it's language-agnostic, so JS/TS stacks need no Distil-specific package. Start the proxy (or wrap your agent) and point your SDK's baseURL at it:

# start the proxy (needs Python 3.11+ — see Install)
distil proxy --port 8788 --upstream https://api.anthropic.com

// then in your Node/TS app — no code change beyond baseURL
const client = new Anthropic({ baseURL: "http://localhost:8788" });

Or wrap an existing CLI/agent in one shot: distil wrap -- <your-command> starts the proxy and injects ANTHROPIC_BASE_URL automatically.

Homebrew

brew tap dshakes/tap
brew install dshakes/tap/distil
distil proxy --port 8788