Metadata-Version: 2.4
Name: routesmith-llm
Version: 0.8.0
Summary: Adaptive LLM execution engine - intelligent routing, cascading, caching, and budget management
Project-URL: Homepage, https://github.com/routesmith/routesmith
Project-URL: Documentation, https://github.com/routesmith/routesmith#readme
Project-URL: Repository, https://github.com/routesmith/routesmith
Author: routesmith contributors
License-Expression: MIT
License-File: LICENSE
Keywords: ai,caching,cascading,cost-optimization,llm,optimization,routing
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: litellm>=1.40.0
Requires-Dist: numpy>=1.24.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: simpleeval>=0.9.13
Provides-Extra: all
Requires-Dist: routesmith[anthropic,autogen,cache,crewai,docs,dspy,langchain,predictor,proxy,tui]; extra == 'all'
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.25.0; extra == 'anthropic'
Provides-Extra: autogen
Requires-Dist: pyautogen>=0.2.0; extra == 'autogen'
Provides-Extra: cache
Requires-Dist: faiss-cpu>=1.7.0; extra == 'cache'
Requires-Dist: sentence-transformers>=2.2.0; extra == 'cache'
Provides-Extra: crewai
Requires-Dist: crewai>=0.80.0; extra == 'crewai'
Provides-Extra: dev
Requires-Dist: mypy>=1.0.0; extra == 'dev'
Requires-Dist: pip-audit>=2.0.0; extra == 'dev'
Requires-Dist: prometheus-client>=0.16.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.21.0; extra == 'dev'
Requires-Dist: pytest-cov>=4.0.0; extra == 'dev'
Requires-Dist: pytest>=7.0.0; extra == 'dev'
Requires-Dist: ruff>=0.1.0; extra == 'dev'
Requires-Dist: scikit-learn>=1.3.0; extra == 'dev'
Provides-Extra: docs
Requires-Dist: mkdocs-material>=9.5.0; extra == 'docs'
Requires-Dist: mkdocstrings[python]>=0.25.0; extra == 'docs'
Requires-Dist: pymdown-extensions>=10.0.0; extra == 'docs'
Provides-Extra: dspy
Requires-Dist: dspy-ai>=2.0.0; extra == 'dspy'
Provides-Extra: langchain
Requires-Dist: langchain-core>=0.3.0; extra == 'langchain'
Provides-Extra: langchain-agents
Requires-Dist: langchain-core>=0.3.0; extra == 'langchain-agents'
Requires-Dist: langgraph-supervisor>=0.0.20; extra == 'langchain-agents'
Requires-Dist: langgraph-swarm>=0.1.0; extra == 'langchain-agents'
Requires-Dist: langgraph>=1.0.0; extra == 'langchain-agents'
Provides-Extra: predictor
Requires-Dist: scikit-learn>=1.3.0; extra == 'predictor'
Requires-Dist: sentence-transformers>=2.2.0; extra == 'predictor'
Provides-Extra: proxy
Requires-Dist: certifi>=2024.0.0; extra == 'proxy'
Requires-Dist: httpx>=0.24.0; extra == 'proxy'
Requires-Dist: prometheus-client>=0.16.0; extra == 'proxy'
Requires-Dist: pyyaml>=6.0.0; extra == 'proxy'
Requires-Dist: questionary>=2.0.0; extra == 'proxy'
Provides-Extra: tui
Requires-Dist: textual>=0.40.0; extra == 'tui'
Description-Content-Type: text/markdown

# RouteSmith

**The smart router for AI coding tools.** 40-60% cost savings. Zero quality loss. Backed by contextual bandit research.

```bash
pip install "routesmith[proxy]"
routesmith init
routesmith serve
# → Proxy at http://localhost:9119/v1
```

RouteSmith sits between your AI coding tool and the LLM. It routes every request
to the best model for that specific task — cheap models for simple edits, frontier
models for complex refactors. You never think about model IDs again.

| | OpenRouter Auto | RouteSmith |
|---|---|---|
| Learns from your traffic | ❌ | ✅ Online bandit learning |
| Self-hosted / no data leak | ❌ | ✅ Open source, self-hosted |
| Custom model pool | ❌ | ✅ Any model, any provider |
| Custom rewards & policies | ❌ | ✅ Per-role, configurable reward fns |
| Decision audit log | ❌ | ✅ Full routing decision trace |
| Budget caps (daily/hourly) | ❌ | ✅ Monthly, per-request, per-project |
| Conversation stickiness | ✅ | ✅ 

**v0.7.0**: Per-project cost stats, decision audit log, per-role policy CLI, Anthropic `/v1/messages` endpoint. Phase 5 complete.

## Who it's for

**💰 You pay for API access. Cut your bill 40-60%.**
Your Claude Code or Codex session burns through tokens. RouteSmith sends typos and
formatting to gpt-4o-mini, saves Claude Opus for architecture decisions.

**🆓 You use free models. Get better answers.**
Free models are good individually — but none is great at everything. RouteSmith
orchestrates them: hard problems get the strongest free model, easy ones get the
fastest, and weak answers cascade to second opinions.

```python
from routesmith import RouteSmith

# Free models — zero cost, smart routing
rs = RouteSmith.with_free_models()

# Or bring your own models
rs = RouteSmith()
rs.register_model("gpt-4o-mini", cost_per_1k_input=0.15, cost_per_1k_output=0.60, quality_score=0.85)
rs.register_model("claude-sonnet-4", cost_per_1k_input=3.0, cost_per_1k_output=15.0, quality_score=0.92)

response = rs.completion(messages=[{"role": "user", "content": "Explain recursion"}])
```

## AI coding tools

Point any AI coding tool at `http://localhost:9119/v1`:

| Tool | Config |
|------|--------|
| Claude Code | Enable Codex plugin, set `OPENAI_BASE_URL=http://localhost:9119/v1` |
| Codex | `export OPENAI_BASE_URL="http://localhost:9119/v1"` |
| OpenClaw | `routesmith openclaw-config` |
| pi | `routesmith openclaw-config` (OpenClaw-compatible provider) |
| OpenCode | Set `base_url` to `http://localhost:9119/v1` in provider config |
| Anthropic SDK | `export ANTHROPIC_BASE_URL=http://localhost:9119` |

> **Anthropic-native endpoint** (`POST /v1/messages`): Set `ANTHROPIC_BASE_URL=http://localhost:9119` and any Anthropic SDK client routes through RouteSmith. Supports streaming and non-streaming.

[Integration guides →](https://github.com/yunpengl9071/routesmith/tree/dev/docs/integrations)

## Why RouteSmith

| | Raw OpenRouter | Manual routing | RouteSmith |
|---|---|---|---|
| Picks model per query | ❌ | 😓 You do it | ✅ Automatic |
| Cascades when answer is weak¹ | ❌ | ❌ | ✅ |
| Caches repetitive queries | ❌ | ❌ | ✅ |
| Enforces budget limits | ❌ | ❌ | ✅ |
| Tracks costs per model | ✅ | ❌ | ✅ |
| Works with 100+ models | ✅ | ❌ | ✅ |
| Zero-config start | ✅ | ❌ | ✅ |
| Learns from feedback | ❌ | ❌ | ✅ |

¹ Today all strategies select a single model; cascade execution lands in Phase 2 (see [ROADMAP.md](ROADMAP.md)).

## Features

### Intelligent Routing
- **7 predictor types**: LinUCB, LinTS, NeuralUCB, REINFORCE, WarmStart LinUCB, Adaptive (random forest), Embedding
- **35-dimensional feature space**: query type classification, difficulty estimation, model metadata
- **Online learning**: bandits improve from the first query onward — no pretraining labels needed
- **Multi-model routing**: scales to $K$ arms naturally (validated on 5-model deployments)

### Enterprise
- **Provisioned throughput support**: prioritize pre-paid capacity, overflow to on-demand
- **Compliance routing**: tag-based filtering (HIPAA, SOC2, PCI)
- **Budget enforcement**: FAIL, FALLBACK, QUEUE behaviors
- **Multi-project isolation**: per-project cost allocation and stats

### Production
- **Semantic cache**: embedding-based dedup, configurable similarity
- **Framework adapters**: LangChain, DSPy, CrewAI, AutoGen, Anthropic, OpenClaw
- **OpenAI-compatible proxy**: works with any tool, zero code changes
- **Observability**: Prometheus metrics, structured logging, cost tracking, dashboard TUI
- **Resilience**: circuit breakers, retry with backoff, health checks, Docker

## Research

RouteSmith is backed by a research paper evaluating contextual bandit routing:

- **LinTS-35d** achieves 46% cost reduction with APGR=0.593 on MMLU
- **LinUCB-35d** achieves APGR=1.126 by selective strong-arm routing
- **5-arm routing**: 45% cost savings across GPT-4o, Claude-Sonnet-4.5, Qwen-Plus, MiniMax-M1, DeepSeek-V3
- **Zero pretraining labels** — learns from ~100 queries vs. 55K+ required by supervised routers
- Sub-millisecond routing latency (<0.5ms P99)

Paper: [`paper/main.pdf`](paper/main.pdf) | Compile with: `cd paper && tectonic main.tex`

## Framework integrations

```python
# Anthropic SDK
from routesmith.integrations.anthropic import RouteSmithAnthropic
client = RouteSmithAnthropic.with_openrouter_models()

# LangChain
from routesmith.integrations.langchain import ChatRouteSmith
llm = ChatRouteSmith.with_openai_models()

# DSPy
from routesmith.integrations.dspy import RouteSmithLM
lm = RouteSmithLM()

# CrewAI
from routesmith.integrations.crewai import routesmith_crewai_chat_model
llm = routesmith_crewai_chat_model()

# AutoGen
from routesmith.integrations.autogen import routesmith_autogen_agents
assistant, user = routesmith_autogen_agents()
```

## Quick start

```bash
# Interactive setup: browse OpenRouter catalog, pick models
routesmith init

# Start the proxy
routesmith serve

# Check stats
routesmith stats

# View routing decisions
routesmith audit

# Manage per-role routing policies
routesmith roles list
routesmith roles set --role coder --model-pool gpt-4o-mini gpt-4o
routesmith roles unset --role coder
```

```python
# Python API
from routesmith import RouteSmith

rs = RouteSmith.with_free_models()
response = rs.completion(messages=[{"role": "user", "content": "Hello!"}])
print(response.choices[0].message.content)

# Learn from user feedback
rs.record_outcome(response._routesmith_request_id, score=0.9)
```

## Examples

| File | Requires | Description |
|------|----------|-------------|
| `examples/quickstart_python.py` | — | Register models, completion, stats, feedback |
| `examples/quickstart_proxy.sh` | `routesmith[proxy]` | Proxy via CLI: init → serve → curl |
| `examples/multi_agent_roles.py` | — | Per-role routing (planner/coder/summarizer) |
| `examples/langgraph_agents.py` | `langchain_core` | 2-node LangGraph with per-role `ChatRouteSmith` |
| `examples/crewai_crew.py` | `crewai` | 2-agent CrewAI crew with shared RouteSmith |
| `examples/autogen_pair.py` | `autogen` | AutoGen agent pair via proxy |
| `examples/dspy_pipeline.py` | `dspy` | DSPy `Predict` with `RouteSmithLM` |
| `examples/openai_agents_sdk.py` | `openai` | OpenAI SDK pointed at proxy |
| `examples/pydantic_ai_agent.py` | `pydantic_ai` | Pydantic AI agent via proxy |
| `examples/llamaindex_engine.py` | `llama_index` | LlamaIndex `OpenAILike` via proxy |

## Documentation

- [CHANGELOG.md](CHANGELOG.md)
- [CONTRIBUTING.md](CONTRIBUTING.md)
- [Integration guides](docs/integrations/)
- [CLI reference](docs/cli.md)
- [Concepts](docs/concepts/)

## Installation

```bash
# Proxy + interactive setup (recommended)
pip install "routesmith[proxy]"

# Core Python API only
pip install routesmith

# With specific integrations
pip install "routesmith[langchain]"
pip install "routesmith[anthropic]"
pip install "routesmith[cache]"
pip install "routesmith[all]"
```

Requires Python 3.10+. Set `OPENROUTER_API_KEY` to use OpenRouter models.

## License

MIT — see [LICENSE](LICENSE)