Metadata-Version: 2.4
Name: runtimerouter
Version: 0.1.0
Summary: Optimize the entire AI session, not just the next model call.
Project-URL: Homepage, https://github.com/chenyu-dev25/RuntimeRouter
Project-URL: Documentation, https://github.com/chenyu-dev25/RuntimeRouter#readme
Project-URL: Repository, https://github.com/chenyu-dev25/RuntimeRouter
Project-URL: Issues, https://github.com/chenyu-dev25/RuntimeRouter/issues
Project-URL: Changelog, https://github.com/chenyu-dev25/RuntimeRouter/blob/main/ROADMAP.md
Author: RuntimeRouter Contributors
License-Expression: MIT
License-File: LICENSE
Keywords: agent,langgraph,litellm,llm,model-selection,pydantic-ai,routing
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: litellm>=1.40.0
Requires-Dist: pydantic>=2.0
Provides-Extra: dev
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Provides-Extra: docs
Requires-Dist: mkdocs-material>=9.5; extra == 'docs'
Requires-Dist: mkdocs>=1.6; extra == 'docs'
Description-Content-Type: text/markdown

# RuntimeRouter

> **Optimize the entire AI session, not just the next model call.**

[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![PyPI version](https://img.shields.io/pypi/v/runtimerouter.svg)](https://pypi.org/project/runtimerouter/)

RuntimeRouter is a **Python-first LLM routing library** for the AI ecosystem. It sits **above** agent frameworks — LangGraph, PydanticAI, CrewAI, AutoGen — as a **Runtime Router**, not a replacement for them.

You write agents with your favorite framework. RuntimeRouter decides **which model** to call, **when**, and **why** — across the full session lifecycle.

---

## Vision

Traditional routers optimize a single inference call: given a prompt, pick the cheapest or fastest model.

RuntimeRouter's long-term goal is different: **optimize the entire AI session** — model selection, cost, latency, context, caching, and privacy — as a unified runtime decision layer.

```
┌─────────────────────────────────────────────────────────┐
│  Your Agent Framework (LangGraph / PydanticAI / …)      │
├─────────────────────────────────────────────────────────┤
│  RuntimeRouter  ← session-aware routing (this library)  │
├─────────────────────────────────────────────────────────┤
│  LiteLLM / Provider APIs (OpenAI, Anthropic, …)         │
└─────────────────────────────────────────────────────────┘
```

---

## Why RuntimeRouter?

| Problem | RuntimeRouter approach |
|---------|------------------------|
| Hard-coding `model="gpt-4o"` everywhere | `AutoLLM()` picks the right model per request |
| Simple tasks burning frontier-model budget | Complexity routing sends easy tasks to cheap models |
| No visibility into routing decisions | Every call returns a `RouteDecision` with reason |
| Framework lock-in | Framework-agnostic; works with any LiteLLM-compatible stack |
| Proxy-only routers | Designed for **session-level** optimization (roadmap) |

---

## vs OpenRouter

| | **OpenRouter** | **RuntimeRouter** |
|---|----------------|-------------------|
| **What it is** | Unified API proxy to 100+ models | Python routing **library** embedded in your app |
| **Scope** | Single HTTP request → model | Entire AI **session** (roadmap) |
| **Integration** | Replace your API base URL | Drop-in `AutoLLM()` or `Router` in Python code |
| **Policies** | Provider-side routing rules | Pluggable Python **Policy** classes you own |
| **Framework** | Language-agnostic HTTP | **Python-first**, native LangGraph/PydanticAI hooks (v0.2+) |

OpenRouter is excellent as a model gateway. RuntimeRouter is a **runtime decision engine** you control in-process.

---

## vs Not Diamond

| | **Not Diamond** | **RuntimeRouter** |
|---|-----------------|-------------------|
| **What it is** | Managed routing SaaS | Open-source Python library |
| **Deployment** | Cloud API | In-process, self-hosted |
| **Customization** | Platform-configured | Full **plugin Policy** architecture |
| **Session scope** | Per-call model selection | Session-aware optimization (roadmap) |
| **Cost** | SaaS pricing | Free & open (MIT) |

Not Diamond provides intelligent routing as a service. RuntimeRouter gives you the same **concept** as extensible, auditable Python code.

---

## Quick Start

### Install

```bash
pip install runtimerouter
```

### Basic usage

```python
from runtimerouter import AutoLLM

llm = AutoLLM()

# RuntimeRouter automatically selects Claude, Gemini, DeepSeek, OpenAI, etc.
response = llm.invoke("帮我分析整个代码仓库")

print(response.choices[0].message.content)
```

### Inspect routing decisions

```python
decision = llm.route_only("Summarize this paragraph in one sentence.")
print(decision.selected_model)  # e.g. "gpt-4o-mini"
print(decision.reason)          # why this model was chosen
print(decision.complexity)      # e.g. ComplexityLevel.SIMPLE
```

### Configure policies

```python
from runtimerouter import AutoLLM, RouterConfig

config = RouterConfig(
    enable_complexity_routing=True,
    enable_cost_routing=True,
    fallback_model="gpt-4o-mini",
    policy_order=["complexity", "cost"],
)

llm = AutoLLM(config=config)
```

### Use Router directly (without invoking)

```python
from runtimerouter import Router, RouteContext

router = Router()
decision = router.route(RouteContext(prompt="Explain quantum entanglement"))
print(decision.selected_model)
```

---

## Architecture

```
runtimerouter/
├── autollm.py          # User-facing entry point
├── router.py           # Routing pipeline orchestrator
├── classifier.py       # Task complexity & token estimation
├── types.py            # Shared data contracts (Pydantic models)
├── config.py           # Config loading utilities
├── policies/           # Pluggable routing policies
│   ├── base.py         # RoutingPolicy ABC
│   ├── complexity.py   # Complexity-based routing
│   └── cost.py         # Cost-based routing
├── providers/          # Model catalog & provider metadata
│   ├── base.py         # ModelProvider ABC
│   └── registry.py     # ModelRegistry (default model catalog)
└── integrations/       # External execution layers
    └── litellm.py      # LiteLLM completion wrapper
```

**Routing pipeline (v0.1):**

```
User prompt
    │
    ▼
TaskClassifier.enrich()     ← complexity, token estimate
    │
    ▼
Policy chain (ordered)      ← complexity → cost
    │
    ▼
RouteDecision               ← selected model + reason
    │
    ▼
LiteLLMIntegration          ← execute completion
```

See [docs/architecture.md](docs/architecture.md) for module boundaries and extension points.

---

## Roadmap

| Version | Focus |
|---------|-------|
| **v0.1** *(current)* | Auto Model Selection, Complexity Routing, Cost Routing, LiteLLM Integration |
| **v0.2** | LangGraph Integration, PydanticAI Integration |
| **v0.3** | Cache-aware Routing |
| **v0.4** | Context-aware Routing |
| **v0.5** | Session-aware Optimization |

Full details: [ROADMAP.md](ROADMAP.md)

---

## Contributing

We welcome contributions! See [CONTRIBUTING.md](CONTRIBUTING.md).

```bash
git clone https://github.com/chenyu-dev25/RuntimeRouter.git
cd RuntimeRouter
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
pytest
```

---

## License

MIT — see [LICENSE](LICENSE).

---

<p align="center">
  <strong>RuntimeRouter</strong><br>
  Optimize the entire AI session, not just the next model call.
</p>
