Metadata-Version: 2.4
Name: routiq
Version: 0.1.5
Summary: Routiq client SDK — connect to a Routiq gateway and manage it via Docker
Author-email: Yash Bafna <yashbafnanitrr@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/yashbafna/routiq-feedback
Project-URL: Bug Tracker, https://github.com/yashbafna/routiq-feedback/issues
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: openai>=1.30
Requires-Dist: httpx>=0.27
Dynamic: license-file

# Routiq

> **The LLM gateway that picks the right model, right provider, right price — every time.**

Routiq is a local-first LLM gateway you run yourself. It sits between your app and LLM providers and reduces costs automatically — through caching, complexity-based routing, and cost tracking. Your prompts never leave your machine.

**Supported providers:** OpenAI · Anthropic · DeepSeek · Gemini

**How it reduces costs:**

| Mechanism | What it does | Typical saving |
|-----------|-------------|----------------|
| Exact cache | Returns cached response for identical requests — zero provider cost | 20–40% |
| Semantic cache | Matches similar prompts using embeddings — catches paraphrases | 20–40% |
| Complexity routing | Sends simple prompts to cheap models, hard ones to powerful models | 40–60% |

**Real numbers from testing:** 84% cache hit rate · up to 88% cost savings · 9ms p50 overhead

---

## Quick Start

**Tip:** create a dedicated folder for Routiq before installing — both paths create config files in your current directory.

```bash
mkdir routiq && cd routiq
```

### Option A — Docker (recommended, one command, includes Redis)

**Prerequisites:** [Docker Desktop](https://www.docker.com/products/docker-desktop/) (Mac/Windows) or Docker Engine with `docker compose` (Linux).

```bash
# 1. Download install files
curl -O https://raw.githubusercontent.com/yashbafna/routiq-feedback/main/docker-compose.yml
curl -O https://raw.githubusercontent.com/yashbafna/routiq-feedback/main/.env.example
mv .env.example .env

# 2. Edit .env — set your gateway key and at least one provider key
#    ROUTIQ_API_KEY=any-secret-you-choose
#    GEMINI_API_KEY=AIza...   ← free at https://ai.google.dev

# 3. Start (Redis included automatically)
docker compose up -d

# 4. Health check
sleep 5 && curl http://localhost:8001/health
# → {"status": "ok", "version": "0.1.5"}

# 5. Make your first call
curl http://localhost:8001/v1/chat/completions \
  -H "Authorization: Bearer YOUR_ROUTIQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "auto", "messages": [{"role": "user", "content": "Say hi"}]}'
```

### Option B — pip (Python 3.9+)

**Caching note:** the pip path runs Routiq without caching unless you install Redis separately. You'll still get routing and cost tracking — just not the full cache hit rate. Use Option A for the full experience in one command.

**Recommended:** use `pipx` or a virtual environment — `pip install` against system Python fails on macOS and Ubuntu 22.04+.

```bash
# 1. Install
pipx install routiq
# Or: python3 -m venv venv && source venv/bin/activate && pip install routiq

# 2. First run — creates .env in current directory
routiq start
# → .env created. Edit it and run again.

# 3. Edit .env
#    ROUTIQ_API_KEY=any-secret-you-choose
#    GEMINI_API_KEY=AIza...   ← free at https://ai.google.dev

# 4. Start the gateway
routiq start
# → Gateway healthy at http://localhost:8001

# 5. Health check
curl http://localhost:8001/health
# → {"status": "ok", "version": "0.1.5"}

# 6. Make your first call
curl http://localhost:8001/v1/chat/completions \
  -H "Authorization: Bearer YOUR_ROUTIQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "auto", "messages": [{"role": "user", "content": "Say hi"}]}'
```

To enable caching:

```bash
brew install redis && brew services start redis        # macOS
sudo apt install redis-server && sudo systemctl start redis  # Linux
```

Restart `routiq start` after Redis is up — it auto-detects Redis at `localhost:6379`.

---

## Troubleshooting

**Version mismatch (Docker).** Your Docker image is cached. Force a fresh pull:
```bash
docker compose down && docker compose pull && docker compose up -d --force-recreate
```

**`.env` not found.** `routiq start` reads `.env` from your current working directory. `cd` into the folder where `.env` lives before running it.

**"Gateway unreachable" in the dashboard.** Hard-refresh your browser (`Cmd+Shift+R` / `Ctrl+Shift+R`). Usually a stale dashboard polling against a restarted backend.

**No provider keys — all requests fail.** At least one of `GEMINI_API_KEY`, `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, or `DEEPSEEK_API_KEY` must be set in `.env`.

---

## Connecting Your App

```python
from routiq import RoutiqClient

client = RoutiqClient(
    api_key="your-ROUTIQ_API_KEY",      # the key you set in .env
    base_url="http://localhost:8001"    # optional, this is the default
)

# Let Routiq pick the model automatically based on prompt complexity
response = client.chat(messages=[{"role": "user", "content": "Hello"}])
print(response.choices[0].message.content)

# Pin a specific provider and model
client.chat(model="claude-latest", messages=[...])
client.chat(model="gemini-flash-latest", messages=[...])
client.chat(model="deepseek-latest", messages=[...])
client.chat(model="gpt-4o-latest", messages=[...])

# Async support
response = await client.achat(messages=[...])
```

**Already using the OpenAI SDK?** Point it directly at Routiq:

```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8001/v1", api_key="your-ROUTIQ_API_KEY")
```

---

## Model Aliases

Aliases are pinned — provider releases won't silently break your app.

| Alias | Resolves To | Provider |
|-------|-------------|----------|
| `gpt-4o-latest` | gpt-4o-2024-11-20 | OpenAI |
| `gpt-4o-mini-latest` | gpt-4o-mini | OpenAI |
| `claude-latest` | claude-sonnet-4-20250514 | Anthropic |
| `claude-haiku-latest` | claude-haiku-4-5-20251001 | Anthropic |
| `deepseek-latest` | deepseek-chat | DeepSeek |
| `deepseek-reasoner-latest` | deepseek-reasoner | DeepSeek |
| `gemini-flash-latest` | gemini-2.5-flash | Gemini |
| `gemini-flash-lite-latest` | gemini-2.5-flash-lite | Gemini |
| `gemini-pro-latest` | gemini-2.5-pro | Gemini |
| `auto` | complexity-based selection | — |

Canonical model names (`gpt-4o`, `claude-sonnet-4-20250514` etc.) also work directly.

---

## Complexity Routing

When you use `model="auto"`, Routiq classifies each prompt and routes it to the cheapest model that can handle it:

| Complexity | Triggers | Default model |
|------------|----------|--------------|
| Simple | Short prompt, no tools | `gemini-flash-latest` |
| Medium | Moderate length or reasoning | `gpt-4o-mini` |
| Complex | Tools present, long context, keywords like `debug`, `implement`, `analyze` | `claude-latest` |

Override tiers in `.env`:
```env
SIMPLE_MODEL=gemini-flash-latest,gpt-4o-mini
MEDIUM_MODEL=gpt-4o-mini,claude-haiku-latest
COMPLEX_MODEL=claude-latest,gpt-4o
```

Models are tried left to right — if the first fails, the next is used.

---

## Dashboard

Open `http://localhost:8001/dashboard` to see live stats: total requests, cache hit rate, total spend, total saved, latency (p50/p95/p99), breakdown by model and complexity, recent request table with MISS / HIT / SIMILAR badges.

---

## Environment Variables

**Required:**
```env
ROUTIQ_API_KEY=any-secret-you-choose
GEMINI_API_KEY=AIza...               # easiest — free at ai.google.dev
```

**Optional:**
```env
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
DEEPSEEK_API_KEY=...
REDIS_URL=redis://localhost:6379
SQLITE_PATH=./data/routiq.db
LOG_LEVEL=INFO
SEMANTIC_CACHE_ENABLED=true
SEMANTIC_THRESHOLD=0.82
SIMPLE_MODEL=gemini-flash-latest,gpt-4o-mini
MEDIUM_MODEL=gpt-4o-mini,claude-haiku-latest
COMPLEX_MODEL=claude-latest,gpt-4o
```

Only providers with a key set are active. Adapters for unconfigured providers are skipped at startup.

---

## Feedback & Issues

[→ github.com/yashbafna/routiq-feedback](https://github.com/yashbafna/routiq-feedback)

---

## License

MIT
