Metadata-Version: 2.4
Name: forge-gateway
Version: 0.3.0.dev0
Summary: Intelligent AI Gateway for Coding Agents — task-aware routing, policy engine, transparent failover
Project-URL: Homepage, https://github.com/WhiteJbb/forge
Project-URL: Repository, https://github.com/WhiteJbb/forge
Author: WhiteJbb
License: MIT
License-File: LICENSE
Keywords: anthropic,coding-agents,gateway,litellm,llm,openai,router
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
Requires-Python: >=3.10
Requires-Dist: fastapi>=0.115.0
Requires-Dist: httpx>=0.27.0
Requires-Dist: litellm>=1.50.0
Requires-Dist: pydantic-settings>=2.0.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: uvicorn[standard]>=0.30.0
Description-Content-Type: text/markdown

# Forge

> Intelligent AI Gateway for Coding Agents — point every coding agent at one endpoint, and stop thinking about models.

Forge sits between your coding agents (Cline, Aider, Continue, …) and your LLM providers. It analyzes each request, picks the best available model for the task, and fails over transparently when a model is rate-limited or down.

```text
Cline / Aider / Continue / RooCode ──▶ localhost:4000 ──▶ Forge ──▶ best model right now
```

**Why not just LiteLLM?** LiteLLM routes by model group. Forge routes by **request content**: it detects what you're doing (refactoring? debugging? writing docs?), filters out models that can't handle the request (no function calling? context too small?), keeps a conversation pinned to one model for prompt-cache hits, and scores the rest on live health, latency, and capability. Every routing decision is explainable — no black-box learned router.

> **Status: pre-release (v0.3-dev).** Core routing, failover, policies, and Anthropic-format support work. The dashboard UI is still in progress. Expect breaking changes.

## Features

- **OpenAI-compatible API** — `/v1/chat/completions` (streaming included), `/v1/embeddings`, `/v1/models`
- **Anthropic-compatible API** — `/v1/messages` with full streaming-event and tool-use conversion, so Claude Code can use any provider behind Forge
- **Policy engine** — manage *policies*, not models: first-match YAML rules pick candidate pools by task/client/context size, with hard constraints like `allow_paid: false`
- **Task-aware routing** — request analysis → capability matrix → best model, with `auto:refactor`-style hints when you know better
- **Hard compatibility filter** — requests needing tools / JSON mode / vision / large context never land on a model that can't serve them
- **Session affinity** — the same conversation stays on the same model (prompt-cache hits, consistent behavior); moves only on failover
- **Proactive rate limiting** — per-provider token buckets steer traffic away *before* the 429 happens; reactive cooldowns remain as the safety net
- **Self-healing failover** — 429 → instant cooldown (honors `Retry-After`) → next candidate; context-overflow → retry on a bigger-context model; all invisible to the client
- **Explainable decisions** — `POST /v1/route/explain` dry-runs any request and shows the matched policy, every exclusion reason, and the score table
- **Auto discovery & hot reload** — new provider models register at boot; edit `forge.yaml` and `POST /admin/reload` without dropping in-flight requests
- **Real metrics** — per-model latency (TTFT for streams), success rate, token usage, cost — stored locally in SQLite

## Quickstart

Requires Python 3.10+.

```bash
git clone https://github.com/WhiteJbb/forge.git
cd forge
python -m venv .venv
.venv/Scripts/pip install -e .     # Windows  (macOS/Linux: .venv/bin/pip install -e .)

export NVIDIA_API_KEY=nvapi-...    # or put it in .env
forge doctor                       # check keys & provider connectivity
forge start
```

Forge listens on `http://127.0.0.1:4000`. Check `http://127.0.0.1:4000/health` to see your model pool, or run `forge models` for an offline view. No config yet? `forge init` generates a `forge.yaml` from the API keys it finds in your environment.

> If the `forge` command collides with Foundry's on your PATH, use the `forge-gw` alias.

Configuration lives in [`forge.yaml`](forge.yaml) — providers, model tiers/capabilities, cooldowns, timeouts. It ships with a working NVIDIA free-tier setup.

## Connect your coding agent

Use model **`auto`** and let Forge choose, or `auto:debug` / `auto:refactor` / `auto:documentation` / `auto:testing` to force a task type. Unless you set `FORGE_API_KEY`, the API key can be any non-empty string.

**Cline (VS Code)** — Settings → API Provider: *OpenAI Compatible*
```text
Base URL: http://127.0.0.1:4000/v1
API Key:  forge
Model ID: auto
```

**Continue** — `config.json`:
```json
{
  "models": [{
    "title": "Forge",
    "provider": "openai",
    "model": "auto",
    "apiBase": "http://127.0.0.1:4000/v1",
    "apiKey": "forge"
  }]
}
```

**Aider**
```bash
export OPENAI_API_BASE=http://127.0.0.1:4000/v1
export OPENAI_API_KEY=forge
aider --model openai/auto
```

**Claude Code**
```bash
export ANTHROPIC_BASE_URL=http://127.0.0.1:4000
export ANTHROPIC_API_KEY=forge        # or your FORGE_API_KEY
```
Claude Code speaks the Anthropic Messages API; Forge converts it (tool use and streaming events included) and routes to whatever provider your policies choose.

Every response carries routing metadata in headers: `X-Forge-Model`, `X-Forge-Tier`, `X-Forge-Task`, `X-Forge-Attempt`.

## Endpoints

| Endpoint | Description |
| --- | --- |
| `POST /v1/chat/completions` | OpenAI-compatible chat, streaming + transparent failover |
| `POST /v1/messages` | Anthropic-compatible chat (Claude Code) |
| `POST /v1/route/explain` | Dry-run: matched policy, exclusion reasons, score table |
| `POST /v1/embeddings` | Embeddings (explicit model id) |
| `GET /v1/models` | Model pool + `auto` aliases |
| `GET /health` | Gateway + per-model health |
| `GET /v1/stats` | Usage / latency / cost metrics (JSON) |
| `GET /dashboard` | Dashboard data (JSON), incl. throttle state |
| `POST /admin/reload` | Hot-reload `forge.yaml` (loopback only) |
| `POST /admin/cooldown/{model}/clear` | Manually clear a cooldown (loopback only) |

## Privacy

Forge runs entirely on your machine. **No telemetry, ever.** Prompt and response bodies are never stored — metrics keep numbers only (latency, token counts, status codes). API keys are read from environment variables and never written to disk or logs.

## Roadmap

See [DESIGN.md](DESIGN.md) — up next: YAML policy engine ("route by policy, not by model"), Anthropic `/v1/messages` support for Claude Code, proactive rate limiting for free tiers, `/v1/route/explain` dry-run, and a web dashboard.

## License

[MIT](LICENSE)
