Metadata-Version: 2.4
Name: agentic-llm-router
Version: 0.2.0
Summary: Cost-aware multi-provider LLM router with role tiers (cheap/mid/chief/audit).
Author: krivonosoff161
License-Expression: MIT
Project-URL: Homepage, https://github.com/krivonosoff161/llm-router
Project-URL: Repository, https://github.com/krivonosoff161/llm-router
Keywords: llm,openai,router,agents,cost,qwen,yandex
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: aiohttp>=3.9
Provides-Extra: dev
Requires-Dist: build>=1.2.2; extra == "dev"
Requires-Dist: pytest>=7; extra == "dev"
Requires-Dist: ruff>=0.6; extra == "dev"
Requires-Dist: setuptools>=77; extra == "dev"
Requires-Dist: wheel>=0.44; extra == "dev"
Dynamic: license-file

# llm-router

Ecosystem role and current integration status: [component roadmap](docs/component-roadmap.md).
The public cross-repository plan is owned by the
[Agentic Security Harness ecosystem roadmap](https://github.com/krivonosoff161/agentic-security-harness/blob/main/docs/ecosystem-roadmap.md).

[![Tests](https://github.com/krivonosoff161/llm-router/actions/workflows/tests.yml/badge.svg)](https://github.com/krivonosoff161/llm-router/actions/workflows/tests.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![Python 3.9+](https://img.shields.io/badge/python-3.9%2B-blue.svg)](https://www.python.org/)

**A tiny, dependency-light async LLM router with role tiers and per-call cost logging.**
One `call()` interface for the tested OpenAI-compatible request shape and a
separate Yandex AI Studio path. Provider compatibility depends on each endpoint's
current API contract and must be verified before use.

> The public library demonstrates a cheap-to-chief routing pattern with offline
> tests. It does not publish or verify a production deployment claim. No SDKs or
> models are hardcoded in the routing logic.

The repository now publishes a source-owned, offline invocation-receipt contract and is
therefore `contract_only` in the ecosystem. Its source tree builds the unique distribution
candidate `agentic-llm-router==0.2.0`, imported as `llm_router`. It is not yet published or
automatically activated by Harness.

> Supply-chain boundary: the generic PyPI name `llm-router` belongs to another project.
> Do not install or declare that coordinate for this repository. The only planned public
> distribution coordinate is `agentic-llm-router`.

---

## Why

In agentic systems most LLM calls are cheap bulk work (extract, classify, filter) and a few are high-stakes (the final decision). Paying flagship prices for everything is wasteful; juggling provider SDKs is annoying. `llm-router` gives you:

- **Role tiers** — `cheap` / `mid` / `chief` / `audit`, each mapped to a model via env. Route volume to `cheap`, escalate only candidates to `chief`.
- **Provider flexibility** — a custom `OPENAI_BASE_URL` can target endpoints that
  implement the tested request/response contract; Yandex AI Studio has a separate
  path. Provider identity, terms, availability, and exact compatibility are external
  gates.
- **Per-call cost** — every call returns token counts and cost in **USD + a configurable local currency** (set `LLM_FX` / `LLM_CCY`). Aggregate the dicts to a budget log.
- **Budget helpers** — aggregate usage records, check a daily cap, and estimate savings
  versus sending the same tokens to the `chief` model.
- **Resilience** — retries on `429` / `5xx` with exponential backoff.
- **Canonical receipt contract** — strict, digest-bound attempt, usage, pricing, and FX
  evidence without credentials, endpoints, prompts, output text, response bodies, or
  exception messages.

---

## Features

- Single `async call(role, system, user) -> (text | None, usage)` interface.
- Four configurable role tiers, models set per provider via env.
- OpenAI-compatible **and** Yandex AI Studio providers.
- `json_mode=True` → adds `response_format={"type":"json_object"}` (OpenAI-compatible).
- Cost estimation from an override-able price table (`LLM_PRICE_<MODEL>_IN/OUT_USD_PER_1M`).
- Budget helpers for logs you own: `summarize_usage`, `budget_status`, and
  `build_savings_report`.
- Zero secrets cached at import — all config read live from env.
- ~150 LOC, one runtime dependency (`aiohttp`).

---

## Install

```bash
git clone https://github.com/krivonosoff161/llm-router
cd llm-router
python -m build
python -m pip install dist/agentic_llm_router-0.2.0-py3-none-any.whl
```

For editable development use `python -m pip install -e .[dev]`. Requires **Python 3.9+**.
CI builds and installs the exact wheel on Linux and Windows. Harness `main` declares a
source-only `router` extra using the unique `agentic-llm-router` distribution name, but
this package is not on PyPI and published Harness `v1.3.0` metadata does not contain that
extra. Public `pip install agentic-security-harness[router]` support is therefore
unavailable; package publication and newer Harness package metadata remain separate release gates.

---

## Quickstart

```python
import asyncio
from llm_router import call

async def main():
    text, usage = await call("cheap", "You are concise.", "Name 3 primary colors.")
    print(text)
    print(usage)   # {provider, model, role, input_tokens, output_tokens,
                   #  total_tokens, cost_usd, cost_local, currency}

asyncio.run(main())
```

Set at least a provider + key first (see **Configuration**). For OpenAI:

```bash
export OPENAI_API_KEY=sk-...
```

---

## Providers

| Provider | Set | Auth |
|---|---|---|
| **OpenAI** | `LLM_PROVIDER=openai` (default), `OPENAI_API_KEY` | `Bearer` |
| **Alibaba Qwen** | `OPENAI_BASE_URL=<dashscope compatible-mode/v1>` + `OPENAI_API_KEY` | `Bearer` |
| **OpenRouter / Together / Ollama / vLLM** | `OPENAI_BASE_URL=<their /v1>` + `OPENAI_API_KEY` | `Bearer` |
| **Yandex AI Studio** | `LLM_PROVIDER=yandex`, `YANDEX_API_KEY`, `YANDEX_FOLDER_ID` | `Api-Key` |

> The base URL must NOT include `/chat/completions` — the router appends it.
> **Yandex** requires `YANDEX_FOLDER_ID` (or an explicit `YANDEX_<ROLE>_MODEL`); otherwise `model_for` raises a clear configuration error (fail-fast) instead of sending an empty model.

---

## Roles

```python
from llm_router import call, model_for

model_for("cheap")   # -> e.g. "gpt-4o-mini" (or your LLM_CHEAP_MODEL)
model_for("chief")   # -> e.g. "gpt-4o"

# pattern: cheap for volume, chief only when it matters
facts, u1 = await call("cheap", EXTRACT_PROMPT, raw_text)
if looks_important(facts):
    verdict, u2 = await call("chief", DECIDE_PROMPT, facts, json_mode=True)
```

---

## Cost logging

```python
text, usage = await call("cheap", sys, user)
# usage["cost_usd"]   -> e.g. 0.0001
# usage["cost_local"] -> cost_usd * LLM_FX
# usage["currency"]   -> LLM_CCY (e.g. "RUB")
```

Append each `usage` to a JSONL file and you have a per-call budget log. Prices come from a small built-in table and are **illustrative** — override per model:

```bash
export LLM_PRICE_GPT_4O_MINI_IN_USD_PER_1M=0.15
export LLM_PRICE_GPT_4O_MINI_OUT_USD_PER_1M=0.60
```

Summarize a batch of usage records:

```python
from llm_router import summarize_usage, budget_status, build_savings_report

usages = [u1, u2]  # dicts returned by call()
print(summarize_usage(usages).as_dict())
print(budget_status(usages, limit_usd=1.00).as_dict())
print(build_savings_report(usages, counterfactual_role="chief").as_dict())
```

`LLM_BUDGET_USD_DAY` can be used as a default budget cap for `budget_status(...)`.
The router stays stateless; you decide where the JSONL budget log lives.

## Canonical invocation receipts

[`router-invocation-receipt-v1.0`](docs/invocation-receipt.md) is a separate offline
interchange surface for already-observed sanitized values. It uses canonical UTF-8 JSON,
domain-separated content identities, contiguous attempt accounting, strict token totals,
and integer nano-unit cost arithmetic. Pricing and FX inputs bind caller-supplied source
artifact digests; those digests are evidence references, not authenticity proofs.

The receipt builder never calls a provider and the existing `call()` return value is
unchanged. A receipt contains digests of request, response, output, model, and producer
identity—not their raw bytes. It always declares `invoice_authoritative=false` and
`operational_authority=none`.

Deterministic hashes are content-minimizing, not anonymizing: they remain linkable and can
be guessed when the source space is small. A receipt is not automatically safe to publish.

```python
from llm_router import InvocationAttemptV1

attempt = InvocationAttemptV1(
    attempt_index=1,
    outcome="network_error",
    http_status=None,
    reason_code="provider.network_error",
    response_payload_sha256=None,
)
# Supply only already-observed sanitized values; see docs/invocation-receipt.md.
```

---

## Configuration (env)

| Variable | Default | Purpose |
|---|---|---|
| `LLM_PROVIDER` | `openai` | `openai` (compatible) or `yandex` |
| `OPENAI_API_KEY` | — | key for the OpenAI-compatible endpoint |
| `OPENAI_BASE_URL` | `https://api.openai.com/v1` | point at Alibaba/OpenRouter/Ollama/... |
| `LLM_CHEAP_MODEL` / `LLM_MID_MODEL` / `LLM_CHIEF_MODEL` / `LLM_AUDIT_MODEL` | gpt-4o-mini / gpt-4o-mini / gpt-4o / gpt-4o | role → model |
| `YANDEX_API_KEY`, `YANDEX_FOLDER_ID` | — | Yandex AI Studio |
| `YANDEX_<ROLE>_MODEL` | wraps `gpt://<folder>/<name>/latest` | override a Yandex role model URI |
| `LLM_FX` | `1.0` | USD → local currency multiplier |
| `LLM_CCY` | `USD` | local currency label |
| `LLM_DEFAULT_TIMEOUT` | `60` | per-call timeout (s) |
| `LLM_MAX_RETRIES` | `2` | retries on 429/5xx |
| `LLM_PRICE_<MODEL>_IN/OUT_USD_PER_1M` | from table | override price per model |
| `LLM_BUDGET_USD_DAY` | unset | optional cap used by budget helpers |

See [.env.example](.env.example).

---

## Examples & tests

```bash
python examples/basic.py          # one call
python examples/role_tiers.py     # cheap vs chief + cost
python -m pytest -q               # offline unit tests (no network)
```

What each example shows and what it does *not* prove: [examples/README.md](examples/README.md).

---

## Docs

- [Component roadmap](docs/component-roadmap.md) — source-owned ecosystem role,
  platform evidence, historical projections, and integration gates.
- [Project map](docs/project-map.md) — modules, what exists today vs not included, reviewer checklist.
- [Use cases](docs/use-cases.md) — who this is for, practical workflows, limitations.
- [Operating model](docs/operating-model.md) — role budgets, usage records, escalation gates, and residual risk.
- [Invocation receipt V1](docs/invocation-receipt.md) — canonical codec, attempt state
  machine, fixed-point arithmetic, privacy boundary, and non-claims.
- [Harness ecosystem roadmap](https://github.com/krivonosoff161/agentic-security-harness/blob/main/docs/ecosystem-roadmap.md)
  — the canonical public ordering for cross-repository integration work.

---

## Limitations / non-goals

- Chat completions only (no streaming, embeddings, tools/function-calling, vision — kept intentionally small).
- One system + one user message per call (no multi-turn history helper).
- The price table is illustrative; confirm real prices with your provider.
- Not a full framework — it's a focused routing + cost-logging utility you drop into your own agent loop.
- Not the portfolio flagship, policy authority, or security boundary. Larger
  systems own their own validation, authorization, storage, and safety rules.

---

## License

MIT — see [LICENSE](LICENSE).
