# Backstop

> In-process token budget enforcement, circuit breaking, and concurrency control for OpenAI and Anthropic SDK clients. The default `Backstop.wrap()` path injects a transport (`httpx` or `httpx2`, whichever the SDK uses) in-process — no network hop, no call-site changes. An optional OpenAI-compatible reverse proxy also ships as `backstop serve`; it is a separate mode, not what `wrap()` does.

- Documentation: /README.md
- Quick start: /docs/quickstart.md
- Install guide: /docs/install.md
- SDK compatibility matrix: /docs/sdk-matrix.md
- Compatibility details: /docs/compatibility.md
- Architecture: /docs/architecture.md
- Benchmarks: /docs/benchmarks.md
- Examples: /examples/
- Committed benchmark snapshot: /docs/benchmark-results-2026-10-20.md
- Concurrency and scale limits: /docs/concurrency.md
- Spend ledger, chargeback export and runaway-spend detection: /docs/ledger.md
- Dollar budgets: /docs/usd-budgets.md
- Runaway-spend detection: /docs/detection.md
- Budget alerts and webhooks: /docs/alerts.md
- Framework adapters: /docs/frameworks.md
- Built-in dashboard: /docs/dashboard.md
- Threat model: /docs/threat-model.md
- Use cases and walkthroughs: /usecases/

## Install (0.8.0 published)

The distribution is `backstop-ai`; the import is `backstop`. 0.8.0 is on PyPI
and on the `v0.8.0` GitHub Release. Note that `backstop-ai` on **npm** is a
separate, partial TypeScript port — not this package.

```bash
pip install backstop-ai               # OpenAI + base
pip install "backstop-ai[anthropic]"  # OpenAI + Anthropic
pip install "backstop-ai[redis]"      # shared/distributed budget
pip install "backstop-ai[otel]"       # OpenTelemetry metrics export
pip install "backstop-ai[fastapi]"    # gateway/sidecar mode
```

To work against unreleased work past the tag:

```bash
git clone https://github.com/RavaniRoshan/backstop.git
cd backstop && pip install -e ".[anthropic]"
```

## Quick start

```python
from openai import OpenAI
from backstop import Backstop
from backstop.exceptions import BudgetExceededError

client = Backstop.wrap(OpenAI(), budget=50_000)
try:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": "Hello."}],
    )
except BudgetExceededError:
    print("Budget exhausted")
```

A ceiling in dollars, on top of the token one:

```python
from decimal import Decimal
from backstop import Backstop

client = Backstop.wrap(OpenAI(), budget=50_000, budget_usd=Decimal("5.00"))
```

A dollar ceiling is a *second* ceiling, not a replacement: a request is
admitted only if it fits under both. It requires the price catalog, so it
turns the spend ledger on. See /docs/usd-budgets.md.

## Keyless proof (no API key needed)

```bash
backstop verify   # 8 checks, purely offline, exits 0
backstop demo     # side-by-side runaway loop comparison
```

## Priority admission

Exactly three priorities: `critical`, `default`, `background`. There is no
`normal`. The gate **queues and prioritises; it does not shed** — nothing is
cancelled. `starvation_after_seconds` (default 1.0) releases the oldest ticket
in any queue once it has waited that long, so a low-priority ticket cannot wait
forever. `queue_timeout` bounds the wait with an error instead.

A queue timeout raises `AdmissionTimeoutError`, which is both a
`BackstopError` and a builtin `TimeoutError`, so either handler catches it.

## Retries

Wrapping sets the wrapped client to `max_retries=0`. Backstop performs the
retries itself, in the transport, with its own backoff and circuit state. Do
not expect the SDK's retry layer to still be active.

## SDK compatibility

- OpenAI: `>=2.37,<4` on Python 3.10–3.12 ✅
- Anthropic: `>=0.98,<2` on Python 3.10–3.12 ✅

Those are the *tested* ranges. CI runs one job on ubuntu-latest across
Python 3.10/3.11/3.12 × openai 2.37.0/3.14.0 × anthropic 0.99.0/1.6.0, plus a
single `openai`-latest cross-check. It lints with `ruff`, and it runs the
TypeScript package's typecheck, build and tests. It does not install the
`redis`, `otel`, `fastapi` or `tokenizers` extras, and it has no coverage
gate. Below the floors the SDK catches `BudgetExceededError` and re-raises it
as `APIConnectionError`, hiding the guardrail.

`backstop doctor` is a wrap-and-import smoke test only — it does not send a
request through the wrapped transport. Trust `backstop verify`, not
`doctor`.

## CLIs

Every command, and what it is for. All of them except `real-openai` and
`real-anthropic` run with no API key and no network.

**Prove it**

- `backstop verify` — proves enforcement offline with 8 checks; `--live` adds a `GET /models` auth probe
- `backstop demo` — side-by-side runaway loop (unprotected vs wrapped)
- `backstop doctor` — SDK versions and wrap path; see the caveats above
- `backstop benchmark` — overhead measurement; latencies are wall-clock, and the `error-storm` / `budget-hit` scenario counts depend on retry backoff and circuit cooldown, so only `burst` and `steady-state` reproduce exactly
- `backstop harness` — a local mock-provider load scenario

**Money**

- `backstop ledger demo|show|export` — the priced spend ledger: a synthetic run, a readable report, and a finance-grade CSV
- `backstop budget --ledger PATH --usd CEILING` — where the spend stands against a ceiling, and the headroom left
- `backstop detect --ledger PATH` — the four runaway-spend detectors over a ledger file
- `backstop forecast --ledger PATH --usd CEILING` — when the budget runs out, at the burn rate the ledger measured
- `backstop reconcile --ledger PATH --invoice PATH` — the ledger against a provider statement
- `backstop audit --path PATH` — verify the tamper-evident audit chain

**Run it**

- `backstop serve` — optional OpenAI-compatible gateway (needs the `fastapi` extra)
- `backstop metrics` — Prometheus endpoint
- `backstop dashboard` — the built-in dashboard for this process

**Live check**

- `backstop real-openai` / `real-anthropic` — live smoke tests (require API key)

Every report-taking command takes `--json` as well as the default Markdown.

## Benchmarks

The only committed overhead figures are in
`docs/benchmark-results-2026-10-20.md`: 0.097 ms at p50 for a 94-byte body
and 0.101 ms at p50 for a 4 KB body, over 2,000 requests against a local
`httpx.MockTransport`, with no network. The overhead is body-proportional, so
both sizes are published.

That snapshot records the host, the OS, the interpreter and the package
versions, so it can be checked on another machine. Re-measure with
`PYTHONPATH=src python3 benchmarks/local_overhead.py --requests 2000`. The
overhead check inside `verify` passes below a 5 ms p99 bound, not at a
specific figure, and CI runs the benchmark as a threshold-free smoke test.

## Agentic-installer quickstart

If you are an agent installing Backstop into a codebase:

1. Check SDK compatibility: OpenAI `>=2.37,<4`, Anthropic `>=0.98,<2`.
2. `pip install "backstop-ai[anthropic]"` (or install from source for unreleased work).
3. Integration: `client = Backstop.wrap(client, budget=50_000)` — one line.
4. Verify: `backstop verify` proves enforcement without any API key.
5. Catch `backstop.exceptions.BudgetExceededError` to handle budget exhaustion.
6. If the budget is a *money* budget, pass `budget_usd=Decimal("...")` as well —
   and read /docs/usd-budgets.md, because a dollar ceiling turns the ledger on
   and refuses models it cannot price.
