# Backstop

> In-process token budget enforcement, circuit breaking, and concurrency control for OpenAI and Anthropic SDK clients. The default `Backstop.wrap()` path injects a transport (`httpx` or `httpx2`, whichever the SDK uses) in-process — no network hop, no call-site changes. An optional OpenAI-compatible reverse proxy also ships as `backstop serve`; it is a separate mode, not what `wrap()` does.

- Documentation: /README.md
- Quick start: /docs/quickstart.md
- Install guide: /docs/install.md
- SDK compatibility matrix: /docs/sdk-matrix.md
- Compatibility details: /docs/compatibility.md
- Architecture: /docs/architecture.md
- Benchmarks: /docs/benchmarks.md
- Examples: /examples/
- Committed benchmark snapshot: /docs/benchmark-results-2026-07-20.md
- Concurrency and scale limits: /docs/concurrency.md
- Spend ledger, chargeback export and runaway-spend detection: /docs/ledger.md
- Threat model: /docs/threat-model.md

## Install (0.6.0 published)

The distribution is `backstop-ai`; the import is `backstop`. 0.6.0 is on PyPI
and on the `v0.6.0` GitHub Release. Note that `backstop-ai` on **npm** is a
separate, partial TypeScript port — not this package.

```bash
pip install backstop-ai               # OpenAI + base
pip install "backstop-ai[anthropic]"  # OpenAI + Anthropic
pip install "backstop-ai[redis]"      # shared/distributed budget
pip install "backstop-ai[otel]"       # OpenTelemetry metrics export
pip install "backstop-ai[fastapi]"    # gateway/sidecar mode
```

To work against unreleased work past the tag:

```bash
git clone https://github.com/RavaniRoshan/backstop.git
cd backstop && pip install -e ".[anthropic]"
```

## Quick start

```python
from openai import OpenAI
from backstop import Backstop
from backstop.exceptions import BudgetExceededError

client = Backstop.wrap(OpenAI(), budget=50_000)
try:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": "Hello."}],
    )
except BudgetExceededError:
    print("Budget exhausted")
```

## Keyless proof (no API key needed)

```bash
backstop verify   # 8 checks, purely offline, exits 0
backstop demo     # side-by-side runaway loop comparison
```

## Priority admission

Exactly three priorities: `critical`, `default`, `background`. There is no
`normal`. The gate **queues and prioritises; it does not shed** — nothing is
cancelled. `starvation_after_seconds` (default 1.0) releases the oldest ticket
in any queue once it has waited that long, so a low-priority ticket cannot wait
forever. `queue_timeout` bounds the wait with an error instead.

## Retries

Wrapping sets the wrapped client to `max_retries=0`. Backstop performs the
retries itself, in the transport, with its own backoff and circuit state. Do
not expect the SDK's retry layer to still be active.

## SDK compatibility

- OpenAI: `>=2.37,<4` on Python 3.10–3.12 ✅
- Anthropic: `>=0.98,<2` on Python 3.10–3.12 ✅

Those are the *tested* ranges. CI runs one job on ubuntu-latest across
Python 3.10/3.11/3.12 × openai 2.37.0/3.14.0 × anthropic 0.99.0/1.6.0, plus a
single `openai`-latest cross-check. It does not install the `redis`, `otel`,
`fastapi` or `tokenizers` extras, does not run the TypeScript package, and has
no lint, typecheck, or coverage step. Below the floors the SDK catches
`BudgetExceededError` and re-raises it as `APIConnectionError`, hiding the
guardrail.

`backstop doctor` is a wrap-and-import smoke test only — it does not send a
request through the wrapped transport, and it exits 1 whenever `httpx2` is
absent, which is not a declared dependency. Trust `backstop verify`, not
`doctor`.

## CLIs

- `backstop verify` — proves enforcement offline with 8 checks; `--live` adds a `GET /models` auth probe
- `backstop demo` — side-by-side runaway loop (unprotected vs wrapped)
- `backstop doctor` — SDK versions and wrap path; see the caveats above
- `backstop benchmark` — overhead measurement; latencies are wall-clock, and the `error-storm` / `budget-hit` scenario counts depend on retry backoff and circuit cooldown, so only `burst` and `steady-state` reproduce exactly
- `backstop serve` — optional OpenAI-compatible gateway (needs the `fastapi` extra)
- `backstop metrics` — Prometheus endpoint
- `backstop real-openai` / `real-anthropic` — live smoke tests (require API key)

## Benchmarks

The only committed overhead figures are in
`docs/benchmark-results-2026-07-20.md`: 0.07 ms at p50, p95 and p99 over 1,000
requests against a local `httpx.MockTransport`. That snapshot records the date,
seed, mock transport and request count; it does **not** record CPU, OS, Python
version or SDK version. Re-measure with `backstop benchmark`. The overhead check
inside `verify` passes below a 5 ms p99 bound, not at a specific figure, and CI
runs the benchmark as a threshold-free smoke test.

## Agentic-installer quickstart

If you are an agent installing Backstop into a codebase:

1. Check SDK compatibility: OpenAI `>=2.37,<4`, Anthropic `>=0.98,<2`.
2. `pip install "backstop-ai[anthropic]"` (or install from source for unreleased work).
3. Integration: `client = Backstop.wrap(client, budget=50_000)` — one line.
4. Verify: `backstop verify` proves enforcement without any API key.
5. Catch `backstop.exceptions.BudgetExceededError` to handle budget exhaustion.
