Metadata-Version: 2.4
Name: backtrack_route101
Version: 0.1.1
Summary: Local-first API observability for Python backends — live RPS / latency / errors / CPU / memory dashboard, ORM query tracking, outgoing-call dependency graph, optional LLM-powered diagnostics. Zero external services.
Project-URL: Homepage, https://github.com/arcitech/backtrack
Project-URL: Repository, https://github.com/arcitech/backtrack
Project-URL: Issues, https://github.com/arcitech/backtrack/issues
Project-URL: Changelog, https://github.com/arcitech/backtrack/blob/main/CHANGELOG.md
Author: Arcitech
License: MIT
License-File: LICENSE
Keywords: apm,dashboard,django,fastapi,local-first,metrics,monitoring,n+1,observability,sqlalchemy,tracing
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: Django
Classifier: Framework :: FastAPI
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Internet :: WWW/HTTP :: HTTP Servers
Classifier: Topic :: Software Development :: Debuggers
Classifier: Topic :: System :: Monitoring
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: click>=8.1
Requires-Dist: fastapi>=0.110
Requires-Dist: httpx>=0.27
Requires-Dist: psutil>=5.9
Requires-Dist: uvicorn[standard]>=0.27
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == 'dev'
Requires-Dist: django>=4.2; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Requires-Dist: sqlalchemy>=2.0; extra == 'dev'
Requires-Dist: twine>=5.0; extra == 'dev'
Provides-Extra: django
Requires-Dist: django>=4.2; extra == 'django'
Provides-Extra: sqlalchemy
Requires-Dist: sqlalchemy>=2.0; extra == 'sqlalchemy'
Description-Content-Type: text/markdown

<!-- omit in toc -->
<div align="center">

# backtrack

**Local-first API observability for Python backends.**

Drop-in middleware → live dashboard at `localhost:9876` → zero external services.

[![PyPI](https://img.shields.io/pypi/v/backtrack?color=58a6ff)](https://pypi.org/project/backtrack_route101/)
[![Python](https://img.shields.io/pypi/pyversions/backtrack)](https://pypi.org/project/backtrack_route101/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![FastAPI](https://img.shields.io/badge/FastAPI-supported-009688)](https://fastapi.tiangolo.com)
[![Django](https://img.shields.io/badge/Django-supported-092e20)](https://www.djangoproject.com)

</div>

---

```
┌──────────────────┐    HTTP POST    ┌────────────────────────┐    poll    ┌───────────────┐
│  Your app        │ ─── /ingest ──► │   backtrack collector  │ ◄────────► │   Dashboard   │
│  + middleware    │     batch       │   :9876                │   /api/*   │   (browser)   │
│  (FastAPI/Django)│                 │   ┌─ SQLite store      │            │   Plotly +    │
└─────────┬────────┘                 │   ├─ FastAPI server    │            │   Cytoscape   │
          │ on boot:                 │   ├─ static dashboard  │            └───────┬───────┘
          │ scan source code         │   └─ source scanner    │                    │
          └──────────────────────────►                        │           direct call (key
                                    └────────────────────────┘            in localStorage)
                                                                                   ▼
                                                                       ┌──────────────────────┐
                                                                       │  api.anthropic.com   │
                                                                       │  api.openai.com      │
                                                                       └──────────────────────┘
```

---

## Table of contents

- [Why backtrack](#why-backtrack)
- [What you get](#what-you-get)
- [Install](#install)
- [Quickstart: FastAPI](#quickstart-fastapi)
- [Quickstart: Django](#quickstart-django)
- [The dashboard, tour by tour](#the-dashboard-tour-by-tour)
- [AI assist (bring your own key)](#ai-assist-bring-your-own-key)
- [Distributed traces with correlation IDs](#distributed-traces-with-correlation-ids)
- [Configuration reference](#configuration-reference)
- [How it works](#how-it-works)
- [FAQ & troubleshooting](#faq--troubleshooting)
- [Roadmap](#roadmap)
- [License](#license)

---

## Why backtrack

You want to see what your Python backend is doing — RPS, latency, errors, slow SQL, who you're calling out to — **without** spinning up Datadog, signing up for Sentry, or shipping any data off your machine.

backtrack is one `pip install`, one middleware line, and one `backtrack start` away from a live dashboard at `http://127.0.0.1:9876`. It runs entirely on your laptop. Nothing leaves the box unless you click **analyze** and explicitly send a metric snapshot to your own LLM key.

> **Local-first means local-first.** Your API keys (if you use the AI assist) live in your browser's `localStorage`. The collector never sees them. No telemetry, no phone-home, no signup. The whole tool is one Python package and one SQLite file under `~/.backtrack/`.

---

## What you get

<table>
<tr>
<td width="33%">

### Live metrics
- Per-route RPS, error rate, avg / max latency
- Service-level CPU & memory (psutil)
- In-flight request counter
- 1m / 5m / 15m / 1h window selector
- Plotly time-series for RPS + latency

</td>
<td width="33%">

### Code-aware
- Static scan finds every FastAPI / Django route
- Tree view of folders → files → endpoints
- Highlights endpoints with no runtime traffic
- Click any folder → filter routes & analysis to it

</td>
<td width="33%">

### ORM tracking
- Counts SQL queries per request
- Tracks DB time per request
- Captures top-3 slowest queries
- N+1 hint: routes with ≥10 q/req turn red
- Django auto-instrumented; SQLAlchemy one-liner

</td>
</tr>
<tr>
<td>

### Error grouping
- Tracebacks captured with frame fingerprints
- Same bug from same place = one group
- Count, first seen, last seen, affected routes
- Click a group → expand traceback inline

</td>
<td>

### Outgoing HTTP
- Auto-instruments `httpx` + `requests`
- Per-host call counts, error rate, latency
- Bipartite graph: your routes → external hosts
- Edge width = volume, color = error rate

</td>
<td>

### AI assist (optional)
- "Analyze" button on every route + folder
- Paste an Anthropic or OpenAI key once
- Key stays in your browser — never sent to us
- Get root-cause + mitigation suggestions

</td>
</tr>
</table>

---

## Install

```bash
pip install backtrack_route101
```

That's it. The package ships with everything: collector, dashboard, FastAPI + Django middleware. The dashboard is plain HTML/JS/CSS — no Node build step required.

> **Python ≥ 3.10** required. Tested on 3.10, 3.11, 3.12.

---

## Quickstart: FastAPI

<details open>
<summary><b>Three lines of code, one terminal command</b></summary>

**1. Add the middleware to your app**

```python
# main.py
from fastapi import FastAPI
from backtrack.sdk.fastapi import BacktrackMiddleware

app = FastAPI()
app.add_middleware(BacktrackMiddleware, service="my-api")  # <─ this line

@app.get("/users/{user_id}")
def get_user(user_id: int):
    return {"id": user_id}
```

**2. Start your app as usual**

```bash
uvicorn main:app --reload
```

**3. Start the collector in another terminal**

```bash
backtrack start --scan . --scan-service my-api
```

**4. Open the dashboard**

```
http://127.0.0.1:9876
```

Hit a few endpoints in your app, watch the tiles light up.

</details>

<details>
<summary><b>Add SQLAlchemy query tracking (one line)</b></summary>

```python
from sqlalchemy import create_engine
from backtrack.sdk import instrument_sqlalchemy

engine = create_engine("postgresql://…")
instrument_sqlalchemy(engine)        # <─ call once per engine
```

Now the routes table shows `DB q` (avg queries per request) and `DB ms` (avg time spent in the DB) per route. Routes that fire ≥10 queries per request are flagged in red — a strong N+1 signal.

</details>

---

## Quickstart: Django

<details open>
<summary><b>One settings file edit</b></summary>

In `settings.py`:

```python
MIDDLEWARE = [
    "backtrack.sdk.django.BacktrackMiddleware",   # <─ add to the top
    "django.middleware.security.SecurityMiddleware",
    # …existing middleware…
]

BACKTRACK_SERVICE = "my-django-app"
# Optional:
# BACKTRACK_ENDPOINT = "http://127.0.0.1:9876/ingest"   # default
# BACKTRACK_INSTRUMENT_OUTGOING = True                  # default
```

Then in another terminal:

```bash
backtrack start --scan /path/to/your/project --scan-service my-django-app
python manage.py runserver
```

That's all. Django's ORM is auto-instrumented — every `Model.objects.…` shows up in the per-route DB columns. Class-based views, function-based views, `path()` and `re_path()` URLconfs are all picked up by the source scanner.

</details>

---

## The dashboard, tour by tour

Open `http://127.0.0.1:9876` after `backtrack start`. From top to bottom:

<details open>
<summary><b>Service & window selector</b></summary>

- **service** — which app to look at. Pick from the dropdown (populated by services that have sent heartbeats or been `--scan`ed).
- **window** — 1m / 5m / 15m / 1h. All charts and aggregates respect this.
- **settings** — AI assist key configuration (see below).

</details>

<details>
<summary><b>The seven tiles</b></summary>

| Tile | Meaning |
|---|---|
| **RPS** | Requests per second across the window |
| **error rate** | % of responses with status ≥ 500 or an unhandled exception |
| **avg latency** | Average response time |
| **max latency** | Slowest response in the window |
| **in-flight** | Currently open requests (from service heartbeat) |
| **CPU** | App process CPU %, averaged across pids of the same service |
| **memory** | App process RSS, summed across pids |

</details>

<details>
<summary><b>RPS / latency charts</b></summary>

Two Plotly charts updated every 2 seconds. The left one overlays RPS (blue, left axis) and error rate (red dotted, right axis). The right one is average latency over time.

</details>

<details>
<summary><b>Project section — tree & graph tabs</b></summary>

**tree** — folder hierarchy of your project. Each file shows the routes defined in it; each route shows its live RPS / error / avg-latency if it's been hit. Hover a folder → click the "filter" chip to scope the routes table and AI assist to that folder only.

**graph** — bipartite **routes → external hosts** view. Your routes on the left (sized by outbound volume), the external services they call on the right. Edges show traffic and error rate. Tells you "what does this service depend on, and where does it break?" — the question a routes table can't show.

</details>

<details>
<summary><b>Routes table</b></summary>

| Column | What it means |
|---|---|
| **method / route** | HTTP method + route template (with placeholders) |
| **RPS** | Requests per second |
| **err %** | Status ≥ 500 or exception |
| **avg ms / max ms** | Response latency |
| **DB q** | Average SQL queries per request (red if ≥ 10) |
| **DB ms** | Average DB time per request |
| **out** | Average outgoing HTTP calls per request |
| **total** | Total request count in window |
| **analyze** | Ask the LLM to diagnose this route |

</details>

<details>
<summary><b>Errors (grouped)</b></summary>

Sentry-style error grouping. Identical tracebacks from the same code path collapse into one row. Click a row to expand the sample traceback. Group counts, first/last seen, affected routes all shown.

</details>

<details>
<summary><b>Outgoing HTTP</b></summary>

Flat table of every external host your app called, with counts, error rate, and latency. Populates as soon as your app makes a request via `httpx` or `requests`.

</details>

---

## AI assist (bring your own key)

Click **settings** in the top-right, pick a provider (Anthropic or OpenAI), paste your key, optionally name a model, click save.

```
Your key is stored in this browser's localStorage only.
The backtrack collector never sees it — analyze requests
go directly from this page to the provider you choose.
```

Then anywhere you see an **analyze** button:
- Each route row in the table
- Folder click → "analyze folder" in the banner

The browser fetches a JSON snapshot (latency, errors, DB stats, outgoing-call breakdown) from your local collector, builds a prompt that asks for root causes + mitigations + robustness improvements, and POSTs directly to `api.anthropic.com` / `api.openai.com` with your key. The collector logs zero of this. If you `tcpdump localhost:9876`, you'll see ingest traffic and dashboard polls — never AI calls.

**Works with any model in either family** — including o1/o3/gpt-5 (reasoning models that require `max_completion_tokens`) and any Claude 3.x or 4.x.

---

## Distributed traces with correlation IDs

If your service calls another service that's also instrumented, you can stitch the chain together:

```python
import httpx
from backtrack.sdk import current_headers

async with httpx.AsyncClient() as client:
    # Merge backtrack's trace headers into your outgoing call:
    r = await client.get(other_url, headers={**current_headers(), **other_headers})
```

That's the only line you need to write. The downstream service will:
- See this hop as its `parent_id`
- Stay on the same `correlation_id`
- Show up linked in the data store

Backtrack uses three headers (simplified W3C trace context):
- `X-Request-ID` — this hop's id (generated if absent)
- `X-Correlation-ID` — chain id, shared across all hops
- `X-Parent-Request-ID` — the request_id of the caller

You can query the chain via `GET /api/route_chains?service=…&w=…`.

---

## Configuration reference

<details open>
<summary><b>FastAPI middleware</b></summary>

```python
app.add_middleware(
    BacktrackMiddleware,
    service="my-api",                              # required: name shown in dashboard
    endpoint="http://127.0.0.1:9876/ingest",       # default, override for non-standard ports
    instrument_outgoing=True,                       # auto-patch httpx + requests
)
```

</details>

<details>
<summary><b>Django settings</b></summary>

```python
MIDDLEWARE = ["backtrack.sdk.django.BacktrackMiddleware", …]

BACKTRACK_SERVICE = "my-django-app"
BACKTRACK_ENDPOINT = "http://127.0.0.1:9876/ingest"  # optional
BACKTRACK_INSTRUMENT_OUTGOING = True                  # optional (default True)
```

</details>

<details>
<summary><b>Environment variables (SDK side)</b></summary>

| Variable | Default | What it does |
|---|---|---|
| `BACKTRACK_ENDPOINT` | `http://127.0.0.1:9876/ingest` | Where the SDK pushes metric batches |

</details>

<details>
<summary><b>Collector CLI</b></summary>

```bash
backtrack start \
    --host 127.0.0.1 \             # bind host (KEEP localhost unless you really know)
    --port 9876 \                  # bind port
    --db ~/.backtrack/backtrack.db \   # SQLite database location
    --retention 3600 \             # seconds of raw events to keep
    --scan ./my-project \          # source path(s) to scan at startup (repeatable)
    --scan-service my-api          # service name for the corresponding --scan (positional)
```

Multiple `--scan` flags can be paired with multiple `--scan-service` flags positionally.

</details>

<details>
<summary><b>SQLAlchemy</b></summary>

```python
from backtrack.sdk import instrument_sqlalchemy

instrument_sqlalchemy(engine)   # call once per engine, idempotent
```

</details>

---

## How it works

<details>
<summary><b>Per-request overhead</b></summary>

| Step | Cost |
|---|---|
| Middleware on hot path | ~100 µs (lock + list append + status capture) |
| SDK background flush | 1 s interval, ~5–20 ms per batch, async to your app |
| Dashboard poll | 2 s interval, 5 parallel REST calls, ~10–50 ms each |

The middleware **never blocks on the network**. The background reporter buffers events in memory (5,000-event cap) and POSTs them to the collector. If the collector is down, batches are dropped. Your app is never slowed down or blocked by backtrack.

</details>

<details>
<summary><b>Storage</b></summary>

One SQLite file at `~/.backtrack/backtrack.db`. WAL mode, three tables:

- `events` — one row per request (retained for `--retention` seconds, default 1 h)
- `outgoing_calls` — one row per outbound HTTP
- `service_state` — last heartbeat per `(service, host, pid)`
- `discovered_routes` — output of the source scanner

All aggregation runs on read — no precomputed rollups. Plenty fast for the volume a local-only tool sees.

</details>

<details>
<summary><b>Source scanning</b></summary>

The collector walks Python source files (skipping `.venv`, `node_modules`, `__pycache__`, etc.) and uses regex pickers for:

- **FastAPI**: `@app.get/post/…`, `@router.X`, `@app.api_route(methods=[…])`
- **Django**: `urls.py` files, `path()` / `re_path()` patterns, recursive `include()` resolution with prefix stitching, class-based view detection

Output goes into `discovered_routes` and powers the tree view + AI folder analysis.

</details>

<details>
<summary><b>Database query tracking</b></summary>

- **Django**: Auto-installed on first request — adds a wrapper to every `connection.execute_wrappers` list (existing connections plus any future ones via the `connection_created` signal).
- **SQLAlchemy**: Listens on the engine's `before_cursor_execute` / `after_cursor_execute` events. You call `instrument_sqlalchemy(engine)` once per engine at startup.

SQL strings are normalized (literal values, IN-lists, numbers → `?`) so identical query templates collapse — N+1 patterns become visible by their template-repeat count.

</details>

<details>
<summary><b>Error fingerprinting</b></summary>

When an exception bubbles out of the handler, we capture the full traceback and compute a blake2b digest of `filename + qualname + lineno` for every frame. Same exception from the same code path = same fingerprint = one group. Paths are normalized to be portable across machines (everything after the last `site-packages/`, `src/`, or `Lib/` segment).

</details>

<details>
<summary><b>Outgoing HTTP instrumentation</b></summary>

The middleware monkey-patches `httpx.Client.send`, `httpx.AsyncClient.send`, and `requests.Session.send` at construction time. Each call is timed, the host extracted, and an `OutgoingCall` record sent to the collector's separate `outgoing_calls` table. A per-request aggregate via contextvars also adds `outgoing_count` + `outgoing_total_ms` to the originating Event.

Caveats: only `httpx` and `requests` are instrumented today. `urllib`, `aiohttp`, raw sockets won't show up.

</details>

---

## FAQ & troubleshooting

<details>
<summary><b>The dashboard shows nothing for my service</b></summary>

1. Is the middleware actually loaded? Hit any endpoint, then check `http://127.0.0.1:9876/api/services` — your service name should appear.
2. Is the collector running? `backtrack start` printed its bind address.
3. Is `BACKTRACK_ENDPOINT` (env var) pointing at the wrong port?
4. Firewall? localhost-only by default, but worth checking.

</details>

<details>
<summary><b>"no outgoing HTTP calls recorded yet"</b></summary>

Either your app hasn't made an outbound call yet, or it uses a library backtrack doesn't instrument (`urllib`, `aiohttp`, etc.). The fastest way to verify: add one `httpx.get("https://httpbin.org/uuid")` call somewhere in a handler, hit that endpoint, refresh.

</details>

<details>
<summary><b>The graph view stays empty</b></summary>

The deps graph only draws edges from runtime outgoing-call data — same caveat as above. If your **outgoing HTTP** table at the bottom is populated but the graph is empty, the service dropdown is set to "all" instead of a specific service.

</details>

<details>
<summary><b>Dashboard shows old code after I updated</b></summary>

Hard-reload (Ctrl+F5 / Ctrl+Shift+R). Static files have no cache headers, but the browser's session cache can hold onto the old JS until you force a refresh.

</details>

<details>
<summary><b>Schema error after upgrade ("no such column: …")</b></summary>

Run once to migrate, or nuke and start fresh:

```bash
# Migrate (automatic on next start, just re-run)
backtrack start

# Or wipe and recreate
rm ~/.backtrack/backtrack.db*
backtrack start
```

</details>

<details>
<summary><b>Windows: "Exception in callback _ProactorBasePipeTransport"</b></summary>

A benign asyncio quirk on Windows that backtrack already suppresses. If you're seeing it, you're on an old version — `pip install -U backtrack` and restart the collector.

</details>

<details>
<summary><b>Is it safe to bind --host 0.0.0.0?</b></summary>

**No.** The collector has zero authentication. Anyone with network access can read your metrics, scan your filesystem (`POST /api/scan`), and re-trigger source scans. Bind to `127.0.0.1` (the default) unless you've put it behind a real auth proxy.

</details>

<details>
<summary><b>Does this slow down my app?</b></summary>

The middleware adds ~100 µs to each request (one lock acquisition + list append + status capture). The actual transport runs on a background thread and never blocks the request path. If the collector is unreachable, batches drop silently — your app stays up.

</details>

---

## Roadmap

Things being considered, not promised:

- [ ] Percentile latencies (p50 / p95 / p99) — avg/max hides tail issues
- [ ] Background-task instrumentation (Celery, Dramatiq, RQ)
- [ ] Longer retention with downsampled rollups (7–30 day history)
- [ ] Slack / webhook alerts on rules (error rate > 5% for 5 min …)
- [ ] Local LLM provider (Ollama) for AI assist without an API key
- [ ] Per-user / per-API-key segmentation tags
- [ ] OpenTelemetry ingestion (accept OTLP from non-Python services)
- [ ] Trace explorer (waterfall view for one full chain)

Open an issue if there's something you'd actually use.

---

## License

[MIT](LICENSE) — Copyright (c) 2026 Arcitech.
