Black-box LLM inference profiler

Inference Autopsy

A local CLI that records OpenAI-compatible inference traces, diagnoses latency failures, generates static reports, replays workload shapes, and fails CI when a model deployment regresses.

View sample report GitHub repo PyPI package

What it proves

Trace-first benchmarking

Every request becomes JSONL evidence, including TTFT, TTFB, ITL, latency, stalls, statuses, and workload metadata.

Failure-aware diagnosis

Runs are grouped by profile and concurrency, then explained with evidence-backed labels like TTFT Pekka and Queue Queen.

CI regression gates

Baseline and candidate traces can be compared with explicit thresholds, giving automation a clean pass/fail contract.

Try it locally

pip install inference-autopsy
autopsy generate-fake --output examples/traces/fake.jsonl
autopsy report examples/traces/fake.jsonl --html report.html

The public report is generated from synthetic traces, so it is safe to share and does not expose prompts, endpoints, or API keys.