Metadata-Version: 2.4
Name: inferlens
Version: 0.1.0.dev0
Summary: Engine-neutral observability for LLM inference: record scheduler, batching, and KV-cache behavior and explore it as an interactive timeline.
Project-URL: Homepage, https://github.com/rabrooks/inferlens
Author: InferLens Team
License-Expression: Apache-2.0
License-File: LICENSE
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: System :: Monitoring
Requires-Python: <3.15,>=3.10
Provides-Extra: vllm
Requires-Dist: msgspec>=0.18; extra == 'vllm'
Requires-Dist: pyzmq>=25.0; extra == 'vllm'
Description-Content-Type: text/markdown

# InferLens

**Engine-neutral observability for LLM inference engines.** Record what vLLM
(and soon SGLang) is actually doing internally — scheduling, batching,
KV-cache behavior — and explore it as an interactive timeline that answers
*"why was this request slow?"*

> **Status: pre-alpha.** The trace schema and vLLM collector are under
> active development. Nothing here is stable yet — including the trace
> format.

## Why

Today's tooling for inference engines is bimodal: Grafana dashboards of
aggregate metrics (too high-level to explain a single slow request) and
kernel-level profiler traces (too low-level to see scheduler semantics).
InferLens targets the missing middle layer:

- per-step scheduler state: queue depths, batch composition, preemptions
- KV-cache behavior: usage, prefix-cache hits, evictions
- per-request lifecycle: queued → prefill → decode, with token timings

…captured with a pip-installed plugin (no engine fork), written to a portable
trace file, and rendered in a local viewer.

## How it will work

```text
vLLM ──(stat-logger plugin + KV events)──►  trace file (.ilens.gz)  ──►  inferlens view
SGLang ──(collector, planned)───────────►       one shared schema        interactive timeline
```

```bash
pip install inferlens
inferlens record -o run.ilens.gz -- vllm serve Qwen/Qwen2.5-1.5B-Instruct   # in development
inferlens info run.ilens.gz
inferlens view run.ilens.gz                                                  # in development
```

## Development

```bash
uv sync
uv run pre-commit install
uv run pytest
```

See [CONTRIBUTING.md](CONTRIBUTING.md) for coding standards (Google style,
DCO sign-off, vLLM-style PR conventions) and
[docs/TRACE_SPEC.md](docs/TRACE_SPEC.md) for the trace format.

## License

[Apache-2.0](LICENSE)
