Metadata-Version: 2.4
Name: perferox
Version: 0.1.0
Summary: Agentic benchmark explorer for SGLang.
Author: Naman Chetwani
Keywords: sglang,benchmark,fuzzing,inference,agents
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Requires-Dist: langchain
Requires-Dist: langgraph
Requires-Dist: langchain-openai
Requires-Dist: textual
Requires-Dist: paramiko
Requires-Dist: pydantic
Requires-Dist: numpy
Requires-Dist: sentence-transformers>=5.6.0

<h1 align="center">perferox</h1>

Perferox is a small agentic benchmark explorer for inference engines and ML
systems performance. The first target is SGLang.

This is alpha software. It is not ready for common use yet. Expect rough edges,
missing docs, changing APIs, and places where you still need to understand the
repo before trusting a run.

The shape is intentionally simple: give an agent a bounded objective, let it
delegate benchmark workers, run experiments through narrow tools, and save the
parts worth keeping. The point is not to make a giant benchmark platform. The
point is to make it easy to poke at weird corners of an inference stack without
losing track of what already happened.

The first use case is SGLang serving benchmarks. Perferox is meant to try
different models, backends, cache settings, request patterns, chips, and setup
conditions, then preserve the useful results in SQLite. It records the normal
perf signals, like request throughput, token throughput, TTFT, TPOT, cache hit
rate, accept length, and error rate. It also has a place to store human-readable
anomalies, because a lot of the interesting bugs are not just "number went down"
but "this specific setup behaved strangely in a way a person should inspect."

The important constraint is that the host owns state. Agents do not choose run
IDs, write SQLite directly, or decide whether a benchmark should bypass caps.
Real experiments go through a benchmark tool. Started runs get logged. Failed
runs still count once they actually start. The database is the source of truth
for runs, experiments, anomalies, trace references, and compact exploration
state.

Current pieces:

- a `perferox` CLI
- a Textual TUI for starting and stopping exploration
- a LangGraph main coordinator
- tmux-wrapped benchmark subagents
- Paramiko remote sessions
- structured SGLang `bench_serving` command construction and parsing
- SQLite-backed runs, experiments, anomalies, notifications, and ExplorerState
- local SGLang docs search plumbing

This is still early software. The goal is to keep the code small enough that it
stays easy to inspect and change. The preferred path is boring Python, explicit
state, small tools, and very few abstractions.

```bash
uvx perferox --help
```
