Metadata-Version: 2.5
Name: signalfloor
Version: 0.1.0
Summary: Find out if a drop in your LLM eval scores is a real regression or just noise.
Project-URL: Repository, https://github.com/dch2r/signalfloor
Project-URL: Issues, https://github.com/dch2r/signalfloor/issues
Project-URL: Changelog, https://github.com/dch2r/signalfloor/blob/main/CHANGELOG.md
Author: Diwesh Chaurasia
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: bootstrap,evals,evaluation,llm,llm-as-judge,regression testing,statistics
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Testing
Classifier: Typing :: Typed
Requires-Python: >=3.11
Requires-Dist: numpy>=1.26
Description-Content-Type: text/markdown

# signalfloor

Find out if a drop in your LLM eval scores is a real regression or just noise.

```bash
pip install signalfloor
```

Three commands, each reading eval results as JSON Lines with one test case per line:

```bash
signalfloor compare   baseline.jsonl candidate.jsonl   # is this change real or noise?
signalfloor calibrate judge.jsonl human.jsonl          # can you trust your LLM judge?
signalfloor flaky     run1.jsonl run2.jsonl run3.jsonl # which cases are flaky, and what is the noise floor?
```

Each command exits with code 1 when its check fails, so you can use it to fail a CI job.

Full docs and examples are on [GitHub](https://github.com/dch2r/signalfloor).
