Metadata-Version: 2.5
Name: evalsi
Version: 0.1.0
Summary: One entrypoint for evaluating ML models, LLMs and agents.
Project-URL: Homepage, https://abhishek-rnjn.github.io/Evals.si/
Project-URL: Documentation, https://abhishek-rnjn.github.io/Evals.si/
Project-URL: Repository, https://github.com/Abhishek-Rnjn/Evals.si
Project-URL: Issues, https://github.com/Abhishek-Rnjn/Evals.si/issues
Project-URL: Changelog, https://github.com/Abhishek-Rnjn/Evals.si/blob/main/CHANGELOG.md
License-Expression: Apache-2.0
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Requires-Dist: httpx>=0.27
Requires-Dist: protobuf>=7.35
Requires-Dist: pyyaml>=6.0
Provides-Extra: analytics
Requires-Dist: duckdb>=1.1; extra == 'analytics'
Provides-Extra: anthropic
Requires-Dist: anthropic>=1.0; extra == 'anthropic'
Provides-Extra: grpc
Requires-Dist: grpcio>=1.84; extra == 'grpc'
Provides-Extra: hf
Requires-Dist: datasets>=2.18; extra == 'hf'
Provides-Extra: jsonschema
Requires-Dist: jsonschema>=4.0; extra == 'jsonschema'
Provides-Extra: math
Requires-Dist: sympy>=1.13; extra == 'math'
Provides-Extra: server
Requires-Dist: grpcio-health-checking>=1.84; extra == 'server'
Requires-Dist: grpcio>=1.84; extra == 'server'
Description-Content-Type: text/markdown

# evalsi

The Python SDK and CLI for [Evals.si](https://github.com/Abhishek-Rnjn/Evals.si): one entrypoint for evaluating ML models, LLMs and agents.

```bash
pip install evalsi                 # core
pip install 'evalsi[anthropic]'    # Claude as a judge
pip install 'evalsi[hf]'           # hf:// datasets
```

```bash
evalsi eval --data qa.jsonl --evaluators exact-match,numeric-match
evalsi catalog
```

```python
import evalsi

result = evalsi.evaluate(
    "qa.jsonl",
    ["exact-match", "llm-judge"],
    judge=evalsi.JudgeConfig(provider="anthropic", model="claude-opus-5-5"),
)
print(result.table())
```

See the repository README for the full guide.
