Metadata-Version: 2.5
Name: verifiers
Version: 0.4.1.dev5
Summary: Verifiers: Environments for LLM Reinforcement Learning
Project-URL: Homepage, https://github.com/primeintellect-ai/verifiers
Project-URL: Documentation, https://github.com/primeintellect-ai/verifiers
Project-URL: Repository, https://github.com/primeintellect-ai/verifiers.git
Project-URL: Issues, https://github.com/primeintellect-ai/verifiers/issues
Author-email: William Brown <williambrown97@gmail.com>
License-Expression: MIT
License-File: LICENSE
Keywords: agentic-rl,agents,environments,eval,grpo,harness,llm,multi-turn,reinforcement-learning,rl,rlvr,tool-use,train,verifiers
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: <3.15,>=3.11
Requires-Dist: aiohttp>=3.14.1
Requires-Dist: anthropic>=1.0.0
Requires-Dist: gepa>=0.0.6
Requires-Dist: h11>=0.16.0
Requires-Dist: httpx2>=2.12.0
Requires-Dist: httpx>=0.27.0
Requires-Dist: loguru>=0.7.0
Requires-Dist: math-verify>=0.8.0
Requires-Dist: mcp==2.0.0
Requires-Dist: msgpack>=1.1.2
Requires-Dist: numpy>=2.1.0
Requires-Dist: openai<3.0.0,>=2.54.0
Requires-Dist: prime-pydantic-config[toml]>=0.4.3
Requires-Dist: prime-runs>=0.1.4
Requires-Dist: prime-sandboxes>=0.4.2
Requires-Dist: prime-tunnel>=0.1.14
Requires-Dist: pydantic>=2.12.3
Requires-Dist: pyzmq>=27.1.0
Requires-Dist: renderers[multimodal]>=0.1.12
Requires-Dist: rich>=11.0.0
Requires-Dist: tenacity>=8.5.0
Requires-Dist: tomli-w>=1.0.0
Requires-Dist: typing-extensions>=4.12.2
Requires-Dist: uvicorn>=0.52.0
Provides-Extra: e2b
Requires-Dist: e2b<3,>=2.35.0; extra == 'e2b'
Provides-Extra: harbor
Requires-Dist: harbor==0.24.0; (python_full_version >= '3.12') and extra == 'harbor'
Provides-Extra: modal
Requires-Dist: modal>=1.5.4; extra == 'modal'
Provides-Extra: openenv
Requires-Dist: openenv>=0.4.1; extra == 'openenv'
Provides-Extra: ta
Requires-Dist: nltk>=3.9.2; extra == 'ta'
Requires-Dist: textarena==0.7.4; extra == 'ta'
Description-Content-Type: text/markdown

<p align="center">
  <picture>
    <source media="(prefers-color-scheme: light)" srcset="https://github.com/user-attachments/assets/40c36e38-c5bd-4c5a-9cb3-f7b902cd155d">
    <source media="(prefers-color-scheme: dark)" srcset="https://github.com/user-attachments/assets/6414bc9b-126b-41ca-9307-9e982430cde8">
    <img alt="Prime Intellect" src="https://github.com/user-attachments/assets/6414bc9b-126b-41ca-9307-9e982430cde8" width="312" style="max-width: 100%;">
  </picture>
</p>

---

# Overview

verifiers is our library for creating environments to train and evaluate LLMs.

verifiers is tightly integrated with the [Environments Hub](https://app.primeintellect.ai/dashboard/environments?ex_sort=most_stars), as well as our training framework [prime-rl](https://github.com/PrimeIntellect-ai/prime-rl) and our [Hosted Training](https://app.primeintellect.ai/dashboard/training) platform.

## Installation

We recommend installing the [Prime CLI](https://github.com/PrimeIntellect-ai/prime) to interact with the environments.

```bash
# install uv
curl -LsSf https://astral.sh/uv/install.sh | sh
# install the prime CLI
uv tool install prime
```

## Further reading

- The [docs](docs/) contain short, human-written guides and overviews about the architecture.
- The [AGENTS.md](AGENTS.md) and [skills](skills/) are for coding agents and go into more details.

## Contributing

We welcome community contributions. We use [issues](https://github.com/PrimeIntellect-ai/verifiers/issues) to track bugs and feature requests. If you hit a bug, a pain point, or have an idea for a feature, please open an issue.

Contributions are welcome via PR. Please follow these guidelines:

1. Install the [pre-commit hooks](#pre-commit-hooks) so your code is formatted and type-checked.
2. Keep your PR in "Draft" until it is ready for review.
3. If your PR resolves an issue, link the issue in the PR description.
4. Run the [tests](#tests) for the areas you touched.

### Pre-commit hooks

```bash
uv run pre-commit install
```

### Tests

Unit tests run one component in isolation, with no network, and mirror the `verifiers/v1` layout. Integration tests run real evals: they need `PRIME_API_KEY` and skip without it.

```bash
uv run pytest tests/unit -n auto                       # unit
uv run pytest tests/integration -n 8                   # integration
uv run pytest tests/integration/test_single_turn.py    # one behavior
uv run pytest tests/integration -m "docker and codex"  # one runtime and harness
```

## Citation

Originally created by Will Brown ([@willccbb](https://github.com/willccbb)).

```bibtex
@misc{brown_verifiers_2025,
  author       = {William Brown},
  title        = {{Verifiers}: Environments for LLM Reinforcement Learning},
  howpublished = {\url{https://github.com/PrimeIntellect-ai/verifiers}},
  year         = {2025}
}
```
