Metadata-Version: 2.3
Name: pytest-pyeval
Version: 0.2.0
Summary: pytest plugin integrating pydantic-evals
Author: Alex Ward
Author-email: Alex Ward <alxwrd@googlemail.com>
Requires-Dist: pytest>=8.0
Requires-Dist: pydantic-evals>=1.67
Requires-Dist: logfire ; extra == 'logfire'
Requires-Python: >=3.10
Provides-Extra: logfire
Description-Content-Type: text/markdown

<div align="center">
    <h1><code>pytest-pyeval</code></h1>
    <p align="center"><i>
        A <code>pytest</code> plugin integrating <code>pydantic-evals</code>
    </i></p>
    <img width="256px" src=".github/assets/wizard-768.png">

Run [evals](https://ai.pydantic.dev/evals/) via
[pytest](https://docs.pytest.org/en/stable/) with the power of fixtures and
using a familiar Arrange, Act, Evaluate pattern.
</div>


## Example

```python
from pyeval import dataset, Case, EqualsExpected, Contains


def uppercase_text(text: str) -> str:
    return text.upper()


@dataset(
    Case(
        name="uppercase_basic",
        inputs="hello world",
        expected_output="HELLO WORLD",
    ),
    Case(
        name="uppercase_with_numbers",
        inputs="hello 123",
        expected_output="HELLO 123",
    ),
)
def eval_uppercase(case: Case):
    result = execute(uppercase_text, case)

    result.evaluate(EqualsExpected())
    result.evaluate(Contains(value="HELLO", case_sensitive=True))
```

```plain
$ uv run pytest

============================== test session starts ==============================
platform darwin -- Python 3.13.1, pytest-9.0.2, pluggy-1.6.0
plugins: anyio-4.12.1, pyeval-0.1.0
collected 2 items

tests/evals/eval_example.py ●●                                                                         [100%]

============================= 2 evaluated in 0.02s ==============================
```


## Installation

```shell
uv add --dev pytest-pyeval
```
