Metadata-Version: 2.5
Name: genkit-evaluators
Version: 0.10.0
Summary: Genkit Evaluators Plugin (regex, deep_equal, jsonata)
Project-URL: Bug Tracker, https://github.com/genkit-ai/genkit/issues
Project-URL: Documentation, https://firebase.google.com/docs/genkit
Project-URL: Homepage, https://github.com/genkit-ai/genkit
Project-URL: Repository, https://github.com/genkit-ai/genkit/tree/main/py
Author: Google
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: ai,eval,evaluator,genkit,ragas
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: genkit
Requires-Dist: jsonata-python>=0.6.0
Description-Content-Type: text/markdown

# Genkit Evaluators Plugin

Provides three rule-based evaluators matching the Go and JS implementations:

- **regex** – Tests output against a regex pattern (reference = regex string)
- **deep_equal** – Tests equality of output against reference
- **jsonata** – Evaluates a JSONata expression (reference) against output; pass if result is truthy

No LLM or API keys required.

## Installation

```bash
uv add genkit-evaluators
```

## Usage

```python
from genkit import Genkit
from genkit.evaluator import BaseDataPoint
from genkit_evaluators import register_genkit_evaluators

ai = Genkit()
register_genkit_evaluators(ai)

results = await ai.evaluate(
    evaluator='genkitEval/regex',
    dataset=[
        BaseDataPoint(input='sample', output='banana', reference='ba?a?a'),
        BaseDataPoint(input='sample', output='apple', reference='ba?a?a'),
    ],
)
```

Or from the CLI:

```bash
genkit eval:run datasets/example.json --evaluators=genkitEval/regex
```

## Evaluators

- **genkitEval/regex** – Reference is a regex string. Output (stringified if needed) must match.
- **genkitEval/deep_equal** – Reference is the expected value. Output must equal reference.
- **genkitEval/jsonata** – Reference is a JSONata expression. Evaluated against output; pass if truthy.
