Metadata-Version: 2.5
Name: kitaru-evaluator
Version: 0.1.2
Summary: Built-in offline evaluators for Kitaru.
Project-URL: Homepage, https://kitaru.ai
Project-URL: Documentation, https://docs.zenml.io/kitaru/guides/deterministic-evaluations
Project-URL: Repository, https://github.com/zenml-io/kitaru
Project-URL: Issues, https://github.com/zenml-io/kitaru/issues
Project-URL: Changelog, https://github.com/zenml-io/kitaru/blob/develop/plugins/packages/evaluator/CHANGELOG.md
Author-email: ZenML GmbH <info@zenml.io>
License-Expression: Apache-2.0
Keywords: ai-agents,evaluation,kitaru,llm,testing
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.11
Requires-Dist: kitaru>=0.22.0
Description-Content-Type: text/markdown

# Kitaru built-in evaluators

Run offline evaluations over recorded and imported Kitaru sessions. This package contains the evaluator implementations registered under the `kitaru/` namespace by the default server and resolved by Kitaru workers.

Most users do not install or call this package directly. Select one or more registered evaluators from the CLI:

```bash
kitaru session evaluate "$SESSION_ID" \
  --evaluator kitaru/session-diagnostics@latest \
  --evaluator kitaru/tool-health@latest \
  --wait
```

The evaluators read stored session evidence. They do not run the agent, call a model provider, invoke a live tool, replay a session, or query an external service. Descriptive evaluators report findings without forcing a pass/fail judgment; configured policy evaluators can pass, fail, or hold when evidence is insufficient.

See the [deterministic evaluations guide](https://docs.zenml.io/kitaru/guides/deterministic-evaluations) for the evaluator catalog, parameters, evidence semantics, and versioning guidance.

## Links

- [Kitaru documentation](https://docs.zenml.io/kitaru)
- [Source code](https://github.com/zenml-io/kitaru)
- [Issue tracker](https://github.com/zenml-io/kitaru/issues)

Licensed under Apache-2.0.
