Metadata-Version: 2.3
Name: graphrag-eval
Version: 6.3.0
Summary: For assessing question answering systems' final answers and intermediate steps, against a given set of questions, reference answers and steps.
License: Apache-2.0
Author: Philip Ganchev
Author-email: philip.ganchev@graphwise.ai
Requires-Python: >=3.12,<3.13
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Provides-Extra: llm
Requires-Dist: pydantic (==2.12.5)
Requires-Dist: python-dateutil (==2.9.0.post0)
Requires-Dist: ragas (==0.4.3) ; extra == "llm"
Project-URL: Repository, https://github.com/Ontotext-AD/graphrag-eval
Description-Content-Type: text/markdown

<p align="center">
  <img alt="Graphwise Logo" src="https://github.com/Ontotext-AD/graphrag-eval/blob/main/.github/Graphwise_Logo.jpg">
</p>

# QA Evaluation

This is a Python library for assessing the quality of question-answering systems, such as systems built with LLM-based agents. It is agnostic to the agent implementation and the LLM it uses.

The evaluation is based on a user-provided reference dataset containing queries, reference responses, and optional reference steps, such as expected tool uses. The evaluator compares these references with the agent's actual responses and executed steps. Reference steps can be grouped to allow some expected steps to occur in any order.

The library provides built-in evaluation metrics and supports user-defined custom metrics ([§ Metrics](https://github.com/Ontotext-AD/graphrag-eval/blob/main/docs/metrics.md)).

## Documentation

- [Quickstart](https://github.com/Ontotext-AD/graphrag-eval/blob/main/docs/quickstart.md)
- [Metrics](https://github.com/Ontotext-AD/graphrag-eval/blob/main/docs/metrics.md)
- [Configuration](https://github.com/Ontotext-AD/graphrag-eval/blob/main/docs/config.md)
- [Input](https://github.com/Ontotext-AD/graphrag-eval/blob/main/docs/input.md)
- [Output](https://github.com/Ontotext-AD/graphrag-eval/blob/main/docs/output.md)

## Maintainers

Developed and maintained by [Graphwise](https://graphwise.ai/). For issues and feature requests, please open a [GitHub issue](https://github.com/Ontotext-AD/graphrag-eval/issues).

## License

Apache-2.0 License. See the [LICENSE](https://github.com/Ontotext-AD/graphrag-eval/blob/main/LICENSE) file for details.

