Metadata-Version: 2.4
Name: simpleval
Version: 1.1.0
Summary: Simple Evaluation Framework for LLM use cases
Project-URL: Repository, https://github.cyberng.com/InnoAI/simple-llm-eval
Author-email: Roy Ben Yosef <roy.benyosef@cyberark.com>
License-File: LICENSE
License-File: NOTICES.txt
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.11
Requires-Dist: boto3<2,>=1.42.25
Requires-Dist: click<9,>=8.3.1
Requires-Dist: colorama<0.5,>=0.4.6
Requires-Dist: google-genai<2,>=1.57.0
Requires-Dist: httpcore<2,>=1.0.9
Requires-Dist: inquirerpy<0.4,>=0.3.4
Requires-Dist: litellm>=1.80.13
Requires-Dist: pydantic<3,>=2.12.5
Requires-Dist: tabulate<0.10,>=0.9.0
Requires-Dist: tenacity<10,>=9.1.2
Requires-Dist: tqdm<5,>=4.67.1
Requires-Dist: urllib3>=2.6.3
Description-Content-Type: text/markdown

# Simple LLM Evaluation

[![version](https://img.shields.io/github/v/release/cyberark/simple-llm-eval)](https://pypi.org/project/simpleval/)
![Build Status](https://github.com/cyberark/simple-llm-eval/actions/workflows/ci.yml/badge.svg)
![Code Coverage](https://raw.githubusercontent.com/cyberark/simple-llm-eval/refs/heads/badges/ci/badges/coverage-updated.svg)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/simpleval)
[![license](https://img.shields.io/badge/license-Apache%202.0-blue?style=flat-square)](https://raw.githubusercontent.com/cyberark/simple-llm-eval/refs/heads/main/LICENSE)
[![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/cyberark/simple-llm-eval/badge)](https://scorecard.dev/viewer/?uri=github.com/cyberark/simple-llm-eval)

![Simpleval Banner](https://raw.githubusercontent.com/cyberark/simple-llm-eval/main/docs/media/simpleval-banner.jpeg)

Welcome to the simple LLM evaluation framework—**simpleval**, for short.

**simpleval** is a Python package designed to make evaluating Large Language Models (LLMs) easier, using the "LLM as a Judge" technique.

It supports a [variety of LLM providers](https://cyberark.github.io/simple-llm-eval/latest/getting-started/judge-authentication/), including OpenAI, Google (Gemini API, Vertex), AWS Bedrock, Anthropic, Azure, and more (via LiteLLM).

**simpleval** also includes several reports to help you analyze, compare, and summarize your evaluation results. See the [available reports](https://cyberark.github.io/simple-llm-eval/latest/getting-started/reporting/) for more details.

## Getting Started

See the [📚 Quickstart Guide 📚](https://cyberark.github.io/simple-llm-eval/latest/getting-started/quickstart/)

## Documentation

See [📚 Project Documentation 📚](https://cyberark.github.io/simple-llm-eval/)

## Contributing

We appreciate your help in making this project better! ✨

If you would like to contribute to this project, please follow the guidelines outlined in the [CONTRIBUTING.md](https://github.com/cyberark/simple-llm-eval/blob/main/CONTRIBUTING.md) file.

## License

simpleval is released under the [Apache License](https://www.apache.org/licenses/LICENSE-2.0). See the [LICENSE](https://github.com/cyberark/simple-llm-eval/blob/main/LICENSE) file for more details.

## Contact

If you have any questions or suggestions, feel free to join our [GitHub discussions forum](https://github.com/cyberark/simple-llm-eval/discussions) 💬

If you want to report a bug or request a feature, please open an issue in the [GitHub issues tracker](https://github.com/cyberark/simple-llm-eval/issues) 🐛

<br>
