Metadata-Version: 2.4
Name: aligneval
Version: 0.1.0
Summary: An open-source Python framework for evaluating LLM systems.
Author-email: Antigravity <antigravity@example.com>
License-Expression: MIT
License-File: LICENSE
Requires-Python: >=3.8
Requires-Dist: aiohttp>=3.8.0
Requires-Dist: numpy>=1.20.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: tqdm>=4.65.0
Provides-Extra: dev
Requires-Dist: black>=23.0.0; extra == 'dev'
Requires-Dist: isort>=5.0.0; extra == 'dev'
Requires-Dist: mypy>=1.0.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.21.0; extra == 'dev'
Requires-Dist: pytest>=7.0.0; extra == 'dev'
Description-Content-Type: text/markdown

# AlignEval

AlignEval is an open-source Python framework for evaluating LLM systems. It helps you measure accuracy, hallucination, relevancy, and safety across your RAG pipelines, agents, and custom chatbots.

## Features

- **Async Evaluation**: Evaluate multiple test cases concurrently for speed.
- **Pluggable Metrics**: Use built-in metrics (Accuracy, Hallucination, Safety) or define your own.
- **Caching**: Avoid re-running expensive evaluations with built-in caching.
- **Synthetic Data**: Generate test cases using an LLM to bootstrap your evaluation dataset.
- **Pytest Integration**: Run evaluations as part of your standard test suite.

## Installation

```bash
pip install aligneval
```

## Quick Start

```python
import asyncio
from aligneval import Evaluator
from aligneval.metrics import Accuracy, Hallucination

async def main():
    evaluator = Evaluator()
    dataset = [
        {"input": "What is the capital of France?", "output": "Paris", "context": "France is a country in Europe. Its capital is Paris.", "expected": "Paris"},
        # ... more cases
    ]
    
    results = await evaluator.evaluate(
        dataset,
        metrics=[Accuracy(), Hallucination()]
    )
    
    print(results)

if __name__ == "__main__":
    asyncio.run(main())
```
