Metadata-Version: 2.4
Name: assertllm
Version: 0.3.0
Summary: The pytest for LLMs — fast, Pydantic-based assertions for AI applications
Project-URL: Homepage, https://assertllm.dev
Project-URL: Documentation, https://docs.assertllm.dev
Project-URL: Repository, https://github.com/bahadiraraz/llmtest
Project-URL: Bug Tracker, https://github.com/bahadiraraz/llmtest/issues
Project-URL: Changelog, https://github.com/bahadiraraz/llmtest/releases
Author-email: Bahadır Araz <bahadiraraz@protonmail.com>
Maintainer-email: Bahadır Araz <bahadiraraz@protonmail.com>
License: MIT
License-File: LICENSE
Keywords: ai,assertions,evaluation,llm,pydantic,pytest,testing
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: Pytest
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.10
Requires-Dist: pydantic>=2.0
Requires-Dist: pytest>=7.0
Provides-Extra: all
Requires-Dist: anthropic>=0.30; extra == 'all'
Requires-Dist: httpx>=0.25; extra == 'all'
Requires-Dist: openai>=1.0; extra == 'all'
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.30; extra == 'anthropic'
Provides-Extra: dev
Requires-Dist: coverage>=7.0; extra == 'dev'
Requires-Dist: mypy>=1.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.21; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Provides-Extra: ollama
Requires-Dist: httpx>=0.25; extra == 'ollama'
Provides-Extra: openai
Requires-Dist: openai>=1.0; extra == 'openai'
Description-Content-Type: text/markdown

<p align="center">
  <img src=".github/icon.svg" alt="assertllm" width="80" height="80" />
</p>

<h1 align="center">assertllm</h1>

<p align="center">
  <strong>The pytest for LLMs.</strong> Test your AI outputs like you test your code.
</p>

<p align="center">
  <a href="https://badge.fury.io/py/assertllm"><img src="https://badge.fury.io/py/assertllm.svg" alt="PyPI version" /></a>
  <a href="https://www.python.org/downloads/"><img src="https://img.shields.io/badge/python-3.10+-blue.svg" alt="Python 3.10+" /></a>
  <a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License: MIT" /></a>
</p>

## Quick Start

```bash
pip install assertllm[anthropic]
```

```python
from assertllm import expect, llm_test

@llm_test(
    expect.contains("Paris"),
    expect.latency_under(2000),
    expect.cost_under(0.001),
    model="claude-sonnet-4-6",
)
def test_capital(llm):
    llm("What is the capital of France?")
```

```bash
pytest test_capitals.py -v
```

```
test_capitals.py::test_capital
  AI response: "The capital of France is Paris."
  ✓ contains("Paris")
  ✓ latency_under(2000) — 823ms
  ✓ cost_under(0.001) — $0.000023
  PASSED

────────── assertllm summary ──────────
  LLM tests: 1 passed
  Assertions: 3/3 passed
  Total cost: $0.000023
  Avg latency: 823ms
```

## Features

- **22+ assertions** — text, performance, agent, composable
- **Zero LLM calls** for most checks — deterministic and instant
- **Built on Pydantic** — auto-validation, JSON serialization, schema generation
- **Multi-provider** — OpenAI, Anthropic, Ollama out of the box
- **Agent testing** — tool calls, loop detection, call ordering
- **Retry support** — handle non-deterministic outputs
- **CI/CD native** — pytest markers, JUnit XML, JSON reporters
- **Pydantic AI integration** — test your existing agents

## Installation

```bash
pip install assertllm[anthropic]    # Anthropic
pip install assertllm[openai]       # OpenAI
pip install assertllm[ollama]       # Ollama (local)
pip install assertllm[all]          # Everything
```

## Documentation

Full docs at **[docs.assertllm.dev](https://docs.assertllm.dev)**

## Contributing

```bash
git clone https://github.com/bahadiraraz/llmtest
cd llmtest
uv sync --all-extras
uv run pytest tests/ -v
```

## Roadmap

> **This library is under active development.** More providers (Gemini, DeepSeek, Mistral, etc.) and additional assertion types are coming soon.

## License

MIT
