Metadata-Version: 2.4
Name: pytest-llm-report
Version: 0.2.1
Summary: Human-friendly pytest test reports with optional LLM annotations
Project-URL: Homepage, https://github.com/palakpsheth/pytest-llm-report
Project-URL: Documentation, https://palakpsheth.github.io/pytest-llm-report/
Project-URL: Repository, https://github.com/palakpsheth/pytest-llm-report
Project-URL: Issues, https://github.com/palakpsheth/pytest-llm-report/issues
Author: pytest-llm-report contributors
License-Expression: MIT
License-File: LICENSE
Keywords: coverage,llm,pytest,report,testing
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: Pytest
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.11
Requires-Dist: coverage>=7.0.0
Requires-Dist: jinja2>=3.0.0
Requires-Dist: pytest-cov>=4.0.0
Requires-Dist: pytest>=7.0.0
Provides-Extra: all
Requires-Dist: httpx>=0.24.0; extra == 'all'
Requires-Dist: litellm>=1.0.0; extra == 'all'
Requires-Dist: mypy>=1.0.0; extra == 'all'
Requires-Dist: playwright>=1.40.0; extra == 'all'
Requires-Dist: pytest-xdist>=3.0.0; extra == 'all'
Requires-Dist: ruff>=0.1.0; extra == 'all'
Requires-Dist: urllib3>=2.6.3; extra == 'all'
Provides-Extra: dev
Requires-Dist: httpx>=0.24.0; extra == 'dev'
Requires-Dist: litellm>=1.0.0; extra == 'dev'
Requires-Dist: mypy>=1.0.0; extra == 'dev'
Requires-Dist: playwright>=1.40.0; extra == 'dev'
Requires-Dist: pytest-xdist>=3.0.0; extra == 'dev'
Requires-Dist: ruff>=0.1.0; extra == 'dev'
Requires-Dist: urllib3>=2.6.3; extra == 'dev'
Provides-Extra: gemini
Requires-Dist: httpx>=0.24.0; extra == 'gemini'
Requires-Dist: urllib3>=2.6.3; extra == 'gemini'
Provides-Extra: litellm
Requires-Dist: litellm>=1.0.0; extra == 'litellm'
Provides-Extra: ollama
Requires-Dist: httpx>=0.24.0; extra == 'ollama'
Requires-Dist: urllib3>=2.6.3; extra == 'ollama'
Provides-Extra: pdf
Requires-Dist: playwright>=1.40.0; extra == 'pdf'
Description-Content-Type: text/markdown

# pytest-llm-report

[![CI](https://github.com/palakpsheth/pytest-llm-report/actions/workflows/ci.yml/badge.svg)](https://github.com/palakpsheth/pytest-llm-report/actions/workflows/ci.yml)
[![CodeQL](https://github.com/palakpsheth/pytest-llm-report/actions/workflows/codeql.yml/badge.svg)](https://github.com/palakpsheth/pytest-llm-report/actions/workflows/codeql.yml)
[![Snyk Security](https://github.com/palakpsheth/pytest-llm-report/actions/workflows/snyk.yml/badge.svg)](https://github.com/palakpsheth/pytest-llm-report/actions/workflows/snyk.yml)
[![Docs](https://img.shields.io/badge/docs-online-blue?logo=github)](https://palakpsheth.github.io/pytest-llm-report/)
[![codecov](https://codecov.io/gh/palakpsheth/pytest-llm-report/branch/main/graph/badge.svg)](https://codecov.io/gh/palakpsheth/pytest-llm-report)
[![PyPI version](https://badge.fury.io/py/pytest-llm-report.svg)](https://pypi.org/project/pytest-llm-report/)
[![Python Versions](https://img.shields.io/badge/python-3.11%7C3.12%7C3.13-blue?logo=python&logoColor=white)](https://pypi.org/project/pytest-llm-report/)
[![License](https://img.shields.io/github/license/palakpsheth/pytest-llm-report)](LICENSE)
[![Dependabot](https://img.shields.io/badge/dependabot-enabled-blue.svg?logo=dependabot)](https://github.com/palakpsheth/pytest-llm-report/blob/main/.github/dependabot.yml)
[![pytest plugin](https://img.shields.io/badge/pytest-plugin-0A9EDC?logo=pytest&logoColor=white)](https://docs.pytest.org/)

Human-friendly pytest test reports with optional LLM annotations.

## Features

- Test inventory with status (pass/fail/skip/xfail) and duration
- Per-test covered files and line ranges (using pytest-cov + coverage contexts)
- Per-file source coverage summary (covered/missed/percentage)
- Optional LLM-generated annotations:
  - **Scenario**: What the test verifies
  - **Why needed**: What regression/bug it prevents
  - **Key assertions**: Critical checks performed
  - **Smart Context**: Automatically fetches model context limits (Gemini, Ollama, LiteLLM) to maximize relevant code included in prompts without hitting token limits.
  - **Compact Context**: Automatically strips docstrings and optimizes whitespace to reduce token consumption (enabled by default).
  - **Token Usage Reporting**: Tracks and displays input, output, and total token usage for each annotation and the entire test run.
- HTML and JSON output formats
- Dark mode support (auto-detects system preference)
- Optional PDF generation
- **Detailed Versioning**: Reports include exact plugin and repository versions (plus Git SHA/dirty status) for reproducibility
- Aggregation across multiple test runs (see [Aggregation](https://palakpsheth.github.io/pytest-llm-report/aggregation/))

## Installation

```bash
pip install pytest-llm-report
```

Or with uv:
```bash
uv add pytest-llm-report
```

### Optional Dependencies

Install additional features as needed:

```bash
# Ollama provider
pip install pytest-llm-report[ollama]

# Gemini provider
pip install pytest-llm-report[gemini]

# LiteLLM provider (supports OpenAI, Anthropic, etc.)
pip install pytest-llm-report[litellm]

# PDF generation
pip install pytest-llm-report[pdf]

# All features
pip install pytest-llm-report[all]
```

With uv:
```bash
uv add "pytest-llm-report[ollama]"  # or gemini, litellm, pdf, all
```

## Quick Start

Run pytest with coverage contexts enabled:

```bash
pytest --cov=your_package --cov-context=test --llm-report=report.html
```

## Configuration

Configure via `pyproject.toml`:

```toml
[tool.pytest_llm_report]
provider = "none"  # "none", "ollama", "litellm", or "gemini"
report_json = "reports/pytest_llm_report.json"
```

## Documentation

📖 **Full documentation**: [palakpsheth.github.io/pytest-llm-report](https://palakpsheth.github.io/pytest-llm-report/)

## Requirements

- Python 3.11+
- pytest >= 7.0.0
- pytest-cov >= 4.0.0

## Contributing

Contributions are welcome! Please see:

- [Contributing Guide](CONTRIBUTING.md) - Development setup and guidelines
- [Code of Conduct](CODE_OF_CONDUCT.md) - Community guidelines
- [Security Policy](SECURITY.md) - Reporting vulnerabilities
- [Changelog](CHANGELOG.md) - Version history

## License

MIT
