Metadata-Version: 2.4
Name: promptdrifter
Version: 0.0.2
Summary: One-command CI guardrail that catches prompt drift and fails the build when your LLM answers change.
Project-URL: Homepage, https://github.com/Code-and-Sorts/PromptDrifter
Project-URL: Repository, https://github.com/Code-and-Sorts/PromptDrifter
Project-URL: Bug Tracker, https://github.com/Code-and-Sorts/PromptDrifter/issues
Author: Colby Timm
License: MIT
License-File: LICENSE
Keywords: ci,cli,developer tools,llm,prompt drift,prompt engineering,testing
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
Classifier: Topic :: Utilities
Requires-Python: >=3.11
Requires-Dist: coverage<8.0.0,>=7.8.0
Requires-Dist: httpx<0.29.0,>=0.28.1
Requires-Dist: importlib-resources<7.0.0,>=6.5.2
Requires-Dist: jinja2>=3.1.6
Requires-Dist: jsonschema<5.0.0,>=4.23.0
Requires-Dist: openai<2.0.0,>=1.77.0
Requires-Dist: packaging>=23.2
Requires-Dist: pyyaml<7.0.0,>=6.0.2
Requires-Dist: rapidfuzz<4.0.0,>=3.0.0
Requires-Dist: rich<15.0.0,>=14.0.0
Requires-Dist: typer<0.17.0,>=0.16.0
Provides-Extra: dev
Requires-Dist: pytest-asyncio<1.1.0,>=1.0.0; extra == 'dev'
Requires-Dist: pytest-cov>=6.1.1; extra == 'dev'
Requires-Dist: pytest-mock>=3.14.0; extra == 'dev'
Requires-Dist: pytest<9.0.0,>=8.3.5; extra == 'dev'
Requires-Dist: respx<0.23.0,>=0.22.0; extra == 'dev'
Requires-Dist: ruff>=0.11.8; extra == 'dev'
Provides-Extra: similarity
Requires-Dist: sentence-transformers>=2.9.0; extra == 'similarity'
Description-Content-Type: text/markdown

# What is PromptDrifter?

[PromptDrifter](https://github.com/Code-and-Sorts/PromptDrifter) is a command-line tool to test and evaluate Large Language Model (LLM) prompts across different models and providers. It helps catch "prompt drift" when LLM responses change unexpectedly.

By comparing LLM outputs against version-controlled expected responses or predefined patterns, PromptDrifter helps ensure your LLM-powered features behave predictably.

## Features

- **Automated Guardrails**: Integrates LLM response validation directly into your development and CI/CD workflows.
- **Early Drift Detection**: Catches deviations by comparing LLM outputs against version-controlled expected responses or predefined patterns (like regex).
- **Model Agnostic Design**: Through a flexible adapter system, PromptDrifter can interact with various LLM providers and models (e.g., OpenAI, Ollama, Gemini).
- **Declarative Test Suites**: Define your prompt tests in easy-to-understand YAML files, making them simple to create, manage, and version.
- **CLI Interface**: Simple command-line tool for easy integration and use.
- **Response Caching**: Speeds up test runs by caching responses from LLMs.

## Getting Started

To start using PromptDrifter:

1. **Install the package:**
    ```bash
    pip install promptdrifter
    ```

2. **Initialize a sample configuration:**
    ```bash
    promptdrifter init
    ```

This will create a `promptdrifter.yaml` file in your current directory.

3. **Edit `promptdrifter.yaml`** to define your prompts and expected outcomes.

4. **Run your tests:**
    ```bash
    promptdrifter run promptdrifter.yaml
    ```

## License

PromptDrifter is licensed under the MIT License.
