Metadata-Version: 2.4
Name: agentdrift
Version: 0.1.0
Summary: Detect when your AI agents get dumber. Open-source quality drift monitoring.
Project-URL: Changelog, https://github.com/agentdrift-ai/agentdrift/releases
Project-URL: Homepage, https://github.com/agentdrift-ai/agentdrift
Project-URL: Issues, https://github.com/agentdrift-ai/agentdrift/issues
Project-URL: Repository, https://github.com/agentdrift-ai/agentdrift
License: MIT
License-File: LICENSE
Keywords: agent,ai,drift,evaluation,monitoring,observability
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.10
Requires-Dist: click>=8
Requires-Dist: httpx>=0.25
Requires-Dist: opentelemetry-api>=1.20
Requires-Dist: opentelemetry-sdk>=1.20
Requires-Dist: pydantic>=2
Requires-Dist: rich>=13
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Provides-Extra: scoring
Requires-Dist: anthropic>=0.30; extra == 'scoring'
Requires-Dist: openai>=1.30; extra == 'scoring'
Description-Content-Type: text/markdown

# AgentDrift

[![CI](https://github.com/agentdrift-ai/agentdrift/actions/workflows/ci.yml/badge.svg)](https://github.com/agentdrift-ai/agentdrift/actions/workflows/ci.yml)
[![PyPI version](https://img.shields.io/pypi/v/agentdrift.svg)](https://pypi.org/project/agentdrift/)
[![Python](https://img.shields.io/pypi/pyversions/agentdrift.svg)](https://pypi.org/project/agentdrift/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

**Detect when your AI agents get dumber.**

Open-source quality drift monitoring for AI agents. Track agent outputs over time, detect quality degradation, and get alerts before your agents go off the rails.

## The Problem

AI agents degrade over time. Context pollution, error accumulation, prompt drift, and model updates silently erode output quality. By the time you notice, your users already have.

Research shows **42% of long-running agents** experience quality drift, causing a **3.2x increase in human intervention**.

## Quick Start

```bash
pip install agentdrift
```

```python
from agentdrift import track_agent

@track_agent("my-agent")
def my_agent(query: str) -> str:
    return llm.call(query)
```

Check drift status:

```bash
agentdrift status
```

```
┌──────────────────────────────────────┐
│         Agent Drift Status           │
├──────────┬──────────┬───────┬───────┤
│ Agent    │ Status   │ Score │ Change│
├──────────┼──────────┼───────┼───────┤
│ my-agent │ STABLE   │ 0.847 │ +2.1% │
│ qa-bot   │ DRIFTING │ 0.612 │-18.3% │
└──────────┴──────────┴───────┴───────┘
```

## Features

- **1-line integration** - Decorator or manual logging
- **Quality scoring** - LLM-as-Judge auto-scoring (Anthropic / OpenAI)
- **Drift detection** - Baseline vs recent quality comparison
- **CLI dashboard** - Quick status checks from terminal
- **Golden tests** - Regression testing for agents (coming soon)
- **Alerts** - Slack/Discord/Email webhooks (coming soon)

## How It Works

```
Agent Call → Log Trace → Score Quality → Detect Drift → Alert
```

1. **Track**: Log every agent input/output with the SDK
2. **Score**: LLM-as-Judge rates output quality (0.0-1.0)
3. **Compare**: Recent scores vs baseline period
4. **Alert**: Notify when quality drops below threshold

## Manual Logging

```python
from agentdrift import AgentTracker

tracker = AgentTracker("my-agent")

result = my_agent(query)
tracker.log(
    input=query,
    output=result,
    tokens_in=100,
    tokens_out=50,
    latency_ms=1200,
)
```

## Drift Detection

```python
from agentdrift import DriftDetector

detector = DriftDetector(
    baseline_window=50,   # first N traces as baseline
    recent_window=20,     # last N traces to compare
    threshold_pct=10.0,   # alert if quality drops 10%+
)

report = detector.check("my-agent")
print(report.message)
# "DRIFT DETECTED: my-agent quality dropped 18.3% (baseline: 0.847 -> current: 0.612)"
```

## Examples

See the [`examples/`](examples/) directory:

- [Basic tracking & drift detection](examples/basic_tracking.py)
- [LangChain integration](examples/with_langchain.py)
- [Quality scoring with LLM-as-Judge](examples/with_scoring.py)

## Roadmap

- [x] Agent call tracking (SDK)
- [x] Quality scoring (LLM-as-Judge)
- [x] Drift detection
- [x] CLI dashboard
- [ ] Web dashboard
- [ ] Golden test suite (`agentdrift test init` / `agentdrift test run`)
- [ ] Alert webhooks (Slack, Discord, Email)
- [ ] Multi-agent interaction tracing
- [ ] OpenTelemetry export
- [ ] Managed cloud service

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and guidelines.

## License

MIT
