Metadata-Version: 2.4
Name: driftguard-tokepi
Version: 0.5.1
Summary: Observability for LLM apps and AI agents: drift checks, failed-tool diagnosis, and wasted-token accounting
Author: Sean Pereira
License-Expression: MIT
Project-URL: Homepage, https://github.com/Sean-Pereira-945/DriftGuard
Project-URL: Repository, https://github.com/Sean-Pereira-945/DriftGuard
Project-URL: Documentation, https://github.com/Sean-Pereira-945/DriftGuard/tree/main/docs
Project-URL: Bug Tracker, https://github.com/Sean-Pereira-945/DriftGuard/issues
Keywords: ai,observability,drift,llm,agents,mcp,monitoring
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: System :: Monitoring
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: httpx>=0.27.0
Provides-Extra: server
Requires-Dist: fastapi>=0.140.0; extra == "server"
Requires-Dist: starlette>=1.3.1; extra == "server"
Requires-Dist: uvicorn[standard]>=0.30.6; extra == "server"
Requires-Dist: sqlalchemy>=2.0.30; extra == "server"
Requires-Dist: pyjwt>=2.8.0; extra == "server"
Requires-Dist: python-dotenv>=1.0.0; extra == "server"
Provides-Extra: postgres
Requires-Dist: psycopg[binary]>=3.1; extra == "postgres"
Provides-Extra: redis
Requires-Dist: redis>=5.0.0; extra == "redis"
Provides-Extra: all
Requires-Dist: driftguard-tokepi[postgres,redis,server]; extra == "all"
Provides-Extra: research
Requires-Dist: numpy>=2.0; extra == "research"
Provides-Extra: dev
Requires-Dist: driftguard-tokepi[all,research]; extra == "dev"
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: ruff>=0.6.0; extra == "dev"
Requires-Dist: bandit>=1.7.0; extra == "dev"
Requires-Dist: pip-audit>=2.7.0; extra == "dev"
Dynamic: license-file

# DriftGuard

**Observability for LLM applications and AI agents: catch drift, find the tool
that's blocking your agent, and count the tokens it's wasting.**

When an agent keeps retrying a failing tool, you pay for every attempt and the
task never finishes. DriftGuard receives one event per tool call and names the
blocking tool. It totals the wasted tokens and alerts you, for example: *"The
terminal tool failed 3 times in a row on task fix-tests; 6,600 tokens were spent
on failed or redundant attempts."* It also scores ordinary LLM traffic for drift:
prompt and context bloat, falling retrieval relevance, and falling response
quality.

- **Agent diagnosis.** Per-task status (blocked, failing, recovered, healthy),
  the blocking tool, repeated and redundant attempts, wasted tokens, and advice
  based on the error.
- **Activity logs.** A searchable, live history of every tool call, model call,
  prompt and session, with CSV/JSON export. Inputs and outputs are stored only
  if you opt in per project.
- **Drift scoring.** Four normalized metrics are checked against per-project
  thresholds and turned into a severity, a root cause, and a recommended
  mitigation. The Overview also compares each window with the period before it,
  and says so when there are too few events to judge.
- **Alerts that manage themselves.** Critical drift and blocked tasks raise
  alerts. Agent alerts resolve themselves when the task recovers. Alerts are
  pushed live to the dashboard, Slack, or any webhook.
- **Any agent, no code.** An `driftguard-mcp-proxy` gateway records any MCP
  client's tool calls from the middle of the stdio stream.
- **Framework adapters.** LangChain/LangGraph callbacks, the OpenAI Agents SDK,
  and the Claude Agent SDK, each one or two lines to wire in.
- **Drop-in adapters.** A `@driftguard_tool` decorator, MCP session
  instrumentation, token extraction from OpenAI, Anthropic, and Gemini
  responses, and a no-code [Claude Code hook](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/integrations/claude_code/README.md).
- **Fast to operate.** A status plate and a track diagram tell you what is
  blocked first. Keyboard shortcuts, links that carry your filters, bulk
  resolve, and a Policy page that shows what each threshold would flag.
- **Self-hostable.** A FastAPI server, a React dashboard, and SQLite or
  PostgreSQL, with Redis when you scale out. One `docker compose up`.

DriftGuard observes what your code reports, and it recommends actions rather
than taking them. It doesn't intercept closed agents such as GitHub Copilot, and
it never changes prompts or providers. See
[what it can't observe](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/agent-integration.md#what-driftguard-cannot-observe).

## Start here

| I want to… | Do this |
| --- | --- |
| Set it up on a teammate's laptop | Follow **[SETUP.md](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/SETUP.md)** |
| See it running in two minutes (Windows) | Double-click **`demo.bat`**. Sign in as `demo@driftguard.local` / `driftguard-demo`. |
| Run it from this repo | `pip install -e ".[server]"`, `cd frontend && npm ci && npm run build`, then `driftguard-server`. Details in [Getting started](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/quickstart.md#b-from-source-windows-macos-linux). |
| Fill the dashboard with sample data | `python examples/seed_showcase.py --api-key dg_live_... --project-id showcase` |
| Watch my own Claude Code sessions | [Claude Code hook](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/integrations/claude_code/README.md) |
| Watch any MCP client's tools (no code) | `driftguard-mcp-proxy` ([MCP gateway](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/agent-integration.md#the-mcp-gateway-no-code-any-mcp-client)) |
| Understand how it works | [How DriftGuard works](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/concepts.md), then [Architecture](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/architecture.md) |
| Send retrieval and quality scores | [Measuring retrieval score and response quality](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/scoring-guide.md) |

Every view is described in the [dashboard guide](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/dashboard.md). The
architecture diagram and dashboard screenshots are in
[the docs](https://github.com/Sean-Pereira-945/DriftGuard/tree/main/docs/images).

## Quickstart (SDK)

```bash
pip install "driftguard-tokepi[server]"   # PyPI name is driftguard-tokepi; you still import driftguard
driftguard-server                    # dashboard and API at http://127.0.0.1:8000
```

Create an account and a project in the dashboard, and copy the project API key.
Then, in your app:

```python
from driftguard import DriftGuardClient
from driftguard.adapters import AgentContext, driftguard_tool

client = DriftGuardClient(api_key="dg_live_...", project_id="coding-agent",
                          base_url="http://127.0.0.1:8000")

# LLM telemetry
client.capture_metrics(prompt_tokens=5200, context_length=6100,
                       retrieval_score=0.31, response_quality=0.62)
client.check_drift()   # {'severity': 'critical', 'risk_score': 1.0, 'actions': [...], ...}
client.sync_metrics()

# Agent tool calls
@driftguard_tool("terminal")
def run(cmd): ...

with AgentContext(client, task_id="fix-tests", auto_sync=True) as ctx:
    ctx.record_llm_usage(llm_response)   # attach model tokens to the next tool call
    run("pytest -x")

client.fetch_agent_diagnosis()["diagnosis"]
```

The SDK itself (`pip install driftguard-tokepi`) depends only on `httpx`. The full
walkthrough, including Docker, is in [docs/quickstart.md](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/quickstart.md).

## Documentation

| | |
| --- | --- |
| [Getting started](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/quickstart.md) | Three ways to run it, five ways to send data, troubleshooting |
| [How DriftGuard works](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/concepts.md) | Plain-language concepts, scoring and diagnosis with worked examples |
| [Giving a demo](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/demo.md) | A five-minute live demo, a presentation outline, talking points |
| [What the package offers](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/package-overview.md) | What `pip install driftguard-tokepi` provides (SDK, adapters, gateway) |
| [Using the package](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/package-usage.md) | Install it and wire the SDK, adapters and gateway into your project |
| [Using the dashboard](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/using-the-dashboard.md) | Open, navigate and operate the web UI |
| [Agent integration](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/agent-integration.md) | Decorator, MCP, raw events, and what can't be observed |
| [Dashboard guide](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/dashboard.md) | Every view, card and column, with screenshots |
| [Measuring scores](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/scoring-guide.md) | How to calculate retrieval score and response quality, choosing floors, and what the numbers can't tell you |
| [API reference](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/api.md) | Every endpoint, with real request and response bodies |
| [Architecture](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/architecture.md) | Diagrams, components, request flows, algorithms, data model, security |
| [Configuration](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/configuration.md) | Environment variables and policy settings |
| [Deployment](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/deployment.md) | Docker Compose, TLS, scaling, backups, upgrading from 0.3 |
| [Development](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/development.md) | Repo layout, tests on SQLite/Postgres/Redis, CI, releasing |
| [Design system](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/design-system.md) | The dashboard's look, in plain language ([DESIGN.md](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/DESIGN.md) has the tokens) |
| [Product context](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/PRODUCT.md) | Who it is for, what it does, and the constraints work must keep |
| [Research](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/research/README.md) | A simulation study of floors against the baseline comparison, plus the Phase 1 prototypes |
| [Progress report](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/progress.md) | What is done, what changed, and what is next |
| [Roadmap](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/docs/roadmap.md) | What's done and what's next |
| [Changelog](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/CHANGELOG.md) | Release history |

## Repository layout

```text
driftguard/     the Python package: SDK, adapters, and driftguard.server
frontend/       dashboard source (React + Vite); builds into driftguard/server/static
research/       the baseline study (simulation) and the Phase 1 drift prototypes
integrations/   no-code integrations, starting with Claude Code hooks
tests/          SDK and server test suite
examples/       runnable scripts, including the live demo (demo.bat runs it on Windows)
docs/           documentation, including PRODUCT.md and the screenshots in docs/images
DESIGN.md       the dashboard's design tokens and components
```

## Development

```bash
pip install -e ".[dev]"
pytest                                   # SQLite; see docs/development.md for Postgres/Redis
ruff check . && ruff format --check .
cd frontend && npm install && npm run dev
```

## License

MIT. See [LICENSE](https://github.com/Sean-Pereira-945/DriftGuard/blob/main/LICENSE).
