Metadata-Version: 2.4
Name: provekit
Version: 0.1.0
Summary: Prove your agent works — test any AI agent (LLM/MCP/HTTP/A2A) and run the suite in CI.
Author: ProveKit contributors
License: MIT
Project-URL: Homepage, https://github.com/MobirizerServices/ProveKit
Project-URL: Repository, https://github.com/MobirizerServices/ProveKit
Project-URL: Issues, https://github.com/MobirizerServices/ProveKit/issues
Keywords: agent,llm,testing,evals,mcp,ci,prompt,openai,anthropic
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Testing
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Requires-Dist: fastapi==0.115.6
Requires-Dist: uvicorn[standard]==0.34.0
Requires-Dist: sqlalchemy==2.0.36
Requires-Dist: pydantic==2.10.4
Requires-Dist: pydantic-settings==2.7.1
Requires-Dist: httpx==0.28.1
Requires-Dist: jsonschema==4.23.0
Requires-Dist: cryptography==49.0.0
Requires-Dist: pyyaml==6.0.3
Requires-Dist: alembic==1.18.5
Requires-Dist: psycopg[binary]==3.3.4
Requires-Dist: email-validator==2.2.0
Requires-Dist: redis==5.2.1
Requires-Dist: sentry-sdk[fastapi]==2.20.0
Provides-Extra: dev
Requires-Dist: pytest==9.1.1; extra == "dev"
Requires-Dist: ruff==0.9.2; extra == "dev"

# ProveKit

**Prove your agent works.** ProveKit tests any AI agent — an LLM API, an MCP server, an
HTTP agent, or an A2A agent — turns a run into a regression test, and runs the suite
headless in CI. Open-source, local-first, no account.

This package is the **`provekit` CLI** — the headless test runner. The full visual app
(console, flow builder, live step-debugger) runs as a server; see the
[repo](https://github.com/MobirizerServices/ProveKit).

## Install

```bash
pip install provekit
```

## Use in CI

Write plain-text, git-diffable tests under `.provekit/` (connections referenced by name,
secrets via `${ENV_VAR}` — never in the files), commit them, and run:

```bash
provekit run .provekit/tests/                      # pretty output, non-zero exit on failure
provekit run .provekit/tests/ --format junit -o results.xml
provekit import-promptfoo promptfooconfig.yaml -o .provekit/tests/   # migrate from promptfoo
```

Connections resolve from `.provekit/connections.yaml`:

```yaml
connections:
  OpenAI (prod):
    provider: openai
    api_key: ${OPENAI_API_KEY}
    models: [gpt-4o-mini]
```

## Assertions

`contains` · `equals` · `regex` · `json_path` · `json_schema` · `tool_called` ·
`latency_lt` · `llm_judge` — pass/fail per assertion, non-zero exit on failure.

## Links

- Repository & full app: https://github.com/MobirizerServices/ProveKit
- File format: see `docs/FILE_FORMAT.md` in the repo
- License: MIT
