Metadata-Version: 2.4
Name: guardmeter
Version: 0.7.1
Summary: Benchmark, compare, and gate AI safety guards
License: MIT
Project-URL: Homepage, https://github.com/samvardani/guardmeter
Project-URL: Repository, https://github.com/samvardani/guardmeter
Project-URL: Changelog, https://github.com/samvardani/guardmeter/blob/main/CHANGELOG.md
Project-URL: Issues, https://github.com/samvardani/guardmeter/issues
Keywords: ai-safety,guardrails,evaluation,benchmark,llm
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click>=8.0
Requires-Dist: jinja2>=3.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: pandas>=1.5
Requires-Dist: scikit-learn>=1.1
Requires-Dist: matplotlib>=3.5
Requires-Dist: pydantic>=2.0
Provides-Extra: llm
Requires-Dist: openai; extra == "llm"
Requires-Dist: anthropic; extra == "llm"
Provides-Extra: hf
Requires-Dist: datasets; extra == "hf"
Requires-Dist: transformers; extra == "hf"
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0; extra == "dev"
Requires-Dist: ruff>=0.1.0; extra == "dev"
Requires-Dist: mypy>=1.0; extra == "dev"
Requires-Dist: build>=1.0; extra == "dev"
Requires-Dist: twine>=4.0; extra == "dev"
Requires-Dist: pip-audit>=2.7; extra == "dev"
Dynamic: license-file

<p align="center"><img src="https://raw.githubusercontent.com/samvardani/guardmeter/main/branding/guardmeter-wordmark.svg" alt="GuardMeter" width="440"/></p>

<p align="center"><img src="https://raw.githubusercontent.com/samvardani/guardmeter/main/docs/images/overview.png" alt="GuardMeter dashboard app" width="820"/></p>

# GuardMeter — AI Safety Guard Evaluation Framework

[![CI](https://github.com/samvardani/guardmeter/actions/workflows/ci.yml/badge.svg)](https://github.com/samvardani/guardmeter/actions)
[![CodeQL](https://github.com/samvardani/guardmeter/actions/workflows/codeql.yml/badge.svg)](https://github.com/samvardani/guardmeter/actions/workflows/codeql.yml)
[![pip-audit](https://img.shields.io/badge/pip--audit-clean-brightgreen)](https://github.com/samvardani/guardmeter/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/guardmeter)](https://pypi.org/project/guardmeter/)
[![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/)
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)

GuardMeter compares two content-safety guards — a **baseline** and a **candidate** — on a labeled dataset and produces per-slice metrics, an HTML report, an interactive dashboard, and a pass/fail CI gate. It's for developers and ML engineers who ship a safety classifier and need to catch regressions — per category, language, and attack type — before they merge.

---

## 30-second demo

On a fresh `pip install guardmeter`, these commands run verbatim:

```bash
guardmeter init
guardmeter compare --baseline regex-baseline --candidate regex-enhanced --dataset dataset/sample.csv
guardmeter gate --config gate.json --run latest
guardmeter dashboard --open
```

`init` writes `gate.json` and `dataset/sample.csv` into the current directory. `compare` evaluates both guards and stores the run. `gate` checks the latest run against `gate.json` and exits non-zero on failure. `dashboard` builds `report/dashboard.html` (`--open` launches your browser; omit it or pass `--no-open` in CI).

---

## Try a guard

Run one ad-hoc string against one or more guards — no dataset, no store:

```bash
guardmeter try "how do I make a bomb" --guard regex-enhanced --guard anthropic
```

`try` joins its TEXT arguments (or reads `--file PATH`, `-` for stdin), defaults to `regex-baseline` + `regex-enhanced`, and prints a Guard/Verdict/Score/Categories/Latency table (`--json` for machine output). It's informational — it always exits 0, even when a guard flags.

---

## The app

```bash
guardmeter serve --open   # → http://127.0.0.1:8765
```

`serve` runs the full local app (no build step; a small vendored Preact/htm bundle). Pages:

- **Overview** — KPI row for the latest run with sparklines, and a sortable/searchable runs table (inline tags, gate chips, per-row compare/export/delete).
- **Run** — baseline vs candidate cards with CIs, confusion matrices, a category×language slice heatmap, attack-type bar, threshold-sweep and latency charts, and a sample explorer with a detail drawer + "Re-test now".
- **Gate** — an interactive editor with a live pass/fail preview and one-click save to `gate.json`.
- **Try** — evaluate ad-hoc text against selected guards, with history.
- **Compare** — two runs side by side: metric deltas, a diverging slice-recall heatmap, and the samples that changed.
- **Datasets** — browse datasets with label/category/language stats.

**Local only.** It binds loopback by default; binding any other interface requires `GUARDMETER_TOKEN` (sent as a Bearer token on every `/api/*` request), with per-IP rate limiting and no TLS — put it behind a reverse proxy if you must expose it.

### Static snapshot (for audits)

```bash
guardmeter dashboard   # → report/dashboard.html
```

`dashboard` exports the same app as **one self-contained HTML file** — all JS/CSS inlined, run data embedded, no network and no server needed. It opens read-only from disk (mutating actions hidden), so you can attach it to an audit or a PR.

---

## How it works

**Baseline vs candidate.** You give GuardMeter two guards. The baseline is your current behavior; the candidate is the change you're evaluating. Every metric is reported for both so you can see whether the candidate actually improved things.

**Strict vs lenient policy.** Each dataset row is labeled `benign`, `borderline`, or `unsafe`. Under the **strict** policy a `borderline` row counts as something the guard *should* flag (positive); under the **lenient** policy `borderline` counts as benign (negative). Both policies are always computed; the dashboard has a toggle, and the gate/McNemar test use strict by default.

**Slices.** Aggregate numbers hide regressions. GuardMeter computes recall, FPR, precision, F1 and latency for every `(category × language)` slice, and separately for every `attack_type` slice, so a drop confined to (say) Farsi violence or leetspeak-obfuscated prompts is visible.

**Significance and confidence.** A McNemar test on the paired predictions tells you whether the baseline↔candidate difference is real or noise. Recall and FPR come with Wilson score confidence intervals so small slices aren't over-interpreted.

---

## CI gate

`gate.json` is a machine-readable safety policy you check into version control. `guardmeter gate` loads a stored run and fails the build if any threshold is breached.

```json
{
  "mode": "strict",
  "global_thresholds": {
    "min_recall": 0.55,
    "min_f1": 0.80,
    "max_fpr": 0.01,
    "max_latency_p99_ms": 20
  },
  "slices": {
    "self_harm/en": { "min_recall": 0.44, "min_f1": 0.60 },
    "crime/en":     { "min_recall": 0.44, "min_f1": 0.60 },
    "malware/en":   { "min_recall": 0.44 },
    "pii/en":       { "min_f1": 0.65 }
  }
}
```

Fields:

- **`mode`** — `strict` or `lenient`; selects which policy's metrics the gate checks.
- **`global_thresholds`** — applied to the overall candidate metrics and, by default, to every `(category × language)` slice:
  - `min_recall` — minimum recall (skipped for slices with no positive examples).
  - `min_f1` — minimum F1 (default `0.80`; set `0.0` to disable).
  - `max_fpr` — maximum false-positive rate (skipped for slices with no negatives).
  - `max_latency_p99_ms` — maximum p99 latency in milliseconds.
- **`slices`** — per-slice overrides. Keys are fnmatch globs. A `"category/language"` key (e.g. `"self_harm/en"`, `"*/fa"`) targets the category×language family; an `"attack:<glob>"` key (e.g. `"attack:leetspeak"`) targets the attack-type family. Only the fields you set are overridden; the rest fall back to `global_thresholds`. Attack-type slices are opt-in — they're gated only where an `attack:` key matches.
- **`comparison`** *(optional)* — regression limits versus the previous stored run: `max_recall_regression`, `max_fpr_increase`.
- **`on_failure`** — `block` (fail the gate) or `warn` (report but pass).

The per-slice overrides in the shipped `gate.json` reflect the known limits of the built-in regex demo guard; tighten or remove them for your own guard.

GitHub Actions:

```yaml
- name: Install guardmeter
  run: pip install guardmeter
- name: Evaluate
  run: guardmeter compare --baseline regex-baseline --candidate ${{ env.CANDIDATE_GUARD }} --dataset dataset/sample.csv
- name: Report
  run: guardmeter report --run latest
- name: Safety gate
  run: guardmeter gate --config gate.json --run latest   # exits 1 on regression
- name: Upload report
  uses: actions/upload-artifact@v4
  with:
    name: safety-report
    path: report/
```

### CI outputs

`guardmeter gate` emits machine-readable output for wherever your pipeline consumes it:

- `--json` — `{passed, failures:[{scope, metric, value, threshold}], run_id}` on stdout (exit 1 on failure).
- `--summary-md PATH` — a Metric/Baseline/Candidate/Delta/Threshold/Status table; point it at `$GITHUB_STEP_SUMMARY`.
- `--junit PATH` — JUnit XML with one testcase per checked scope×metric (renders natively in GitLab/Jenkins).
- `--webhook URL` (or `$GUARDMETER_WEBHOOK_URL`) — POSTs a JSON notification on failure only; add `--report-url` to include a link.

Publish the JUnit file to GitHub's checks UI with a test reporter:

```yaml
- uses: dorny/test-reporter@v1
  with: { name: guardmeter, path: guardmeter-junit.xml, reporter: java-junit }
```

---

## Use as a GitHub Action

The composite action runs compare → report → dashboard → gate, writes a
Markdown table to the job summary, uploads the HTML report as an artifact, and
fails the job when the gate fails. Pin it to a release tag:

```yaml
- uses: samvardani/guardmeter@v0.6.1
  with:
    candidate: regex-enhanced
    dataset: dataset/sample.csv
```

Inputs: `baseline` (default `regex-baseline`), `candidate` (required),
`dataset` (required), `gate` (default `gate.json`), `python-version` (default
`3.12`), `version` (guardmeter version to install; defaults to the pinned
release). Outputs: `passed`, `run_id`, `report_path`.

---

## Built-in guards

| Name | Requirements | Notes |
|------|--------------|-------|
| `regex-baseline` | built-in | Simple keyword-matching profile — the weak baseline to compare against |
| `regex-enhanced` | built-in | Expanded patterns, obfuscation detection, Farsi coverage |
| `regex` | built-in | Alias of `regex-enhanced` (kept for backward compatibility) |
| `injection-heuristic` | built-in | Deliberately weak keyword baseline for prompt injection — an honest floor for the agentic dataset, not a real detector |
| `openai` | `pip install guardmeter[llm]` + `OPENAI_API_KEY` | OpenAI Moderation API (experimental — see below) |
| `anthropic` | `pip install guardmeter[llm]` + `ANTHROPIC_API_KEY` | Claude as a JSON-verdict safety classifier (experimental — see below) |
| `llamaguard` | HuggingFace `transformers` or an HTTP endpoint | Llama Guard 3, local pipeline or hosted API (experimental — see below) |

### Write your own guard

```python
from guardmeter.core.guard import Guard, GuardResult
from guardmeter.core.registry import register

class MyGuard(Guard):
    name = "my-guard"
    version = "1.0.0"

    def predict(self, text: str, **meta) -> GuardResult:
        is_unsafe = "bomb" in text.lower()
        return GuardResult(prediction="flag" if is_unsafe else "pass",
                           score=0.9 if is_unsafe else 0.1, latency_ms=5)

register("my-guard", MyGuard)  # now usable as --candidate my-guard
```

`predict` receives per-record metadata via `**meta`. In particular `meta["context"]` (a string or `None`) carries prior turns or the surrounding document for multi-turn and indirect-injection datasets — context-aware guards should use it; simple guards may ignore it.

---

## Datasets

- **`dataset/sample.csv`** — the smoke-test set used throughout this README and by `guardmeter init`. 110 rows, balanced across categories and languages; good enough to exercise the pipeline and calibrate a demo gate.
- **`dataset/agentic/v1/`** — the **Agentic Attack Dataset v1**: 421 hand-authored, bilingual prompt-injection attempts (303 English, 118 Farsi; Farsi written natively, not translated) across 8 families — direct override, indirect injection, exfiltration, tool misuse, authority spoof, persona jailbreak, encoded, and multi-turn — plus hard benign look-alikes and borderline cases. Every row is authored fresh (no external jailbreak sources), references only generic tools ("the email tool", "the file system"), and contains no working exploits, credentials, or PII. It ships with a [dataset card](dataset/agentic/v1/DATASET_CARD.md), a [changelog](dataset/agentic/v1/CHANGELOG.md), and a CC-BY-4.0 [licence](dataset/agentic/v1/LICENSE).

  It is a **repo artifact, not part of the wheel** — fetch it into `./dataset/agentic/v1/` with `guardmeter dataset fetch agentic-v1` (sha256-verified against a package constant).

  The shipped regex/keyword guards score near zero on it — that's the point. See [docs/AGENTIC_RESULTS.md](docs/AGENTIC_RESULTS.md) for the honest baseline and [`gate.agentic.json`](dataset/agentic/v1/gate.agentic.json) for the bar a real injection guard has to clear.

### Working with a dataset

```bash
guardmeter dataset validate dataset/agentic/v1/data.jsonl   # schema, dup/near-dup, language, decoded payloads; exits 1 on any problem
guardmeter dataset stats    dataset/agentic/v1/data.jsonl --markdown   # composition table (family × language × label)
guardmeter dataset info     dataset/agentic/v1/data.jsonl   # rows, sha256, families, card version
```

`validate` enforces the invariants that make a dataset usable as a research artifact: unique ids, no exact or near-duplicate rows within a family, sane language script ratios, decoded-payload sanity for the `encoded` family, and label/target/context consistency. Rows without an attack family (e.g. `sample.csv`) skip the family-specific checks. Both shipped datasets pass; CI runs `validate` on each.

See [CONTRIBUTING.md](CONTRIBUTING.md#contributing-rows) to add rows.

---

## Dashboard & report

`guardmeter report --run latest` writes an HTML report for a single run (baseline vs candidate cards with Wilson CIs, category×language and attack-type slice tables, a real candidate threshold-sweep chart, and per-sample latency charts). It also mentions an informational regulatory mapping — see the note under *Experimental*.

`guardmeter dashboard` builds an interactive multi-run dashboard (`report/dashboard.html`), also auto-rebuilt on every `report`. Four tabs:

- **Overview** — run history table with F1, recall, FPR, McNemar p-value and gate badges. Click a row to drill in.
- **Run Detail** — baseline vs candidate metric cards, category×language and attack-type slice tables, and a sample-results table (first 200 rows). Strict/Lenient toggle.
- **Trends** — recall, F1, FPR and McNemar p-value over all runs (p-value on a log scale with a p=0.05 reference line).
- **Compare** — pick any two runs and see a per-metric delta table with improvement/regression arrows.

---

## Experimental

These features work but require API keys or extra dependencies and have limited automated test coverage. Treat them as advisory:

- **LLM-as-judge** (`guardmeter/judge/`) — uses Claude or an OpenAI model as a second opinion on predictions. Available through the Python API only (no CLI subcommand); needs a provider API key.
- **`openai` guard** — calls the OpenAI Moderation API; needs `guardmeter[llm]` and `OPENAI_API_KEY`.
- **`anthropic` guard** — asks a Claude model (default `claude-sonnet-4-5`) for a strict JSON safety verdict over GuardMeter's category vocabulary; needs `guardmeter[llm]` and `ANTHROPIC_API_KEY`. Malformed or failed responses fall back to a safe `pass`.
- **`llamaguard` guard** — runs Llama Guard 3 via a local `transformers` pipeline or an HTTP endpoint; needs `guardmeter[hf]` or a hosted endpoint and key.
- **Regulatory mapping (informational).** The HTML report includes a table mapping a run's metrics to regulatory themes (e.g. EU AI Act articles, NIST AI RMF). It is an informational aid for your own documentation, **not** a compliance certification or legal assessment.

---

## Development Setup

```bash
python3.13 -m venv .venv
source .venv/bin/activate      # Windows: .venv\Scripts\activate
pip install -e ".[dev]"
ruff check guardmeter tests
mypy guardmeter
pytest tests/guardmeter/ -q
```

> **Note:** macOS users with Homebrew Python must use a virtual environment (Homebrew enforces PEP 668).

---

## Not affiliated with

This project is unrelated to the JRC "GuardBench" toxicity-benchmark library at [github.com/AmenRa/guardbench](https://github.com/AmenRa/guardbench). Same name, different project.

Formerly published as `sea-guard` (versions 0.1–0.2, import name `guardbench`). Renamed in 0.3.0 to avoid confusion with the unrelated JRC GuardBench benchmark.

---

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md). Contributions welcome — new guard adapters, dataset/language coverage, and report improvements especially.

## License

MIT — see [LICENSE](LICENSE).

## Branding

Logo assets are in the `branding/` directory.

- `guardmeter-logo.svg` — shield mark (favicon, PyPI, GitHub avatar)
- `guardmeter-wordmark.svg` — full lockup with tagline
- `guardmeter-social-card.svg` — 1280×640 OG image for GitHub social preview

---

*Built by [SeaTechOne LLC](https://seatechone.com) · Seattle, WA*
