Metadata-Version: 2.4
Name: groundguard-ai
Version: 0.4.0rc1
Summary: A local-first fact gate for tool-using AI agents.
Author: GroundGuard contributors
License-Expression: MIT
Project-URL: Homepage, https://github.com/chasen2041maker/GroundGuard
Project-URL: Documentation, https://chasen2041maker.github.io/GroundGuard/
Project-URL: Repository, https://github.com/chasen2041maker/GroundGuard
Project-URL: Issues, https://github.com/chasen2041maker/GroundGuard/issues
Project-URL: Changelog, https://github.com/chasen2041maker/GroundGuard/blob/main/CHANGELOG.md
Keywords: ai,agents,fact-checking,llm,testing
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Typing :: Typed
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: dev
Requires-Dist: build>=1; extra == "dev"
Requires-Dist: pytest>=7; extra == "dev"
Requires-Dist: twine>=5; extra == "dev"
Provides-Extra: openai
Requires-Dist: openai>=1; extra == "openai"
Provides-Extra: docs
Requires-Dist: mkdocs-material>=9; extra == "docs"
Provides-Extra: otel
Provides-Extra: langfuse
Provides-Extra: phoenix
Dynamic: license-file

<div align="center">

<h1>GroundGuard</h1>

<img src="https://raw.githubusercontent.com/chasen2041maker/GroundGuard/main/assets/brand/groundguard-logo-wordmark.png" alt="GroundGuard logo" width="420">

**Assertions for AI agent answers - deterministic, local, no LLM judge.**

Stop tool-using agents from ignoring the facts they already fetched. Important
numeric claims must trace back to facts recorded from this run, and required
tool facts cannot disappear silently.

Works with OpenAI | LangChain | LangGraph | PydanticAI | CrewAI | AutoGen | FastAPI | promptfoo | DeepEval | Langfuse | Phoenix | GitHub Actions

[![CI](https://github.com/chasen2041maker/GroundGuard/actions/workflows/ci.yml/badge.svg)](https://github.com/chasen2041maker/GroundGuard/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/groundguard-ai.svg?cacheSeconds=300&cacheBust=v0.3.1)](https://pypi.org/project/groundguard-ai/)
![Python](https://img.shields.io/badge/python-3.11%2B-blue)
![License](https://img.shields.io/badge/license-MIT-green)
![Status](https://img.shields.io/badge/status-pre--alpha-orange)
[![Discussions](https://img.shields.io/github/discussions/chasen2041maker/GroundGuard)](https://github.com/chasen2041maker/GroundGuard/discussions)

[PyPI](https://pypi.org/project/groundguard-ai/) | [Docs](https://chasen2041maker.github.io/GroundGuard/) |
[Examples](https://github.com/chasen2041maker/GroundGuard/tree/main/examples) | [Benchmark](https://github.com/chasen2041maker/GroundGuard/tree/main/benchmarks) |
[Releases](https://github.com/chasen2041maker/GroundGuard/releases) |
[Roadmap](https://github.com/chasen2041maker/GroundGuard/blob/main/PLAN.md) | [Discussions](https://github.com/chasen2041maker/GroundGuard/discussions) |
[Contributing](https://github.com/chasen2041maker/GroundGuard/blob/main/CONTRIBUTING.md)

English | [Chinese README](https://github.com/chasen2041maker/GroundGuard/blob/main/README.zh-CN.md)

</div>

**Benchmark signal:** GroundGuard catches `71/71` expected failures in a
200-case bilingual realistic dataset, with `0` false positives and `0` false
negatives, without using an LLM judge.

```python
from groundguard import FactGate

gate = FactGate()
gate.record_tool_result("q3_revenue", "5.2", "billion_usd")  # from a tool call
report = gate.check("Q3 revenue came in at $4.8 billion [fact:q3_revenue].", required=["q3_revenue"])
print(report.output_claims[0].status)  # contradicted
```

The answer says 4.8B; the ledger says 5.2B, so the claim is stopped before
release.

<div align="center">

<img src="https://raw.githubusercontent.com/chasen2041maker/GroundGuard/main/assets/demo.gif" alt="GroundGuard demo: omitted required facts blocked in red, corrected answer verified in green" width="880">

</div>

## Why GroundGuard?

Tool-using agents often fail in ways that look normal:

- A tool returns the right data, but the model says the data was unavailable.
- A tool returns no data, but the model invents a confident-looking number.
- A final answer cites numbers that cannot be traced to the current tool run.

The dangerous version is mundane: your agent fetched Q3 revenue, then writes
last year's number into a report for leadership. Nothing crashes. The answer
just looks professional enough to pass human review.

Tracing tools show what happened. LLM-as-judge tools score an answer after it is
written. GroundGuard fills a smaller gap: it gives the final answer a
deterministic, testable fact gate before you let it pass.

GroundGuard started from a problem I kept seeing in my own workflow: tools had
already returned the data, but the model's final answer still contained numbers
that were not reliably checked. Sometimes the model rewrote a number
incorrectly. Sometimes it omitted a key fact that the tool had already returned.
I wanted the path from tool data to final answer to be transparent and
traceable, with a ledger check that can confirm whether the generated numbers
match the facts that were actually retrieved.

In plain terms, GroundGuard works like bookkeeping for agent facts:

1. When a tool returns an important value, you explicitly record it in a local
   ledger.
2. When the model writes the final answer, GroundGuard extracts the numeric
   claims it made.
3. GroundGuard compares the answer against the ledger and reports what was
   verified, missing, invented, or contradicted.
4. Your policy decides whether to only flag the issue, strip unsafe claims, or
   block the answer before it reaches the user.

That means the model can still write naturally, but the key numbers have to
match the facts your tools actually returned.

## Quick Start

```bash
python -m pip install groundguard-ai
groundguard-init --template github-action
groundguard-demo
groundguard-benchmark
```

The PyPI distribution name is `groundguard-ai`; the Python import name remains
`groundguard`.

- Latest stable release: `0.3.1`
- Current pre-release: `0.4.0rc1`

## 10-Second Demo

```bash
python examples/financial_report_demo/run.py
```

The bundled demo reproduces the failure mode GroundGuard is built for:

```text
Before GroundGuard correction
-----------------------------
passed: False
verified: 0
unverified: 0
contradicted: 0
omitted_required: 2
policy_reason: omitted_required_count=2 > max_omitted_required=0

After fact-key correction
-------------------------
passed: True
verified: 2
unverified: 0
contradicted: 0
omitted_required: 0
```

## What Works Today

- In-memory `Ledger` with TTL filtering and JSONL persistence.
- Explicit `tool_call(...).record_facts(...)` registration.
- Deterministic numeric claim extraction with `[fact:key]` markers, Chinese
  amounts, percentages, compact English magnitudes, and common English USD
  formats.
- Extraction transparency: `CoverageReport.suspected_numbers`,
  `uncovered_numbers`, and `extraction_coverage` show which numeric-looking
  spans were seen but not covered by extractors.
- Pluggable extractor registry through `register_extractor(...)` /
  `unregister_extractor(...)`, plus request-scoped extractor lists for
  multi-tenant services that must not mutate global process state.
- Matching statuses: `verified`, `candidate_match`, `unverified`,
  `contradicted`, and `ambiguous`.
- Required fact coverage checks for "tool had data, model omitted it" failures.
- `CoverageReport` and configurable `Policy` evaluation.
- `grounded_generate()` with report return, blocking, conservative stripping,
  optional tagged-claim repair, and one-shot reask.
- `@grounded(...)` decorator for framework-free Python functions.
- `FactGate` high-level runtime API for config-driven record/check flows.
- `groundguard-init` starter templates for GitHub Actions, OpenAI, promptfoo,
  LangGraph, PydanticAI, CrewAI, AutoGen, and FastAPI.
- `groundguard-report` CLI with native and assertion-style JSON schemas,
  including per-claim `componentResults`, Markdown, HTML, and GitHub comment
  renderers.
- Built-in extractor packs for finance, SaaS, ecommerce, and ops metrics.
- Dedicated promptfoo and DeepEval adapter helpers.
- Dependency-free OpenTelemetry-style event export.
- OpenAI-compatible wrapper, LangChain-compatible callback, and LangGraph-style
  node examples.
- A composite GitHub Action for CI fact gates.
- A tiny dependency-free HTTP server entrypoint for gateway-style checks.

## Deterministic Checkers

`0.4.0rc1` adds request-scoped, synchronous checkers that are passed explicitly
to one `FactGate.check(...)` call. They are not globally registered and do not
call a network, database, or LLM.

This release candidate adds:

- the `Checker` protocol and structured `Issue` findings;
- `Fact` context fields for `subject`, `as_of`, `observed_at`, `source_field`,
  and `fact_type`;
- `OrphanNumberChecker` and `RelativeFreshnessChecker`;
- opt-in `groundguard.rules.finance_cn` checks; and
- backward-compatible `issues`, `hard_issue_count`, and `soft_issue_count`
  additions to `groundguard.report.v1` output.

An `Issue` with `severity="hard"` changes the report to `passed=false`; a
`severity="soft"` issue is diagnostic and leaves the prior pass/fail result
unchanged.

```python
from decimal import Decimal

from groundguard import FactGate
from groundguard.checkers import OrphanNumberChecker
from groundguard.rules.finance_cn import PriceDirectionChecker, PriceLimitChecker

gate = FactGate(session_id="synthetic_quote")
gate.record_tool_result("price", Decimal("10.50"), "CNY", subject="600000")
gate.record_tool_result("previous_close", Decimal("10.00"), "CNY", subject="600000")
gate.record_tool_result("change_pct", Decimal("5.00"), "%", subject="600000")

report = gate.check(
    "Synthetic quote checked.",
    checkers=(
        OrphanNumberChecker(),
        PriceDirectionChecker(),
        PriceLimitChecker(),
    ),
    context={
        "finance_cn": {
            "exchange": "SSE",
            "board": "main",
            "listing_phase": "normal",
            "trade_date": "2026-07-15",
            "subject": "600000",
        }
    },
)

assert report.passed
assert report.issues == ()
```

This example uses synthetic data. `finance_cn` is opt-in: import and construct
its checkers only when the caller provides explicit market context. It supports
SSE/SZSE main boards, SSE STAR, and SZSE ChiNext rules in this release. Missing
required context skips the hard price-limit calculation and produces only a
soft insufficient-context diagnostic. BSE, Hong Kong, US, and other markets
are intentionally unsupported. See the [limitations](https://chasen2041maker.github.io/GroundGuard/limitations/)
for source links, effective dates, and the 0.01 CNY half-up price-tick
calculation boundary.

## Installation

Install the latest stable release:

```bash
python -m pip install groundguard-ai
```

Install the current release candidate exactly:

```bash
python -m pip install --pre groundguard-ai==0.4.0rc1
```

For local development:

```bash
git clone https://github.com/chasen2041maker/GroundGuard.git
cd GroundGuard
python -m pip install -e ".[dev]"
python -m pytest
```

For the live OpenAI SDK example, install the optional SDK extra after cloning:

```bash
python -m pip install -e ".[openai]"
```

## Python API

```python
from decimal import Decimal

from groundguard import Ledger, Policy, grounded_generate, tool_call


def fetch_financials(ticker: str) -> dict[str, str]:
    return {
        "ticker": ticker,
        "net_profit": "823200000",
        "revenue": "3830000000",
    }


def fake_llm(prompt: str) -> str:
    return (
        "Revenue was $3.83 billion [fact:revenue_2025], "
        "and net profit was $823.2 million [fact:net_profit_2025]."
    )


with Ledger(session_id="req_001") as ledger:
    with tool_call("get_company_financials", {"ticker": "ACME"}, ledger) as call:
        result = fetch_financials("ACME")
        call.record_facts(
            {
                "net_profit_2025": (Decimal(result["net_profit"]), "USD"),
                "revenue_2025": (Decimal(result["revenue"]), "USD"),
            },
            raw=result,
        )

    result = grounded_generate(
        prompt="Summarize the latest financial performance.",
        llm_call=fake_llm,
        ledger=ledger,
        required_fact_keys=["net_profit_2025", "revenue_2025"],
        policy=Policy(on_unverified="flag"),
        return_report=True,
    )

print(result.answer)
print(result.report.passed)
```

## Core Concepts

| Concept | What It Means | Why It Matters |
| --- | --- | --- |
| `Fact` | A verifiable value explicitly registered from a tool call. | The only source GroundGuard treats as evidence. |
| `RequiredFact` | A fact key the current answer must cover. | Catches cases where the tool had data but the model ignored it. |
| `OutputClaim` | A numeric claim extracted from the final answer. | Lets GroundGuard verify what the model actually wrote. |
| `CoverageReport` | The final reconciliation report. | Shows verified, candidate, unverified, contradicted, omitted facts, and uncovered numeric-looking spans. |
| `Policy` | Pass/fail thresholds and handling behavior. | Lets you flag, strip, block, repair tagged contradictions, or reask once. |

## How It Works

```mermaid
flowchart LR
    A["Tool call"] --> B["Explicit fact registration"]
    B --> C["Ledger"]
    D["Model final answer"] --> E["Output claim extraction"]
    C --> F["Matcher"]
    E --> F
    F --> G["CoverageReport"]
    H["Required fact keys"] --> G
    G --> I["Policy: flag / strip / block / fix / reask"]
```

GroundGuard v1 is deterministic by design: no hosted service, no database, no
second LLM, and no token-level generation control claims.

Current claim extraction is intentionally narrow: it extracts numeric claims
that include a unit or magnitude marker, such as `$3.83 billion`,
`USD 10.25M`, `1.2M`, `3,830 million dollars`, `21.5%`, or
`21.5 percent`. Bare numbers without units are
not treated as verified claims to avoid false positives. They are still exposed
in `CoverageReport.uncovered_numbers`, so users can see what the deterministic
extractors did not cover.

Registered numeric facts are normalized before matching. For example,
`(Decimal("3.83"), "usd_b")`, `(Decimal("3830"), "usd_m")`, and
`(Decimal("3830000000"), "USD")` all compare as the same USD value.

Custom extractors can be registered when your domain needs additional claim
types:

```python
from groundguard import OutputClaim, register_extractor


@register_extractor("ticker_entity")
def extract_tickers(text: str) -> list[OutputClaim]:
    return []
```

For multi-tenant services, prefer request-scoped extractors so one tenant's
rules do not affect another tenant in the same Python process:

```python
from groundguard import extract_output_claims, registered_extractors

claims = extract_output_claims(
    answer,
    extractors=registered_extractors() | {"ticker_entity": extract_tickers},
)

report = ledger.coverage_report(
    answer,
    extractors=registered_extractors() | {"ticker_entity": extract_tickers},
)
```

`register_extractor(...)` is process-global and is best used at application
startup, not dynamically per request.

## Where It Fits

| Tool family | What it is great at | Where GroundGuard fits |
| --- | --- | --- |
| Langfuse / Phoenix | Traces, sessions, observability, debugging timelines. | Adds a deterministic final-answer gate before release. |
| promptfoo / DeepEval | Test suites, eval assertions, scorecards. | Emits assertion-style JSON so fact coverage can be one metric in those suites. |
| LLM-as-judge evaluators | Semantic quality checks and subjective grading. | Avoids a second model for facts that already came from tools. |
| Constrained decoding | Token-level format or grammar control. | Checks post-generation factual coverage; it does not claim token-level control. |

GroundGuard is deliberately narrow: it asks whether the final answer covered and
cited the facts your tools already returned.

## Benchmark

```bash
groundguard-benchmark
```

The bundled benchmark now runs two deterministic suites:

- a 25-case smoke suite for the core fact-gate contract,
- a 200-case bilingual realistic dataset covering English and Chinese outputs,
  currencies, percentages, basis points, users, orders, tickets, latency,
  storage units, candidate matches, omissions, contradictions, ambiguity, and
  bare-number extraction limits.

Expected signal:

```text
smoke.cases_total: 25
smoke.expected_failures: 14
smoke.detected_failures: 14
smoke.false_positives: 0
realistic_dataset.cases_total: 200
realistic_dataset.expected_failures: 71
realistic_dataset.detected_failures: 71
realistic_dataset.false_positives: 0
realistic_dataset.false_negatives: 0
```

## CLI

For application code, the shortest stable entrypoint is `FactGate`:

```python
from decimal import Decimal

from groundguard import FactGate

gate = FactGate.from_config("groundguard.yml")
gate.record_tool_result("revenue_2025", Decimal("3.83"), "billion_usd")

report = gate.check(
    "Revenue was $3.83 billion.",
    required=["revenue_2025"],
)

assert report.passed
```

Generate a JSON coverage report from a ledger JSONL file and an answer file:

```bash
groundguard-report \
  --ledger-jsonl facts.jsonl \
  --answer-file answer.txt \
  --required-fact net_profit_2025 \
  --required-fact revenue_2025 \
  --fail-on-policy
```

Without an installed console script:

```bash
python -m groundguard.cli.report --ledger-jsonl facts.jsonl --answer-file answer.txt
```

Use a config file for CI and repeatable eval runs:

```bash
cp groundguard.example.yml groundguard.yml
groundguard-report \
  --config groundguard.yml \
  --ledger-jsonl facts.jsonl \
  --answer-file answer.txt
```

Emit a promptfoo/DeepEval-friendly assertion payload:

```bash
groundguard-report \
  --ledger-jsonl facts.jsonl \
  --answer-file answer.txt \
  --required-fact net_profit_2025 \
  --schema assertion
```

The assertion schema includes `pass`, `success`, `score`, `reason`,
`namedScores`, top-level `claims`, per-claim `componentResults`, and the full
GroundGuard report under `metadata.groundguard`. Each claim exposes
`text_span`, `start`, `end`, `status`, `matched_fact_key`, `ledger_value`,
`answer_value`, and `diff` for downstream UI highlighting.

Render human-readable artifacts for PRs and audits:

```bash
groundguard-report \
  --ledger-jsonl facts.jsonl \
  --answer-file answer.txt \
  --required-fact revenue_2025 \
  --format markdown \
  --output groundguard-report.md

groundguard-report \
  --ledger-jsonl facts.jsonl \
  --answer-file answer.txt \
  --format html \
  --output groundguard-report.html
```

`groundguard.yml` can also select scoped extractor packs:

```yaml
extractors:
  packs:
    - finance
    - saas
    - ops
units:
  tolerance: 0.005
report:
  schema: assertion
  format: github
```

## Schema Compatibility

GroundGuard treats `Fact`, `OutputClaim`, `CoverageReport`, `Policy`,
`AssertionReport`, and `DatasetCase` as public protocol objects. Each object
includes `schema_version`. GroundGuard will not remove or rename fields within a major version; new fields may be added with defaults so existing integrations
continue to read older payloads.

Published schema files:

- [`groundguard.report.v1`](https://github.com/chasen2041maker/GroundGuard/blob/main/schemas/groundguard.report.v1.schema.json)
- [`groundguard.config.v1`](https://github.com/chasen2041maker/GroundGuard/blob/main/schemas/groundguard.config.v1.schema.json)

## GitHub Action

Use the composite action in another repository:

```yaml
- name: Run GroundGuard
  uses: chasen2041maker/GroundGuard@v0.3.1
  with:
    ledger-jsonl: groundguard-ledger.jsonl
    answer-file: answer.txt
    config: groundguard.yml
    required-facts: net_profit_2025,revenue_2025
    schema: assertion
    fail-on-policy: "true"
```

For a PR comment example, see the [GitHub Actions PR-comment example](https://github.com/chasen2041maker/GroundGuard/blob/main/docs/examples/github-actions/pr-comment.yml).

## Adapters

OpenAI-compatible chat wrapper:

```python
from groundguard.adapters import openai_chat_llm

llm_call = openai_chat_llm(
    client.chat.completions.create,
    model="gpt-4.1-mini",
)
```

LangChain-compatible callback handler:

```python
from decimal import Decimal

from groundguard.adapters import GroundGuardCallbackHandler

handler = GroundGuardCallbackHandler(
    ledger=ledger,
    fact_mapper=lambda output, context: {
        "net_profit_2025": (Decimal(output["net_profit"]), "CNY"),
    },
)
```

The callback handler intentionally requires an explicit `fact_mapper`; v1 does
not guess which arbitrary JSON fields are evidence.

promptfoo / DeepEval helper functions:

```python
from groundguard.integrations.deepeval import to_deepeval_result
from groundguard.integrations.promptfoo import to_promptfoo_assertion

promptfoo_payload = to_promptfoo_assertion(report)
deepeval_payload = to_deepeval_result(report)
```

## Runnable Examples

Packaged commands:

```bash
groundguard-demo
groundguard-demo --json
groundguard-benchmark
```

Repository examples after cloning:

```bash
python examples/financial_report_demo/run.py
python examples/decorator_demo/run.py
python examples/langgraph_node/run.py
python examples/openai_demo/run.py
python examples/promptfoo_groundguard/run.py
python examples/deepeval_groundguard/run.py
```

`examples/openai_demo/run.py` records tool facts, shows an answer blocked for
omitting those required facts, then shows the corrected fact-key answer passing.

For a live OpenAI SDK call:

```bash
export OPENAI_API_KEY=...
python examples/openai_demo/run.py --live-openai
```

## What GroundGuard Is Not

- Not a tracing or observability platform; export reports to Langfuse, Phoenix,
  or OpenTelemetry instead.
- Not an LLM-as-judge or general hallucination detector; it checks deterministic
  ledger-vs-answer assertions.
- It does not call a second LLM by default, including when deterministic
  checkers are enabled.
- Not a database or hosted service; it runs locally, and important numbers need
  a unit, magnitude, or fact marker to be checked.

## Roadmap

- **Milestone 1: Core library** - Ledger, claim extraction, matching, policy,
  `grounded_generate`, and the financial report demo. Implemented.
- **Milestone 2: Framework adapters** - OpenAI wrapper, LangChain callback,
  LangGraph-style node example, and native `@grounded(...)` decorator. Starter
  coverage is implemented; more framework recipes are welcome.
- **Milestone 3: CI integration** - Assertion-style JSON, composite GitHub
  Action, PR comment workflow example, promptfoo/DeepEval helper adapters, and
  per-claim component results. Implemented.
- **Milestone 4: Visualization** - Deferred until the core library and
  distribution path are stable.

## Documentation

- [Docs home](https://chasen2041maker.github.io/GroundGuard/)
- [Getting started](https://chasen2041maker.github.io/GroundGuard/getting-started/)
- [CLI and config](https://chasen2041maker.github.io/GroundGuard/cli/)
- [Recipes](https://chasen2041maker.github.io/GroundGuard/recipes/)
- [Comparisons](https://chasen2041maker.github.io/GroundGuard/comparisons/)
- [Production guide](https://chasen2041maker.github.io/GroundGuard/production/)
- [Benchmark](https://chasen2041maker.github.io/GroundGuard/benchmark/)
- [Good first issues](https://chasen2041maker.github.io/GroundGuard/good-first-issues/)
- [Architecture](https://github.com/chasen2041maker/GroundGuard/blob/main/ARCHITECTURE.md)
- [Financial report demo](https://github.com/chasen2041maker/GroundGuard/tree/main/examples/financial_report_demo)
- [Brand assets](https://github.com/chasen2041maker/GroundGuard/tree/main/assets/brand)
- [Launch kit](https://github.com/chasen2041maker/GroundGuard/tree/main/docs/launch)
- [Changelog](https://github.com/chasen2041maker/GroundGuard/blob/main/CHANGELOG.md)
- [Contributing guide](https://github.com/chasen2041maker/GroundGuard/blob/main/CONTRIBUTING.md)

## Security

Ledger data may contain prompts, tool outputs, and sensitive business data.
GroundGuard is local-first and does not upload data by default. Always anonymize
fixtures, reports, and examples before sharing them publicly.

## Contributing

Contributions are welcome, especially:

- anonymized "tool had data, model missed it" examples
- claim extraction and matching improvements
- framework integration examples
- API design feedback

Please read the [contributing guide](https://github.com/chasen2041maker/GroundGuard/blob/main/CONTRIBUTING.md)
before opening a larger pull request.

## License

GroundGuard is released under the [MIT License](https://github.com/chasen2041maker/GroundGuard/blob/main/LICENSE).
