Metadata-Version: 2.5
Name: llmground
Version: 0.1.0
Summary: Verify that LLM-extracted field values actually appear in the source document.
Author-email: Anas Muhammad Abid Khan <anasabidkhan.tech@gmail.com>
License: MIT
License-File: LICENSE
Keywords: extraction,grounding,hallucination,llm,pydantic,rag,validation
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.10
Provides-Extra: dev
Requires-Dist: pydantic>=2; extra == 'dev'
Requires-Dist: pytest>=7; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Description-Content-Type: text/markdown

# llmground

**Your schema validated. The value was still made up.**

Structured output libraries guarantee the *shape* of extracted data. None of
them check whether the values are actually in the document. llmground runs
after extraction and tells you which fields are real.

```python
from llmground import verify

report = verify(extracted, source_document)

if report.has_hallucinations:
    route_to_human_review(report.ungrounded)
```

No API calls. No model. No cost. Zero dependencies.

## The bug this catches

```python
extracted = {
    "vendor": "ACME Manufacturing Ltd.",
    "total": 2846.25,          # the invoice says 2,486.25
}
```

Right types. Passes any JSON schema. Passes Pydantic. The number is a
transposition the model invented, and it goes straight into your accounting
system. Shape validation cannot see this, because there is nothing wrong with
the shape.

```
GROUNDED    vendor = 'ACME Manufacturing Ltd.'
            found: ACME Manufacturing Ltd. 14 Sundar Industrial Estate...
UNGROUNDED  total = 2846.25
            nearest number in source is '$2,486.25'

6/8 fields grounded (75%), 0 weak, 2 ungrounded
```

## Install

```bash
pip install llmground
```

## It works with what you already use

llmground doesn't replace Instructor, Outlines, or PydanticAI. It runs after
them. Pydantic models, dataclasses, and plain dicts all work, and Pydantic is
optional rather than required.

```python
import instructor
from llmground import verify

invoice = client.chat.completions.create(response_model=Invoice, ...)

report = verify(invoice, source_pdf_text)
if report.grounding_rate < 0.9:
    escalate(report)
```

## Why matching values is harder than it looks

The naive version of this check is `str(value) in source`, and it produces so
many false alarms that people switch it off within a day. Real documents don't
write values the way models return them.

| Model returns | Document says | Naive check | llmground |
|---|---|---|---|
| `4500.0` | `$4,500.00` | miss | matched |
| `"2026-01-05"` | `5 January 2026` | miss | matched |
| `"O'Brien Ltd"` | `O'Brien Ltd` (smart quote) | miss | matched |
| `1234.56` | `1.234,56` (European) | miss | matched |
| `-1234.00` | `(1,234.00)` (accounting) | miss | matched |
| `"ACME Inc"` | `ACME, Inc.` | miss | matched |

Every one of these is handled deterministically, with no model in the loop.

## Verdicts

| Verdict | Meaning |
|---|---|
| `grounded` | Found in the source |
| `weak` | Found only by fuzzy match. A human should look. |
| `ungrounded` | Not in the source. Likely fabricated. |
| `skipped` | Not verifiable: booleans, nulls, values like `"1"` or `"yes"` |

**Fuzzy matches are `weak`, not `grounded`, by default.** A 90%-similar string
is a different string, and on a financial or medical field that difference is
the entire problem. Opt in with `accept_fuzzy=True` when you're handling OCR
noise and know what you're trading away.

**Skipped fields are excluded from the grounding rate.** Counting booleans as
grounded would inflate the score and hide real problems.

## Spans, for human review

Every match reports character offsets into the source, so you can highlight the
evidence rather than just flagging a field.

```python
for r in report.grounded:
    span = r.match.spans[0]
    highlight(source, span.start, span.end)
```

That's the difference between telling a reviewer "this field is suspicious" and
showing them why.

## CLI

```bash
llmground extracted.json --source invoice.txt --fail-under 0.9
```

Exits non-zero below the threshold, so it drops into CI as a quality gate.

## Options

```python
verify(
    extracted,
    source,
    accept_fuzzy=False,        # count fuzzy matches as grounded
    fuzzy_threshold=0.85,      # similarity floor for reporting a fuzzy match
    number_tolerance=0.01,     # allowed numeric difference
    day_first=True,            # how to read 03/04/2026
    ignore=["metadata.run_id"],  # skip computed fields
)
```

## Limitations

- **Extractive only.** It verifies values that should appear in the source. A
  field that's legitimately inferred rather than copied (a category label, a
  computed total) will read as ungrounded. Use `ignore` for those.
- **Ambiguous dates are not guessed.** `03/04/2026` returns no match unless you
  pass `day_first`, because a silently wrong date is worse than an unverified
  one.
- **Grounded is not correct.** It confirms a value appears in the document, not
  that the model pulled it from the right place. A total that matches a
  different line item's amount will pass.
- **Text sources only.** PDFs and images need to be through OCR first.

## Development

```bash
pip install -e ".[dev]"
pytest
```

MIT.
