Metadata-Version: 2.4
Name: ragas-openttt
Version: 0.1.0
Summary: TTTPS Proof-of-Time provenance metric for Ragas: verify that a response carries a genuine audit-trail timestamp from the public KPP Provenance API
License: MIT
Project-URL: Homepage, https://github.com/Helm-Protocol/OpenTTT
Project-URL: Repository, https://github.com/Helm-Protocol/OpenTTT
Project-URL: Issues, https://github.com/Helm-Protocol/OpenTTT/issues
Keywords: ragas,proof-of-time,provenance,temporal-attestation,audit-trail,llm-evaluation,metric
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: httpx>=0.24
Requires-Dist: ragas<0.5,>=0.4.3
Requires-Dist: langchain-community<0.4.2,>=0.3

# ragas-openttt

TTTPS Proof-of-Time provenance metric for [Ragas](https://github.com/explodinggradients/ragas).
It answers one question about a sample: does this response have a genuine
Proof-of-Time anchor on the public self-serve
[KPP Provenance API](https://kpp.kenosian.com), and does that anchor cover
this exact response?

Deterministic and rule based, in the same family as ragas' own `ExactMatch`
and `StringPresence`. There is no LLM judge and no model call.

This checks a cryptographic audit-trail timestamp and integrity hash. It does
**not** certify legal or regulatory compliance (EU AI Act, FDA, and so on).
Treat it as an audit-trail timestamp, not a compliance claim.

## Install

```bash
pip install ragas-openttt
```

The dependency list pins `langchain-community<0.4.2` on purpose. ragas 0.4.3
imports `langchain_community.chat_models.vertexai` unconditionally at import
time and langchain-community removed that module in 0.4.2, so without the
bound `import ragas` raises `ModuleNotFoundError` on a fresh install.
Measured 2026-08-05: the module is present in 0.4 and 0.4.1, absent from
0.4.2. Drop the bound once ragas no longer needs it.

## Usage

The receipt id travels in `retrieved_context_ids`, as a one element list.
`SingleTurnSample` is a closed pydantic model with no free metadata field,
and a subclass that adds one is rejected by ragas' strict sample type check,
so this is the one slot that keeps a sample usable by real
`ragas.evaluate()`.

```python
from ragas import evaluate
from ragas.dataset_schema import EvaluationDataset, SingleTurnSample
from ragas_openttt import TTTPSProvenanceMetric

sample = SingleTurnSample(
    user_input="how tall is the Eiffel Tower?",
    response="The Eiffel Tower is 330 metres tall.",
    retrieved_context_ids=[receipt_id],       # from POST /v1/anchor
)

result = evaluate(EvaluationDataset(samples=[sample]), metrics=[TTTPSProvenanceMetric()])
print(result)
```

To seal responses that have no receipt yet, pass a key (mint one for free
with `POST https://kpp.kenosian.com/v1/keys`):

```python
TTTPSProvenanceMetric(anchor_if_missing=True, kpp_key="...")   # or KPP_API_KEY
```

## What it checks, and what it cannot

The API is the source of truth. Given a receipt id the metric calls
`POST /v1/verify`, requires `verified: true`, and with `bind_content=True`
(the default) requires the anchored digest to equal
`sha256(sample.response)`.

That design has a consequence worth stating plainly: the metric never reads a
receipt blob held by your pipeline, so edits to that blob are invisible to
it. If the anchor on the server is intact, the metric says so, even when the
receipt JSON you display to a user has been rewritten.

Measured 2026-08-05 over 25 rows of the published corpus
[Pittro/verifiable-ai-provenance-bench](https://huggingface.co/datasets/Pittro/verifiable-ai-provenance-bench)
plus 5 negatives built by mutating real receipts:

| case | `bind_content=False` | default (`bind_content=True`) | `check_receipt_blob()` |
|---|---|---|---|
| genuine receipt lifted onto different output | 1.0 (missed) | 0.0 (caught) | 0.0 (caught) |
| receipt blob edited, one hex nibble flipped | 1.0 (missed) | 1.0 (missed) | 0.0 (caught) |
| timestamp rewritten one year into the past | 1.0 (missed) | 1.0 (missed) | 0.0 (caught) |
| receipt id that was never issued | 0.0 (caught) | 0.0 (caught) | 0.0 (caught) |
| no receipt at all | 0.0 (caught) | 0.0 (caught) | 0.0 (caught) |
| 25 published genuine receipts | 1.00 | 1.00 | not applicable |

Negatives caught by the metric: 2 of 5 without content binding, 3 of 5 with
the default. Separation between the genuine mean and the negative mean with
the default: 0.60.

The two misses are the two blob edits, for the reason above: the server copy
of both receipts is genuine, so the server says verified. If your pipeline
keeps or displays the receipt JSON, close that gap with the helper this
package ships, which reads the blob and caught 5 of 5 in the same run:

```python
from ragas_openttt import check_receipt_blob

ok, reason = check_receipt_blob(receipt_dict, response_text)
```

It checks content binding, existence, digest agreement and timestamp
agreement, and returns `(ok, reason)`.

Reproducing those rows needs only your own key: anchor a text, then (a)
attach the receipt to a different text, (b) flip one hex character of
`content_hash`, (c) rewrite `time` by a year, (d) invent a 24 hex character
receipt id, (e) attach nothing.

What none of it tells you: whether the response is correct, safe or useful.
A receipt proves the content existed no later than the anchored time, and
never that it did not exist earlier.

## Score semantics

Per sample the score is binary, 1.0 or 0.0. The continuous number a Ragas
report shows for a dataset is the mean over N samples, in other words a pass
rate, not a per sample confidence.

## Options

| argument | default | meaning |
|---|---|---|
| `bind_content` | `True` | require `sha256(response)` to equal the anchored digest |
| `anchor_if_missing` | `False` | seal the response when no receipt id is present |
| `kpp_key` | `""` | API key for anchoring, falls back to `KPP_API_KEY` |
| `kpp_base` | `None` | API base, falls back to `KPP_BASE` |
| `timeout_s` | `None` | request timeout in seconds, falls back to `KPP_TIMEOUT_S`, then 1.0 |
| `threshold` | `1.0` | pass threshold |

Set `bind_content=False` only for corpora that publish digests without the
plaintext they cover. With `bind_content=True` and no response text the
metric scores 0.0 rather than silently skipping the check.

Fail-open: a slow or unreachable API degrades the score and records why in
`metric._last_reason`. Scoring never raises out of the eval run. The metric
makes one `POST /v1/verify` round trip per sample.

## License

MIT
