Metadata-Version: 2.4
Name: lightrider-huggingface
Version: 0.1.0
Summary: Hugging Face datasets with provable, reproducible origin: seeded from a named quantum entropy source through Light Rider, with the EMS-signed receipt.
Author: Light Rider
License: Apache-2.0
Project-URL: Homepage, https://entropy.lightriderinc.com/developers
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: lightrider>=1.6.0
Requires-Dist: huggingface_hub>=0.23
Requires-Dist: datasets>=2.19
Requires-Dist: pyarrow>=14
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"

# lightrider-huggingface

Hugging Face datasets with **provable, reproducible origin**. Each dataset is built from a 32-byte
seed drawn from a named quantum entropy source through [Light Rider](https://entropy.lightriderinc.com/developers).
The draw's EMS-signed receipt commits to those exact bytes. Anyone can check where the dataset came
from and regenerate it exactly, without a Light Rider account.

This is an alternative entropy source with verifiable provenance. It is not a claim that quantum
randomness makes a dataset better than one built with a good classical generator.

## Quickstart

```bash
pip install lightrider-huggingface
export LR_API_KEY=lr_live_...   # Light Rider API key (Light Rider Cloud > Settings > API Keys)
export HF_TOKEN=hf_...          # Hugging Face token with write access
```

```python
import lightrider_huggingface as lrhf

# 500 random circuits for benchmarking: 5 qubits, depth 10 (one draw, 1 credit)
ds = lrhf.circuits(500, num_qubits=5, depth=10, gate_set=["h", "sx", "rz", "cx"], source="anu")

# 200 random rows of an existing dataset's test split, pinned to today's commit
sub = lrhf.eval_subset("openai/gsm8k", "test", n=200, config="main", source="anu")   # or fraction=0.1

lrhf.publish(ds, "my-org/random-circuits-5q")      # private by default; parquet + card + receipt
print(lrhf.verify("my-org/random-circuits-5q"))     # anyone can run this; no Light Rider key needed
```

`ds.table` is a `pyarrow.Table` (`ds.to_pandas()` for pandas), and `ds.provenance` is the full
record. `ds.save("dir")` writes the same files locally, and `lrhf.verify_dir("dir")` checks them.

## What gets published

| File | Contents |
|---|---|
| `data/<split>-00000-of-00001.parquet` | the rows |
| `README.md` | dataset card: provenance table, honest claims, how to verify |
| `lightrider_provenance.json` | seed, commitment salt, receipt, PRNG and version, generator, params, content hash, package versions |
| `receipt.json` | the EMS-signed receipt, exactly as received |

**Random circuits** (`random_circuits/1`): one row per circuit with `circuit_id`, `num_qubits`,
`depth`, `num_gates`, `num_two_qubit_gates`, `instructions` (JSON, the `lightrider.Circuit` format)
and `qasm` (OpenQASM 2.0). Each of the `depth` layers acts on every qubit. Two-qubit gates pair
neighbours in a fresh random permutation, and angles are uniform in [0, 2π). The full rule is in
`random_circuits.py`.

**Evaluation subsets** (`eval_subset/1`): `n` distinct rows of one split, unchanged (images and
audio stay encoded), plus `_lr_source_index`. The source is pinned to an exact commit sha, so the
same rows can always be fetched again.

## How the proof works

1. **The seed is one draw's output.** EMS signs `output_commitment = sha256(salt || bytes)` into
   the receipt and returns the salt with the bytes. The salt keeps short draws from being
   brute-forced out of a public receipt. The dataset publishes seed, salt and receipt, so
   `sha256(salt || seed) == receipt.output_commitment` ties the seed to that signed receipt. A
   genuine receipt from some other draw doesn't match.
2. **The receipt is genuine.** Its Ed25519 signature verifies against the key published at
   `https://entropy.lightriderinc.com/api/v1/public-key`. The signed message is the receipt without
   `signature`, keys sorted, compact JSON.
3. **The rows follow from the seed.** Generation uses LR-ChaCha20 v1 (`prng.py`), a short written
   spec on the RFC 8439 ChaCha20 stream. NumPy is deliberately not used: its `Generator` output may
   change between releases. Regeneration is compared row for row through a canonical content
   hash. When the verifier runs the same pyarrow version, the parquet file is also compared byte
   for byte.

What it can't show: that the publisher drew only once and didn't pick a favourite draw. The
receipt's timestamp shows when this draw happened.

`verify(repo_id, indices_only=True)` checks a subset's row indices without re-downloading a large
source split.

## Tests

```bash
pip install -e ../lightrider -e ".[dev]"
pytest                    # mocked entropy site and mocked Hub
LRHF_LIVE=1 LR_API_KEY=lr_live_... HF_TOKEN=hf_... pytest tests/test_live.py   # production + Hub, 1 credit
```

The live test publishes to the private repo `LightRiderInc/lr-hf-livetest`. It overwrites the
repo on every run and never deletes it.

See `examples/provenance_datasets.py` for a runnable example.
