Metadata-Version: 2.4
Name: veridia
Version: 0.1.0
Summary: Veridia - a nutrition label for data: every computed number gets an ancestry.
License: Apache-2.0
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: pandas>=2.0
Requires-Dist: pyarrow>=15
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Provides-Extra: duckdb
Requires-Dist: duckdb>=1.0; extra == "duckdb"
Provides-Extra: postgres
Requires-Dist: psycopg[binary]>=3; extra == "postgres"
Provides-Extra: xlsx
Requires-Dist: openpyxl>=3.1; extra == "xlsx"
Provides-Extra: sign
Requires-Dist: cryptography>=42; extra == "sign"
Provides-Extra: dbt-demo
Requires-Dist: dbt-duckdb>=1.9; extra == "dbt-demo"
Dynamic: license-file

# Pedigree

[![CI](https://github.com/kanishkg91/pedigree/actions/workflows/ci.yml/badge.svg)](https://github.com/kanishkg91/pedigree/actions/workflows/ci.yml)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
<!-- PyPI badge seam: add once the v0.1.0 post-publish smoke is green (docs/releasing.md):
[![PyPI](https://img.shields.io/pypi/v/veridia.svg)](https://pypi.org/project/veridia/) -->

**A nutrition label for data.** Pedigree records the ancestry of every
computed number — which rows, files, code, and versions produced it — and
lets anyone ask a number why it is what it is. Answers are verified by
actually re-running the recorded steps, and every answer carries an honest
confidence label. It is open, local-first infrastructure: Apache-2.0,
SQLite, SHA-256 — no server, no account, no network, no telemetry.

The package name is **`veridia`** (the import and CLI stay `pedigree`):
`pip install "veridia[xlsx]"` — [docs/install.md](docs/install.md) has
the extras and an install-from-checkout form.

## Sixty seconds: find the number somebody typed in

A Q2 revenue workbook. Four regional totals roll up to $985,000. Three are
real `=SUM(...)` formulas; one was typed in by hand — padded $95K over its
actual deals. On the screen they are indistinguishable. Three commands:

```
pedigree ingest xlsx demo/excel_audit/revenue_review.xlsx
pedigree why "xlsx:demo/excel_audit/revenue_review.xlsx#Summary!B6"
pedigree why "xlsx:demo/excel_audit/revenue_review.xlsx#Summary!B5"
```

```
value: 985000
[Recorded]

xlsx.formula  -> .../Summary!B6  [Recorded]
  xlsx.formula  -> .../Summary!B2  [Recorded]     <- North: calculated
    source: .../Detail!C2:C3  [Source]
  xlsx.formula  -> .../Summary!B3  [Recorded]     <- South: calculated
    source: .../Detail!C4:C5  [Source]
  xlsx.formula  -> .../Summary!B4  [Recorded]     <- East: calculated
    source: .../Detail!C6:C8  [Source]
  source: .../Summary!B5  [Source]                <- West: a dead end

no formula cell Summary!B5 recorded in revenue_review.xlsx
```

West has no formula behind it. Somebody typed it. Pedigree found the one
hand-entered number in the report — the classic hardcoded "plug" — in one
click. The full walkthrough is [`demo/excel_audit/SHOW.md`](demo/excel_audit/SHOW.md);
the 15-minute meeting version is [`docs/demo-script.md`](docs/demo-script.md).

## The honesty rule

Every answer is labeled, and the label is load-bearing:

- **`Exact`** — derived by verified replay: every step re-run (twice),
  fingerprints checked against the recorded run.
- **`Recorded`** — step-level truth from receipts; no row-level claim.
- **`Opaque`** — a black-box step (a UDF, an unknown frame, unanalyzable
  SQL). The chain stays connected and labeled, never invented.

Pedigree never silently guesses. If a replay can't be verified it says so
and degrades honestly — a confident wrong ancestry is treated as the worst
possible bug in the project.

## What it is not

Read this before the feature list — same words as the
[honest comparison](docs/compare.md):

- **Not a catalog.** No ownership, discovery, glossary, or governance
  workflows. If your shop runs DataHub or Collibra, Pedigree *complements*
  them — it answers a different question. The word is **and**, never
  *instead*.
- **Coverage is narrow by design.** Live capture for pandas, DuckDB, and
  Postgres; ingestion for Excel workbooks, dbt runs, and declared-IO Python
  tasks. No Snowflake, Spark, SQL Server, or BI tools — and no promised
  dates for any of them.
- **Opaque is common in real pipelines.** First contact with a messy
  codebase will show `Opaque` steps. That label is the tool working.
- **Single-machine.** No shared or team ledger. The sharing mechanism is
  the evidence pack (`pedigree export`) — one self-contained HTML file.

## How it works

**Record cheaply always; reconstruct exactly, only when asked.**

1. **Receipts (always on, near-free).** Every computation step emits a tiny
   content-addressed record naming its inputs, code, and outputs *by
   fingerprint* — never copying data. Measured overhead on the benchmark
   pipeline: **~3.5% median** (budget: 5%, enforced by a test gate forever;
   see [docs/limits.md](docs/limits.md) for measured numbers at scale).
2. **Replay (on demand, exact).** When you ask about a number, the resolver
   walks the receipt chain backward and re-runs only the steps that matter
   against the recorded input versions, deriving row-level detail.
   Expensive, but paid at ask time and cached.

Chains cross process and system boundaries by **content fingerprint
handshake**: a file written by pandas and read by DuckDB has the same
fingerprint on both receipts, so `pandas -> file -> SQL -> report` traces
as one chain. Nothing is stitched by filename, timestamp, or guesswork.

## What works today (v0.1)

- **Taps** (add `import pedigree` to a script — that is the only change):
  **pandas** (read/merge/filter/groupby/reduce/write), **DuckDB** and
  **PostgreSQL** (inputs discovered engine-side, never by regexing your
  SQL; probes can never alter your results or transactions).
- **Ingesters** (for systems that aren't a Python process): **Excel**
  (`pedigree ingest xlsx`, cell-level ancestry from the formula graph),
  **dbt** (`pedigree ingest dbt` after a run), **Airflow-style tasks**
  (a `pedigree_task` decorator), and `pedigree run -- <cmd>` to wrap any
  command as one receipt.
- **Questions:** `why` (step- and row-level ancestry, per-row influence),
  `verify` (drift check), `replay` (exact re-execution), `diff` (why a
  number changed between runs), `export` (a self-contained HTML evidence
  pack for workpapers), `serve` (local viewer on 127.0.0.1).
- **Audit-grade history:** the ledger is hash-chained (`pedigree audit`
  detects removed or edited history and names the first bad entry), with
  opt-in Ed25519 receipt signing. Receipts are append-only forever.
- **A public standard:** the receipt format is a frozen, versioned spec
  ([`spec/receipt-v1.md`](spec/receipt-v1.md) + JSON Schema) with a
  conformance validator (`pedigree validate`) and an adapter SDK —
  [anyone can write a tap](docs/writing-a-tap.md).

## Install

See **[docs/install.md](docs/install.md)** — pip/pipx, optional extras
(`[duckdb]`, `[postgres]`, `[xlsx]`, `[sign]`), and a section for IT
departments. Python 3.11+. From a checkout: `pip install -e ".[xlsx]"`.

Then try it in 60 seconds (the golden two-process demo):

```
python demo/generate_data.py
python demo/script_a.py                  # pandas: clean + join -> parquet
python demo/script_b.py                  # pandas: aggregate -> report.json
pedigree why report:total_revenue        # the whole chain, across processes
pedigree why report:total_revenue --rows # the 1000 contributing rows [Exact]
pedigree audit                           # the history is tamper-evident
```

## Supported platforms

The support matrix is **exactly the set of combinations CI runs and nothing
more** ([docs/install.md](docs/install.md) keeps the authoritative copy):

| OS | Python | pandas | Proven by |
|---|---|---|---|
| Windows, Ubuntu, macOS | 3.11–3.14 | current 3.x | fast suite green per OS x version |
| Ubuntu | 3.11 | 2.0.x floor (`numpy<2`) | fast suite green |
| Ubuntu | 3.12 | current 3.x | fast suite + live PostgreSQL 16 |

CI also proves the demo's artifact fingerprints are **byte-identical across
all three OSes** on every push.

## Documentation

- **[Install](docs/install.md)** · **[Quickstart (10 min)](docs/quickstart.md)**
  · **[Reading a Pedigree answer](docs/reading-answers.md)** (start here if
  you're not a programmer)
- **[CLI reference](docs/cli.md)** · **[FAQ](docs/faq.md)** ·
  **[Limits (measured)](docs/limits.md)** · **[Operations](docs/operations.md)**
- **Security & compliance:** **[Security whitepaper](docs/security.md)**
  (every claim linked to a test or measurement) ·
  **[Compliance mapping](docs/compliance-mapping.md)** (not legal advice —
  says so up front)
- **Making the case internally:** **[Honest comparison](docs/compare.md)** ·
  **[Two-week pilot playbook](docs/pilot-playbook.md)** ·
  **[15-minute demo script](docs/demo-script.md)**
- **Extending:** **[Writing a tap](docs/writing-a-tap.md)** ·
  **[Support policy](docs/support-policy.md)**

## Honest limitations

- Row-level answers stop at opaque steps (UDFs, in-place mutations); past
  them you get `Recorded` with a plain-language reason, not a guess.
- Large intermediates use sampled fingerprints (honest-but-weaker, and
  marked as such); SQL row-level tracing is step-level only.
- Macro-enabled workbooks are refused (a macro can set any cell); volatile
  and dynamic Excel constructs degrade honestly to `Opaque`.
- Signing key files cannot be POSIX-locked on Windows (documented caveat).
- The full register, each with its measured number or test:
  [docs/limits.md](docs/limits.md) and [docs/security.md](docs/security.md) §9.

## Sustainability & the exit story

From the [support policy](docs/support-policy.md), verbatim:

> Pedigree is maintained by one person. If maintenance stops, adopters lose
> nothing they cannot keep: the receipt spec is frozen, public, and
> versioned (`spec/receipt-v1.md` + JSON Schema); the ledger is a plain
> SQLite file of canonical JSON on your own disk; the conformance validator
> and SDK are Apache-2.0; there is no server, no account, and no network
> dependency that can be turned off. Any competent engineer can read a
> ledger with sqlite3 and the schema in an afternoon. **The exit is: copy
> your file.**

Security reports: see [SECURITY.md](SECURITY.md).
