Metadata-Version: 2.4
Name: modelcertify
Version: 0.1.0
Summary: Supply-chain certification for trained models
License: Apache-2.0
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: 3.15
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Requires-Dist: ruff>=0.5; extra == "dev"
Requires-Dist: mypy>=1.10; extra == "dev"
Dynamic: license-file

# modelcertify

Supply-chain certification for trained models.

A model is only as trustworthy as the run that produced it. modelcertify
makes that run auditable and certifies it against a written policy.

* A **run proof** hashes every artefact the run produced (config, metrics,
  checkpoints, weights) and combines the hashes into a Merkle **root**. Change
  one byte of one artefact and the root changes.
* An **attestation** is a signed statement by an issuer that they vouch for a
  proof root.
* A **certificate** is a policy evaluated against a run's evidence: it
  records every gate, how each one came out, the proof root it is bound to,
  and the issuer who signed it.

The certificate never overstates. It carries the gate results, so a reader can
see *why* something was certified, not just that it was.

## Install

```bash
pip install modelcertify
pip install -e ".[dev]"           # from a checkout
```

modelcertify has **no runtime dependencies**: it is stdlib-only on purpose, so
it can be dropped into an air-gapped review environment.

## Relationship to iroh-tinker

modelcertify is a standalone product. It does not import iroh-tinker, and it
does not need to: it certifies a run's *artefacts*, wherever they were
produced.

The natural pairing is that an iroh-tinker run writes its artefacts (the
launch spec, the metrics, the checkpoints) to a directory, and
`modelcertify proof --dir <that directory>` turns them into a proof.

## CLI

```bash
modelcertify proof --run-id run-1 --dir ./artifacts --out proof.json
modelcertify policy --file policy.json --json
modelcertify certify --evidence evidence.json --policy policy.json \
    --issuer scai --key "$MODELCERTIFY_KEY" --out certificate.json
modelcertify verify --certificate certificate.json --key "$MODELCERTIFY_KEY"
```

Every command accepts `--json`.

Exit codes: `0` success, `1` the run was not certified (or the certificate did
not verify), `2` a usage or input error.

The signing key is hex. Pass it with `--key` or set `MODELCERTIFY_KEY`.

## Evidence and policies

An **evidence** document is what a policy is allowed to look at:

```json
{
  "proof": {"run_id": "run-1", "artifacts": [{"name": "model.safetensors", "digest": "..."}]},
  "metrics": {"eval_loss": 0.21, "eval_accuracy": 0.94},
  "provenance": {"dataset_id": "ds-1", "licence": "cc-by-4.0"}
}
```

A **policy** is a set of gates over that evidence:

```json
{
  "policy_id": "baseline",
  "name": "Baseline release policy",
  "required_artifacts": ["config.json", "model.safetensors"],
  "gates": [
    {"name": "loss", "kind": "metric", "field": "eval_loss", "op": "<=", "threshold": 0.5},
    {"name": "dataset", "kind": "provenance", "field": "dataset_id", "op": "!=", "threshold": null}
  ]
}
```

Gate kinds:

| kind | what it checks |
|---|---|
| `metric` | a measured metric against a threshold |
| `artifact` | that a named artefact is in the proof |
| `provenance` | a field of the run's provenance record |

Operators: `>=`, `<=`, `>`, `<`, `==`, `!=`, `in`.

## What it does not do

* It does not prove the *code* was correct, only that the artefacts are the
  ones that were certified.
* The signature is an HMAC, not a public-key signature: the verifier needs
  the issuer's key. That is deliberate for v1 (no key ceremony before there
  is a customer), and the interface is narrow enough to swap in Ed25519 later.
* A failed policy still produces a certificate. The point of a certificate is
  to record the outcome, including a refusal.

## Tests

```bash
pip install -e ".[dev]"
pytest
```

## Layout

```
src/modelcertify/
  proof.py       run proofs, artefacts, the Merkle root
  attestation.py signed statements about a proof root
  certify.py     policies, gates, evidence, certificates
  cli.py         modelcertify proof | policy | certify | verify
```

## Licence

Apache-2.0.
