Metadata-Version: 2.5
Name: scientific-data-contracts
Version: 0.2.0
Summary: Validate computational biology datasets against scientific data contracts.
Project-URL: Homepage, https://github.com/ronfinn/scientific-data-contracts
Project-URL: Repository, https://github.com/ronfinn/scientific-data-contracts
Project-URL: Issues, https://github.com/ronfinn/scientific-data-contracts/issues
Project-URL: Changelog, https://github.com/ronfinn/scientific-data-contracts/releases
Author: Ron Finn
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: anndata,bioinformatics,data-quality,data-validation,single-cell,spatial-omics,spatialdata
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Classifier: Typing :: Typed
Requires-Python: >=3.12
Requires-Dist: anndata>=0.13.3.post0
Requires-Dist: pandas>=2.0
Requires-Dist: pyyaml>=6.0.3
Requires-Dist: spatialdata<0.9,>=0.8
Description-Content-Type: text/markdown

# scientific-data-contracts

`sci-contract` validates a computational biology dataset against a
human-readable YAML contract and reports every failure it finds in one run — as
a report for a person, or as JSON for a pipeline. It reads AnnData `.h5ad`
files and SpatialData `.zarr` stores.

A contract states what a dataset must contain to be usable for a given
analysis: which fields are present, what values they may take, and which
scientific assumptions the data is expected to satisfy.

## Why

A scientific dataset can be perfectly readable and still be unusable. The file
opens, the array has the right shape, and the analysis runs — on cells whose
condition label is spelled three different ways, against a segmentation from a
different run, with no record of what produced any of it.

Those are scientific failures, not structural ones. Nothing in the file format
can catch them, because nothing in the file format knows what the data is
supposed to mean. A contract is where that gets written down: by the people who
define the expectation, in a file the tooling that consumes the data can check.

## Capabilities

| Capability | AnnData | SpatialData |
| --- | --- | --- |
| Required elements/fields | `obs` and `var` columns | element names, by collection |
| Non-null metadata | yes | — |
| Allowed values | yes | — |
| Unique identifiers | `obs_names`/`var_names`, always | — |
| Unique metadata columns | `unique` | — |
| Numeric ranges | `numeric_range` | — |
| Provenance | `uns["provenance"]` fields | — |
| Spatial table–region relationships | — | required regions, usable linkage columns |
| Instance integrity | — | opt-in: every table row's instance exists in its region |
| Coordinate-system relationships | — | required systems, and which elements map into them |
| Text report | yes | yes |
| JSON report | yes | yes |
| Nextflow example | yes | yes |

A contract's `dataset.type` decides how the dataset is read. Nothing sniffs the
path. Neither read is eager — the AnnData expression matrix stays on disk and
SpatialData elements stay as unevaluated Dask graphs — with one exception, the
opt-in instance check, which reads a segmentation raster because that is the
only way to answer it.

## Installation

Requires [uv](https://docs.astral.sh/uv/) and Python 3.12 or newer.

The package is not yet on PyPI. Install it from a clone, which puts
`sci-contract` on your `PATH`:

```
git clone https://github.com/ronfinn/scientific-data-contracts
uv tool install ./scientific-data-contracts
sci-contract --version
```

Or run it without installing anything, from a built wheel:

```
uv build
uvx --from dist/scientific_data_contracts-0.2.0-py3-none-any.whl sci-contract --version
```

To work on the tool rather than use it, sync the clone instead and prefix
commands with `uv run`:

```
uv sync --dev
uv run sci-contract --version
```

Once the package is published, `pip install scientific-data-contracts` and
`uvx scientific-data-contracts` will be the shorter routes. Neither works yet,
and nothing above depends on them.

## Quick start

Both example datasets are committed, so both commands work in a fresh clone.

```
uv run sci-contract validate examples/example.h5ad --contract examples/anndata-basic.yaml
uv run sci-contract validate examples/spatial-example.zarr --contract examples/spatialdata-basic.yaml
```

An abbreviated AnnData contract — `obs` and `var` columns, the values they may
hold, and the provenance the file must carry:

```yaml
version: 1

dataset:
  type: anndata

obs:
  required: [sample_id, condition, cell_barcode, pct_counts_mt]
  non_null: [sample_id, condition]
  unique:   [cell_barcode]
  allowed_values:
    condition: [control, treated]
  numeric_range:
    pct_counts_mt:
      min: 0
      max: 100

var:
  required: [gene_symbol]

provenance:
  required: [pipeline, pipeline_version, git_commit, container_digest]
```

And a SpatialData one — the elements, the space they share, and the table's
link to the segmentation it annotates:

```yaml
version: 1

dataset:
  type: spatialdata

spatial:
  required_elements:
    images: [morphology]
    labels: [cell_segmentation]
    tables: [table]

  required_coordinate_systems: [global]

  # Both elements must be registered into that system, not merely exist
  # alongside it.
  element_coordinate_systems:
    images:
      morphology: [global]
    labels:
      cell_segmentation: [global]

  # A table is not spatial by itself; it reaches coordinates through the region
  # it annotates.
  table_annotations:
    table:
      regions: [cell_segmentation]
      instances_must_exist: true
```

A rule may only name something the contract already requires — `unique` a
required column, a table annotation a required table, an element registration a
required element and a required coordinate system — because a rule about
something the contract does not guarantee could never be evaluated.

Every field is specified in
[`docs/contract-format.md`](docs/contract-format.md).

## Contract examples

Realistic contracts to read and adapt, in
[`examples/contracts/`](examples/contracts/):

- [`single-cell-anndata.yaml`](examples/contracts/single-cell-anndata.yaml) — a
  processed scRNA-seq dataset: experimental design, cell-type annotation,
  unique barcodes, quality measures within possible bounds, and provenance.
- [`xenium-like-spatialdata.yaml`](examples/contracts/xenium-like-spatialdata.yaml)
  — an imaging-based spatial dataset: morphology image, cell and nucleus
  segmentation, transcript points, and a per-cell table resolved against the
  cell mask.
- [`visium-hd-like-spatialdata.yaml`](examples/contracts/visium-hd-like-spatialdata.yaml)
  — a sequencing-based spatial dataset: tissue image, capture-bin geometry, and
  a per-bin table resolved against the bins.

The two spatial contracts are named after the assays that shape them, and that
is where the resemblance stops. This tool validates a SpatialData store that
already exists; it does not ingest a vendor bundle, and the element names in
those files are the names a conversion step chose rather than standard ones.
[`examples/contracts/README.md`](examples/contracts/README.md) says what to
change when adapting them.

## Validation reports

A dataset that satisfies its contract, exit code 0:

```
$ uv run sci-contract validate examples/example.h5ad --contract examples/anndata-basic.yaml
scientific-data-contracts

Dataset   examples/example.h5ad
Contract  examples/anndata-basic.yaml

Dataset
  observations              6
  variables                 4

Contract
  required obs columns      3 / 3
  required var columns      1 / 1

Provenance
  required fields           4 / 4

Validation
  PASS

0 errors
```

One that does not, exit code 1. Every applicable check runs, so one invocation
reports every failure rather than stopping at the first:

```
$ uv run sci-contract validate cohort.h5ad --contract cohort.yaml
...
Validation
  FAIL

Issues
  obs.cell_barcode          duplicate_obs_values
    contains duplicate values: AAACCTGAGAAACCAT
  obs.condition             invalid_obs_value
    contains values not allowed by the contract: placebo
  obs.pct_counts_mt         out_of_range_obs_value
    contains values outside the required range (0 to 100): -3, 130.25

3 errors
```

`--format json` renders the same result as a document with a stable
`schema_version` and stable issue codes:

```json
{
  "schema_version": 1,
  "valid": false,
  "dataset": "cohort.h5ad",
  "contract": "cohort.yaml",
  "dataset_type": "anndata",
  "observations": 6,
  "variables": 2,
  "required_obs_columns": { "required": 4, "present": 4 },
  "required_var_columns": { "required": 1, "present": 1 },
  "dataset_provenance": null,
  "spatial": null,
  "issues": [
    {
      "code": "duplicate_obs_values",
      "location": "obs.cell_barcode",
      "message": "contains duplicate values: AAACCTGAGAAACCAT"
    }
  ]
}
```

The two column-count objects are shown on one line each for brevity; the tool
prints them expanded, and the `issues` array holds all three. Both renderings
come from one result, so they cannot disagree. Every field and every code is
specified in [`docs/report-format.md`](docs/report-format.md).

## Pipeline integration

The JSON report is the interface. A workflow engine or CI job runs the
validation, reads the report, and decides whether the dataset may be used.

```mermaid
flowchart TD
    D["dataset + contract"] --> S[sci-contract]
    S --> T[text report]
    S --> J[JSON report]
    J --> W["Nextflow / CI"]
```

[`examples/nextflow/`](examples/nextflow/) is a working Nextflow workflow that
does exactly this: it validates a dataset, publishes the report either way, and
fails the workflow when the dataset does not satisfy its contract, so the
analysis step never runs on data that was rejected. It is two processes and a
config file — no plugin, no container, no cloud profile.

```
nextflow run examples/nextflow
nextflow run examples/nextflow \
  --dataset  examples/spatial-example.zarr \
  --contract examples/spatialdata-basic.yaml \
  --report   spatial-report.json
```

With the package installed rather than run from a clone, add
`--sci_contract sci-contract`.

## Documentation

- [`docs/contract-format.md`](docs/contract-format.md) — every contract field,
  the errors an invalid contract produces, and the current limitations.
- [`docs/report-format.md`](docs/report-format.md) — the report's JSON schema
  version, every field, every issue code, and the exit codes.
- [`docs/architecture.md`](docs/architecture.md) — how the implementation is put
  together, how it is packaged, and what it deliberately avoids.
- [`examples/contracts/`](examples/contracts/) — realistic contracts, and what
  they do and do not promise.
- [`examples/nextflow/`](examples/nextflow/) — the pipeline quality-gate
  example.
- [`CONTRIBUTING.md`](CONTRIBUTING.md) — the development loop.

## Current release

**v0.2.0**. Distributable as a wheel and an sdist, with realistic single-cell
and spatial-omics contract examples, uniqueness and numeric-range constraints,
and a parameterised Nextflow gate. Contracts written for v0.1.0 remain valid
and behave identically.

## Roadmap

- Publish to PyPI, so installation is one command with no clone.
- Contracts for a SpatialData table's own `obs` and `var`.
- Richer AnnData constraints where a clear semantic model exists — the reasons
  dtypes are still absent are in
  [`docs/contract-format.md`](docs/contract-format.md).
- Dataset checksums and stronger provenance.
- Deeper workflow integration driven by the JSON report.

What is deliberately not implemented is listed under "Current limitations" in
[`docs/contract-format.md`](docs/contract-format.md) and
[`docs/report-format.md`](docs/report-format.md).
