Metadata-Version: 2.4
Name: ord-schema
Version: 0.9.3
Summary: Schema for the Open Reaction Database
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/Open-Reaction-Database/ord-schema
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Operating System :: OS Independent
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: AUTHORS
Requires-Dist: joblib>=1.0.0
Requires-Dist: numpy>=1.26.4
Requires-Dist: openpyxl>=3.0.5
Requires-Dist: pandas>=1.0.4
Requires-Dist: protobuf<6,>=4.22.3
Requires-Dist: pyarrow>=14
Requires-Dist: python-dateutil>=1.10.0
Requires-Dist: rdkit>=2025.03.4
Requires-Dist: tqdm>=4.61.2
Provides-Extra: search
Requires-Dist: duckdb>=1.5.1; extra == "search"
Requires-Dist: inflection>=0.5.1; extra == "search"
Requires-Dist: pydantic>=2; extra == "search"
Provides-Extra: nl
Requires-Dist: anthropic>=0.120; extra == "nl"
Requires-Dist: ord-schema[search]; extra == "nl"
Provides-Extra: docs
Requires-Dist: Pygments>=2.13.0; extra == "docs"
Requires-Dist: Sphinx>=5.3.0; extra == "docs"
Requires-Dist: ipython>=8.4.0; extra == "docs"
Requires-Dist: sphinx-rtd-theme>=1.1.1; extra == "docs"
Requires-Dist: sphinx-tabs>=3.4.0; extra == "docs"
Provides-Extra: examples
Requires-Dist: ord-schema[huggingface]; extra == "examples"
Requires-Dist: matplotlib>=3.3.4; extra == "examples"
Requires-Dist: scikit-learn>=0.24.1; extra == "examples"
Requires-Dist: tensorflow>=2.4.1; python_version < "3.14" and extra == "examples"
Provides-Extra: huggingface
Requires-Dist: huggingface-hub>=0.23; extra == "huggingface"
Provides-Extra: orm
Requires-Dist: inflection>=0.5.1; extra == "orm"
Requires-Dist: psycopg[binary,pool]>=3; extra == "orm"
Requires-Dist: sqlalchemy>=2; extra == "orm"
Requires-Dist: uuid6>=2023; extra == "orm"
Provides-Extra: tests
Requires-Dist: coverage>=5.2.1; extra == "tests"
Requires-Dist: nbclient>=0.7; extra == "tests"
Requires-Dist: nbval>=0.11; extra == "tests"
Requires-Dist: ord-schema[huggingface,nl,orm,search]; extra == "tests"
Requires-Dist: pytest>=7.1.1; extra == "tests"
Requires-Dist: pytest-cov>=3.0.0; extra == "tests"
Requires-Dist: pytest-timeout>=2.4.0; extra == "tests"
Requires-Dist: pytest-xdist>=3.0.2; extra == "tests"
Requires-Dist: ruff>=0.9.9; extra == "tests"
Requires-Dist: testing-postgresql>=1.3.0; extra == "tests"
Requires-Dist: ty>=0.0.28; extra == "tests"
Dynamic: license-file

# Open Reaction Database: Schema (ord-schema)

[![DOI:10.1007/978-3-319-76207-4_15](https://zenodo.org/badge/DOI/10.1021/jacs.1c09820.svg)](https://doi.org/10.1021/jacs.1c09820)
[![PyPI version](https://badge.fury.io/py/ord-schema.svg)](https://badge.fury.io/py/ord-schema)

This repository contains the schema for the Open Reaction Database initiative; please see the documentation at <https://docs.open-reaction-database.org>.

This repository does not contain the database itself; that is stored in [ord-data](https://github.com/open-reaction-database/ord-data). Rather, `ord-schema` is designed to store the database schema and tools for creating, validating, and submitting data to the database.

## Installation

```shell
$ pip install ord-schema
```

This installs the core schema and helpers (building, parsing, validation, and message/Parquet I/O). Heavier, single-purpose features live behind optional extras so the default install stays lightweight:

| Extra | Enables | Install |
| ------- | --------- | --------- |
| `search` | `ord_schema.search`: query derived Parquet artifacts with a validated query grammar (DuckDB) | `pip install "ord-schema[search]"` |
| `huggingface` | `ord_schema.huggingface`: download datasets from the Hugging Face [ord-data](https://huggingface.co/datasets/open-reaction-database/ord-data) mirror, and derived search artifacts from [ord-artifacts](https://huggingface.co/datasets/open-reaction-database/ord-artifacts) | `pip install "ord-schema[huggingface]"` |
| `orm` | `ord_schema.orm`: map the schema into a relational (SQLAlchemy + PostgreSQL) database | `pip install "ord-schema[orm]"` |
| `examples` | running the notebooks under `examples/` (see below) | `pip install "ord-schema[examples]"` |

Extras combine, e.g. `pip install "ord-schema[orm,huggingface]"`. Importing a feature without its extra installed raises a normal `ImportError`.

## Quick start

### Build a reaction

A `Reaction` is a protocol buffer message. Build it field by field, using `message_helpers.build_compound` for the parts with a lot of boilerplate:

```python
from ord_schema import message_helpers
from ord_schema.proto import reaction_pb2

reaction = reaction_pb2.Reaction()
reaction.identifiers.add(type="REACTION_SMILES", value="CC(=O)O.OCC>>CC(=O)OCC.O")

reaction.inputs["acid"].components.add().CopyFrom(
    message_helpers.build_compound(
        smiles="CC(=O)O",
        name="acetic acid",
        amount="10 mmol",
        role="reactant",
        is_limiting=True,
    )
)
reaction.inputs["alcohol"].components.add().CopyFrom(
    message_helpers.build_compound(
        smiles="OCC", name="ethanol", amount="12 mmol", role="reactant"
    )
)

outcome = reaction.outcomes.add()
outcome.reaction_time.CopyFrom(reaction_pb2.Time(value=3, units="HOUR"))
product = outcome.products.add()
product.identifiers.add(type="SMILES", value="CC(=O)OCC")
product.measurements.add(type="YIELD", percentage=reaction_pb2.Percentage(value=87))

reaction.provenance.record_created.time.value = "2026-01-15"
reaction.provenance.record_created.person.name = "Marie Curie"
reaction.provenance.record_created.person.email = "curie@example.edu"
```

### Validate

Validation reports errors and warnings separately. Pass `raise_on_error=False` to inspect them instead of raising:

```python
from ord_schema import validations

output = validations.validate_message(reaction, raise_on_error=False)
print(output.errors)    # [] -- the reaction above is valid
print(output.warnings)  # advisory only; submissions are not blocked on these
```

RDKit writes parse diagnostics straight to stderr. `ord_schema.logging.silence_rdkit_logs()` quiets them.

### Assemble and write a dataset

`updates.update_dataset` assigns the canonical `ord_dataset-*` and `ord-*` IDs and rewrites cross-references between reactions:

```python
from ord_schema import updates
from ord_schema.proto import dataset_pb2

dataset = dataset_pb2.Dataset(
    name="Esterifications", description="Fischer esterification screen"
)
dataset.reactions.append(reaction)
updates.update_dataset(dataset)
```

`datasets.save_dataset` dispatches on the filename suffix:

```python
from ord_schema import datasets

datasets.save_dataset(dataset, "esterifications.parquet")
```

| Suffix | Format |
| --- | --- |
| `.parquet` | Parquet, one row per reaction; streamable and the storage format for ord-data |
| `.pb` / `.binpb` | binary protocol buffer |
| `.pbtxt` / `.txtpb` | text protocol buffer |
| `.json` | protobuf JSON |

Any of these may be gzipped (`.pb.gz`). For a dataset too large to hold in memory, write it a reaction at a time instead — `DatasetWriter` keeps peak memory to one row group and publishes atomically, so an interrupted write leaves no file behind:

```python
from ord_schema import parquet

with parquet.DatasetWriter(
    "esterifications.parquet",
    name="Esterifications",
    description="Fischer esterification screen",
) as writer:
    for reaction in produce_reactions():
        writer.write(reaction)
```

### Read a dataset

`datasets.load_dataset` is the entry point for every format. A Parquet dataset reads back as a `DatasetView`, which takes its scalars and row count from the file footer and reads reactions on demand, so opening one is cheap regardless of dataset size:

```python
from ord_schema import datasets

view = datasets.load_dataset("esterifications.parquet")

view.name             # "Esterifications"
view.dataset_id       # "ord_dataset-..."
len(view.reactions)   # row count, from the footer -- no reactions deserialized
view.reactions[0]     # reads only the row group holding index 0
```

`reactions` behaves like a list — iteration, `len`, truthiness, indexing, and slicing — while reading from the file on demand:

```python
for reaction in view.reactions:  # streams; peak memory is one row group
    print(message_helpers.get_reaction_smiles(reaction))
```

Look a reaction up by ID, or stream IDs without deserializing anything:

```python
view.get_reaction("ord-1f6c...")   # builds an ID index on first call, then O(1)
list(view.iter_reaction_ids())     # reads one column; cheap on huge files
```

Use `iter_reactions()` when you want the ID alongside the message, and `row_group=` to fan out — row groups are the unit of parallelism:

```python
for reaction_id, reaction in view.iter_reactions():
    ...

for i in range(view.num_row_groups):
    executor.submit(process_row_group, "esterifications.parquet", i)
```

When you genuinely need a `Dataset` message — to serialize, convert to JSON, or mutate — materialize one. This deserializes everything, so peak memory scales with the whole dataset:

```python
dataset = view.to_proto()
```

The other formats have no streaming form, so they read back as a `Dataset` message directly:

```python
dataset = datasets.load_dataset("dataset.pb.gz")  # a Dataset, not a view
```

Code that only reads does not need to care which it got — iteration, `len`, indexing, and slicing work on both. Code that needs the protobuf surface and cannot know the format should ask for a message up front, which materializes a Parquet dataset in full:

```python
dataset = datasets.load_dataset(path, as_dataset=True)  # always a Dataset
```

### Fetch a published dataset

With the `huggingface` extra, pull a dataset from the mirror instead of constructing one. `fetch_dataset` downloads the file and returns its path without parsing it; it prefers Parquet and falls back to `.pb.gz` for datasets not yet converted, which is exactly the dispatch `load_dataset` already handles:

```python
from ord_schema import datasets, huggingface

path = huggingface.fetch_dataset("ord_dataset-...")
dataset = datasets.load_dataset(path)  # a DatasetView for .parquet, else a Dataset
```

## Notebook examples

The `examples/` directory contains worked examples of dataset creation and use, drawn from published papers. To run locally:

```shell
$ pip install "ord-schema[examples]"
```

Click here to run the examples with Binder:
[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/open-reaction-database/ord-schema/HEAD?labpath=examples)

## Package layout

[`ord_schema/README.md`](ord_schema/README.md) maps the package: which module builds a message, which stores a dataset, and which of the two query paths answers which kind of question. The subsystems document themselves — [`artifacts/`](ord_schema/artifacts/README.md) for the derived Parquet and its staleness stamps, [`search/`](ord_schema/search/README.md) for the query grammar and its compiler, [`orm/`](ord_schema/orm/README.md) for the relational mapping.

## Development

Editable install, with [uv](https://docs.astral.sh/uv/) or with pip (`pip install -e ".[tests]"`):

```shell
$ git clone https://github.com/open-reaction-database/ord-schema.git
$ cd ord-schema
$ uv sync --extra tests
```

The `tests` extra pulls in the feature extras (`huggingface`, `orm`) it needs to exercise their code paths, so this is enough to run the full suite; add `--extra examples` for the notebooks. [CONTRIBUTING.md](CONTRIBUTING.md) covers the rest: pre-commit hooks, running the same checks by hand, rebuilding the proto wrappers, and how a release is cut.

## Conventions

Some things the schema can hold in more than one shape have an agreed shape to prefer, so
that data from different depositors answers the same query. These are guidance rather than
validation rules; see [CONVENTIONS.md](CONVENTIONS.md).
