Metadata-Version: 2.4
Name: h5rdmtoolbox
Version: 2.8.3
Summary: Supporting a FAIR Research Data lifecycle using Python and HDF5.
Author-email: Matthias Probst <matth.probst@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://h5rdmtoolbox.readthedocs.io/en/latest/
Project-URL: Documentation, https://h5rdmtoolbox.readthedocs.io/en/latest/
Project-URL: Repository, https://github.com/matthiasprobst/h5rdmtoolbox
Project-URL: Bug Tracker, https://github.com/matthiasprobst/h5rdmtoolbox/issues
Keywords: hdf5,research data,fair,metadata
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Operating System :: OS Independent
Classifier: Development Status :: 4 - Beta
Classifier: Topic :: Scientific/Engineering
Requires-Python: <3.14,>=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: appdirs~=1.4
Requires-Dist: importlib_resources<7.0.0,>=6.5.2
Requires-Dist: numpy<3.0.0,>=1.20
Requires-Dist: h5py~=3.8
Requires-Dist: matplotlib>=3.5.2
Requires-Dist: IPython>=7.34.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: xarray>=2022.3.0
Requires-Dist: pint>=0.24.4
Requires-Dist: pint_xarray<=0.6.0,>=0.2.1
Requires-Dist: regex>=2020.7.9
Requires-Dist: packaging>=24.1
Requires-Dist: ontolutils<0.28.0,>=0.27.5
Requires-Dist: python-forge==18.6.0
Requires-Dist: requests>=2.32.4
Requires-Dist: pydantic~=2.8
Requires-Dist: rdflib~=7.1
Requires-Dist: typer>=0.12
Requires-Dist: cftime>=1.6.4
Requires-Dist: pyshacl<1.0.0,>=0.30.1
Provides-Extra: database
Requires-Dist: pymongo<=4.10.1,>=4.6.3; extra == "database"
Provides-Extra: layout-validation
Requires-Dist: tabulate<=0.9.0,>=0.8.10; extra == "layout-validation"
Provides-Extra: csv
Requires-Dist: pandas>=1.4.3; extra == "csv"
Provides-Extra: snt
Requires-Dist: xmltodict<=0.13.0; extra == "snt"
Requires-Dist: tabulate<=0.9.0,>=0.8.10; extra == "snt"
Requires-Dist: python-gitlab; extra == "snt"
Requires-Dist: pypandoc>=1.11; extra == "snt"
Provides-Extra: catalog
Requires-Dist: SPARQLWrapper>=2.0.0; extra == "catalog"
Provides-Extra: gui
Requires-Dist: PyQt5==5.15.10; extra == "gui"
Provides-Extra: test-core
Requires-Dist: build>=1.2; extra == "test-core"
Requires-Dist: setuptools>=78.1.1; extra == "test-core"
Requires-Dist: wheel; extra == "test-core"
Requires-Dist: toml>0.10.0; extra == "test-core"
Requires-Dist: pytest>=8.3.3; extra == "test-core"
Requires-Dist: pytest-cov>=5.0.0; extra == "test-core"
Requires-Dist: pylint; extra == "test-core"
Requires-Dist: ssnolib<3.0.0,>=2.2.0.2; extra == "test-core"
Requires-Dist: mongomock==4.3.0; extra == "test-core"
Requires-Dist: scipy>=1.10.1; extra == "test-core"
Requires-Dist: scikit-image>=0.21.0; extra == "test-core"
Requires-Dist: scikit-learn; extra == "test-core"
Provides-Extra: test
Requires-Dist: h5rdmtoolbox[catalog,csv,database,snt,test-core]; extra == "test"
Provides-Extra: test-server
Requires-Dist: h5rdmtoolbox[server,test]; extra == "test-server"
Requires-Dist: httpx; extra == "test-server"
Provides-Extra: docs
Requires-Dist: Sphinx==8.0.0; extra == "docs"
Requires-Dist: sphinx_book_theme==1.1.3; extra == "docs"
Requires-Dist: sphinx_copybutton==0.5.2; extra == "docs"
Requires-Dist: sphinx_design==0.6.1; extra == "docs"
Requires-Dist: myst_nb==1.2.0; extra == "docs"
Requires-Dist: sphinxcontrib_bibtex==2.6.3; extra == "docs"
Provides-Extra: complete
Requires-Dist: h5rdmtoolbox[gui,test]; extra == "complete"
Provides-Extra: complete-with-docs
Requires-Dist: h5rdmtoolbox[complete,docs]; extra == "complete-with-docs"
Provides-Extra: server
Requires-Dist: fastapi>=0.95; extra == "server"
Requires-Dist: uvicorn[standard]>=0.22; extra == "server"
Requires-Dist: jinja2>=3.1.6; extra == "server"
Dynamic: license-file

# HDF5 Research Data Management Toolbox

![Tests](https://github.com/matthiasprobst/h5RDMtoolbox/actions/workflows/tests.yml/badge.svg)
[![codecov](https://codecov.io/gh/matthiasprobst/h5RDMtoolbox/graph/badge.svg?token=IVG4AQEW47)](https://codecov.io/gh/matthiasprobst/h5RDMtoolbox)
[![Documentation Status](https://readthedocs.org/projects/h5rdmtoolbox/badge/?version=latest)](https://h5rdmtoolbox.readthedocs.io/en/latest/?badge=latest)
[![PyPI version](https://badge.fury.io/py/h5RDMtoolbox.svg)](https://badge.fury.io/py/h5RDMtoolbox)
![pyvers](https://img.shields.io/badge/python-3.9%20%7C%203.10%20%7C%203.11%20%7C%203.12%20%7C%203.13-blue)
[![Paper](https://img.shields.io/badge/Paper-View-green?style=flat&logo=adobeacrobatreader)](https://preprints.inggrid.org/repository/view/23/)

A Python package that helps researchers achieve sustainable 
[**FAIR (Findable, Accessible, Interoperable, Reusable)**](https://www.nature.com/articles/sdata201618) data management with HDF5 files. The toolbox provides a comprehensive
approach to the research data lifecycle through five main stages: **planning**, **collecting**, **analyzing**,
**sharing**, and **reusing** data.

> **Note**: This project is actively under development.




## ✨ Quick Start


One of the core features of the toolbox and success factors for achieving FAIR data, is using semantic metadata.

Let's see how we can use RDF (semantic metadata) together with HDF5 files using the toolbox:

```bash
# Install the package
pip install h5RDMtoolbox
```

```python
import h5rdmtoolbox as h5tbx
import rdflib
import numpy as np

# Create a new HDF5 file with semantic metadata
M4I = rdflib.Namespace("http://w3id.org/nfdi4ing/metadata4ing#")

# Create a new HDF5 file with FAIR metadata
with h5tbx.File("example.h5", "w") as h5:
    ds = h5.create_dataset("temperature", data=np.array([20, 21, 19, 22]))
    ds.attrs["units"] = h5tbx.Attribute(
        value="degree_Celsius",
        rdf_predicate=M4I.hasUnit,
        rdf_object="http://qudt.org/vocab/unit/DEG_C",
    )
    ds.attrs["description", "https://schema.org/description"] = "Room temperature measurements"

    ds_mean = h5.create_dataset("mean_temperature", data=np.mean(h5["temperature"][()]))
    ds_mean.attrs["units", M4I.hasUnit] = "degree_Celsius"
    ds_mean.rdf["units"].object = "http://qudt.org/vocab/unit/DEG_C"
    ds_mean.attrs["description", "https://schema.org/description"] = "Mean room temperature measurements"

    ds_mean.rdf.type = M4I.NumericalVariable
    ds_mean.rdf.data_predicate = M4I.hasNumericalValue

    ttl_ctx = h5.serialize("ttl", structural=False, contextual=True, file_uri="https://example.org#example.h5/")
    ttl_struc = h5.serialize("ttl", structural=True, contextual=False, file_uri="https://example.org#example.h5/")
    ttl_full = h5.serialize("ttl", file_uri="https://example.org#example.h5/")
```

The above saves three different serializations of the HDF5 content as RDF in Turtle (ttl) format.
The first one (`ttl_ctx`) is the **contextual** RDF serialization, which only includes the RDF triples that the user
has enriched the HDF5 file with. The second one (`ttl_struc`) is the **structural** RDF serialization, which includes 
all RDF triples that can be derived from the HDF5 file content. the third one (`ttl_full`) is the combination of the 
first two, which includes both the user enriched RDF triples and the RDF triples that can be derived from the HDF5 file 
content.

Below we show the first two serializations.

`ttl_ctx`:
```ttl
@prefix m4i: <http://w3id.org/nfdi4ing/metadata4ing#> .
@prefix schema: <https://schema.org/> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .

<https://example.org#example.h5/example.h5/mean_temperature> a m4i:NumericalVariable ;
    m4i:hasNumericalValue "20.5"^^xsd:float ;
    m4i:hasUnit <http://qudt.org/vocab/unit/DEG_C> ;
    schema:description "Mean room temperature measurements" .

<https://example.org#example.h5/example.h5/temperature> m4i:hasUnit <http://qudt.org/vocab/unit/DEG_C> ;
    schema:description "Room temperature measurements" .
```

`ttl_struc`:
```ttl
@prefix hdf: <http://purl.allotrope.org/ontologies/hdf5/1.8#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .

<https://example.org#example.h5/example.h5> a hdf:File ;
    hdf:rootGroup <https://example.org#example.h5/example.h5/> .

hdf:H5T_IEEE_F64LE a hdf:Datatype .

hdf:H5T_INTEL_I64 a hdf:Datatype .

<https://example.org#example.h5/example.h5/> a hdf:Group ;
    hdf:member <https://example.org#example.h5/example.h5/mean_temperature>,
        <https://example.org#example.h5/example.h5/temperature> ;
    hdf:name "/" .

<https://example.org#example.h5/example.h5/mean_temperature> a hdf:Dataset ;
    hdf:attribute <https://example.org#example.h5/example.h5/mean_temperature@description>,
        <https://example.org#example.h5/example.h5/mean_temperature@units> ;
    hdf:dataspace <https://example.org#example.h5/example.h5/mean_temperature__dataspace> ;
    hdf:datatype hdf:H5T_FLOAT,
        hdf:H5T_IEEE_F64LE ;
    hdf:layout hdf:H5D_CONTIGUOUS ;
    hdf:maximumSize -1 ;
    hdf:name "/mean_temperature" ;
    hdf:rank 0 ;
    hdf:size 1 ;
    hdf:value 2.05e+01 .

<https://example.org#example.h5/example.h5/mean_temperature@description> a hdf:StringAttribute ;
    hdf:data "Mean room temperature measurements" ;
    hdf:name "description" .

<https://example.org#example.h5/example.h5/mean_temperature@units> a hdf:StringAttribute ;
    hdf:data "degree_Celsius" ;
    hdf:name "units" .

<https://example.org#example.h5/example.h5/mean_temperature__dataspace> a hdf:ScalarDataspace .

<https://example.org#example.h5/example.h5/temperature> a hdf:Dataset ;
    hdf:attribute <https://example.org#example.h5/example.h5/temperature@description>,
        <https://example.org#example.h5/example.h5/temperature@units> ;
    hdf:dataspace <https://example.org#example.h5/example.h5/temperature__dataspace> ;
    hdf:datatype hdf:H5T_INTEGER,
        hdf:H5T_INTEL_I64 ;
    hdf:layout hdf:H5D_CONTIGUOUS ;
    hdf:maximumSize 4 ;
    hdf:name "/temperature" ;
    hdf:rank 1 ;
    hdf:size 4 .

<https://example.org#example.h5/example.h5/temperature@description> a hdf:StringAttribute ;
    hdf:data "Room temperature measurements" ;
    hdf:name "description" .

<https://example.org#example.h5/example.h5/temperature@units> a hdf:StringAttribute ;
    hdf:data "degree_Celsius" ;
    hdf:name "units" .

<https://example.org#example.h5/example.h5/temperature__dataspace> a hdf:SimpleDataspace ;
    hdf:dimension <https://example.org#example.h5/example.h5/temperature__dataspace_dimension_0> .

<https://example.org#example.h5/example.h5/temperature__dataspace_dimension_0> a hdf:DataspaceDimension ;
    hdf:dimensionIndex 0 ;
    hdf:size 4 .
```

## Command Line Interface

The `h5tbx` command provides quick access to linked-data serialization and local inspection from the shell.

Use `h5tbx ld dump example.h5` to print Turtle RDF, or write a specific format with:

```bash
h5tbx ld dump example.h5 -o example.jsonld
```

The dump command accepts `--format`, `--file-uri`, `--prefix`, `--structural=false`, and `--contextual=false`, so you can choose RDF output, assign stable RDF subject URIs, bind a compact prefix, and include only structural or user-provided contextual metadata when needed.

To inspect files in a browser, run:

```bash
h5tbx serve example.h5
h5tbx serve data/ --h5ext=.h5 --h5ext=.hdf5
```
This allows you to serve a single HDF5 file or a directory of HDF5 files, and view their metadata and structure in a web interface.

You may perform SPARQL queries or view the file as a graph, as shown below:

<p align="center">
  <img src="docs/_static/screenshot-served-hdf5-as-graph.png" alt="Serve UI screenshot" width="640">
</p>


The same graph metrics are available from Python:

```python
import h5rdmtoolbox as h5tbx

metrics = h5tbx.compute_metrics("example.h5")

with h5tbx.File("example.h5") as h5:
    metrics = h5.metrics()
```

## 🚀 Key Features

- **🔗 HDF5 + [Xarray](https://docs.xarray.dev/en/stable/) Integration**: Seamless access to metadata during data analysis with native xarray support
- **🏷️ Persistent Identifiers**: Assign globally unique identifiers using RDF triples for FAIR compliance  
- **📋 Standardized Conventions**: Define and enforce community-specific metadata standards
- **☁️ Repository Integration**: Direct upload to Zenodo and other research repositories
- **🗄️ Database Support**: Use HDF5 files with MongoDB or native HDF5 databases
- **🔒 Semantic Enrichment**: Add RDF-based semantic meaning to your data
- **🌐 Catalog Integration**: Search and discover datasets through SPARQL-based catalogs


Find an example code in  [![In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/matthiasprobst/h5RDMtoolbox/blob/main/docs/colab/quickstart.ipynb) or check out the 
[📚 full Documentation](https://h5rdmtoolbox.readthedocs.io/en/latest/)

## Who is the package for?

For everybody, who is...

- ... looking for a management approach for his or her data.
- ... community has not yet established a stable convention.
- ... working with small and big data, that fits into HDF5 files.
- ... looking for an easy way to work with HDF5, especially
  through [Jupyter Notebooks](https://jupyterlab.readthedocs.io/en/stable/getting_started/installation.html).
- ... trying to integrate HDF5 with repositories and databases.
- ... wishing to enrich data semantically with the RDF standard.
- ... looking for a way to do all the above whiles not needing to learn a new syntax.
- ... new to HDF5 and wants to learn about it, especially with respect to the FAIR principles and data management.

## Who is it not for?

For everybody, who ...

- ... is looking for a management approach which at the same time allows high-performance and/or parallel work with HDF5
- ... has already well-established conventions and managements approaches in his or her community

## Package Architecture/structure

The toolbox implements six modules, which are shown below. The numbers reference to their main usage in the stages in
the data lifecycle shown [here](https://h5rdmtoolbox.readthedocs.io/en/latest/gettingstarted/index.html). The wrapper
module implements the main interface between the user and the HDF5 file. It
extends the features of the underlying `h5py` library. Some of the features are implemented in other modules, hence the
wrapper module depends on the convention, database and linked data (ld) module.

<a href="https://h5rdmtoolbox.readthedocs.io/en/latest/"><img src="docs/_static/h5tbx_modules.svg" alt="H5TBX modules" style="width:600px;"></a>

Current implementation highlights in the modules:

- The **wrapper** module adds functionality on top of the `h5py` package. It allows to include so-called standard names,
  which are defined in conventions. And it implements interfaces, such as to the package `xarray`, which allows to carry
  metadata from HDF5 to the user. Other high-level interfaces like `.rdf` allows assigning semantic information to the
  HDF5 file.
- For the **database** module, `hdfDB` and `mongoDB` are implemented. The `hdfDB` module allows to use HDF5 files as a
  database. The `mongoDB` module allows to use mongoDB as a database by mapping the metadata of HDF5 files to the
  database.
- For the **repository** module, a Zenodo interface is implemented. Zenodo is a repository, which allows to upload and
  download data with a persistent identifier.
- For the **convention** module,
  the [standard attributes](https://h5rdmtoolbox.readthedocs.io/en/latest/userguide/convention/index.html)
  are implemented.
- The **layout** module allows to define expectations on the internal layout (object names, location, attributes,
  properties) of HDF5 files.
- The **catalog** module allows interfacing to HDF5 and RDF data published on Zenodo. Via a catalog file, a providers
  describe the data in various zenodo records they want to share. Through the catalog file, users can work with the
  data without downloading the full HDF5 files first.

## Quickstart

A quickstart notebook can be tested by clicking on the following badge:

[![Open Quickstart Notebook](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/matthiasprobst/h5RDMtoolbox/blob/main/docs/colab/quickstart.ipynb)

## Documentation

Please find a comprehensive documentation with many examples [here](https://h5rdmtoolbox.readthedocs.io/en/latest/) or
by click on the image, which shows the research data lifecycle in the center and the respective toolbox features on the
outside:

A paper is published in the journal [inggrid](https://preprints.inggrid.org/repository/view/23/).

## Installation

Use Python 3.9 through 3.13. If you are a regular user, you can install the
package via pip:

    pip install h5RDMtoolbox

### Install from source:

Developers may clone the repository and install the package from source. Clone the repository first:

    git clone --branch main https://github.com/matthiasprobst/h5RDMtoolbox.git

Then, run

    pip install h5RDMtoolbox/

Add `--user` if you do not have root access.

For development installation run

    pip install -e h5RDMtoolbox/

### Dependencies

The core functionality depends on the following packages. Some of them are for general management others are very
specific to the features of the package:

**General dependencies are ...**

- `numpy`: Scientific computing, handling of arrays
- `matplotlib`: Plotting
- `appdirs`: Managing user and application directories
- `importlib_resources`: Accessing package resources consistently
- `packaging`: Version handling
- `IPython`: Pretty display of data in notebooks
- `regex`: Working with regular expressions

**Specific to the package are ...**

- `h5py`: HDF5 file interface
- `xarray`: Working with scientific arrays in combination with attributes. Allows carrying metadata from HDF5
  to user
- `pint`: Allows working with units
- `pint_xarray`: Working with units for usage with xarray
- `python-forge`: Used to update function signatures when using
  the [standard attributes](https://h5rdmtoolbox.readthedocs.io/en/latest/userguide/convention/index.html)
- `pydantic`: Used to
  validate [standard attributes](https://h5rdmtoolbox.readthedocs.io/en/latest/userguide/convention/index.html)
- `pyyaml`: Reading and writing of yaml files, e.g. metadata definitions (conventions). Note, lower versions
  collide with python 3.11
- `requests`: Used to download files from the internet or validate URLs, e.g. metadata definitions (conventions)
- `rdflib`: Used to enable working with RDF
- `ontolutils`: Required to work with RDF and derive semantic description of HDF5 file content
- `cftime`: Handling non-standard calendars and time coordinates
- `pyshacl`: Validating RDF graphs against SHACL shapes
- `typer`: Providing the `h5tbx` command-line interface

#### Optional dependencies

To run unit tests or to enable certain features, additional dependencies must be installed.

Install optional dependencies by specifying them in square brackets after the package name, e.g.:

    pip install h5RDMtoolbox[database]

[database]

- `pymongo`: Database solution for HDF5 files

[csv]

- `pandas`: Mainly used for reading csv and pretty printing

[snt]

- `xmltodict`: Reading of xml files
- `tabulate`: Pretty printing of tables
- `python-gitlab`: Access to gitlab repositories
- `pypandoc`: Conversion of markdown files to html

Other supported extras are `layout_validation`, `catalog`, `gui`, `server`, `test`, `test-server`, `docs`, `complete`, and
`complete-with-docs`. See the [installation guide](https://h5rdmtoolbox.readthedocs.io/en/latest/gettingstarted/installation.html)
for their purpose. The authoritative dependency lists and version constraints are in [`pyproject.toml`](./pyproject.toml).

## Citing the package

If you use the package in your work, cite the software using its version-specific
[Zenodo DOI for version 2.8.3](https://doi.org/10.5281/zenodo.23854129).
This DOI is reserved for the release and will resolve once the Zenodo record is published.
A related paper is published in the
journal [inggrid](https://www.inggrid.org/article/id/4028/). Thank you!

Alternatively or additionally, you can consult the `CITATION.cff` file.

Here is the BibTeX entry:

```
@article{probst2024h5rdmtoolbox,
	author = {Matthias Probst, Balazs Pritz},
	title = {h5RDMtoolbox - A Python Toolbox for FAIR Data Management around HDF5},
	volume = {2},
	year = {2024},
	url = {https://www.inggrid.org/article/id/4028/},
	issue = {1},
	doi = {10.48694/inggrid.4028},
	month = {8},
	keywords = {Data management,HDF5,metadata,data lifecycle,Python,database},
	issn = {2941-1300},
	publisher={Universitäts- und Landesbibliothek Darmstadt},
	journal = {ing.grid}
}
```

## Attribute quantities

Quantity-valued attribute strings keep their original HDF5 text.
`attribute.to_pint()` accepts plain, parenthesized, and square-bracket unit
annotations such as `1.2 m`, `1.2 (m)`, and `1.2 [m]`, including newer Pint parsers.

## Decoded calibration data

When a convention enables scale/offset decoding, returned xarray data omits the
storage attributes `DATA_SCALE`, `DATA_OFFSET`, `IS_DATA_SCALE`, and
`IS_DATA_OFFSET`. Calibration links, role flags, and raw values remain available
on the HDF5 datasets.

## Convention contexts

`with convention:` restores the preceding convention on exit, including when
the same Convention object is nested repeatedly or the body raises an exception.
`h5tbx.use(...)` activates immediately when constructed; create a fresh `use`
manager for each context.

Convention-name lookup is context-local, but activation changes shared wrapper
method signatures and decoder selection. Convention-dependent operations should
run sequentially: different conventions are not isolated across concurrent
threads or asynchronous tasks, and an open file is not an isolated decoding
environment merely because it remembers its opening convention.

## Configuration

Use `h5tbx.set_config(...)` for persistent settings, or as a context manager for
temporary overrides. Nested contexts restore both configuration and the shared
Pint display format on exit, including when an exception is raised. Invalid
keys or values are rejected before settings change.

```python
with h5tbx.set_config(hdf_compression=None, hdf_compression_opts=None):
    # Create datasets without the configured default compression.
    ...
```

`plotting_name_order` accepts a nonempty list or tuple of `plot_name`,
`long_name`, and/or `standard_name` in the desired priority order.

## Contribution

Releases follow a **DOI-first** workflow: supply the reserved, version-specific
Zenodo DOI before bumping the version, then update package and citation metadata
together. See the [release checklist](./docs/development.md#doi-first-release-checklist)
and [agent instructions](./AGENTS.md#fair-versioning-and-release-metadata).
Release preparation and publication are separate steps.

RDF storage refactoring preserves the existing HDF5 attribute representation.
RDF validation and object preparation stay separate from shared mapping updates;
the existing wrapper continues to encode attributes for HDF5.
The compatibility checks in `tests/ld/test_rdf_storage.py` inspect raw h5py
values, string types, shapes, and reopened files. See the
[storage contract](./docs/development.md#rdf-metadata-storage-contract).

Start with the [developer overview](./docs/development.md) for module responsibilities,
initialization behavior, startup measurements, and maintenance priorities. The
[repository guidelines](./AGENTS.md) map code areas to focused tests and describe
the Git workflow. Work on a dedicated branch and keep changes in coherent commits.
For the current modernization, Git operations are authorized without repeated
confirmation, with `git merge` excluded and commits confined to the working branch.

Subagent investigations require explicit approval before agents are started.
Approved architecture, quality, feature, or difficult-problem discussions are
documented under `.work/<topic>/`, with evidence, alternatives, and a final
recommendation. See the repository guidelines for the delegation workflow.

The [architecture and quality review](./.work/architecture-quality/README.md)
records independently reviewed CI, installed-wheel, test-coverage, and convention
state findings, with reproductions and prioritized follow-up checks.

The toolbox loads the SHACL validation engine and notebook HTML display support
when those features are first used. A normal package import avoids loading them;
both remain included in the required dependencies. See the developer overview
for the startup measurement protocol and local before/after results.

Internally, wrappers and shared utilities access configuration through its
defining module; public configuration helpers remain available on `h5tbx`.
Utility logging continues to use the shared `h5rdmtoolbox` logger and handlers.

Distribution builds copy the canonical root `codemeta.json` into package data,
so `h5tbx.get_package_meta()` also works outside an editable checkout. The test
extra includes the build frontend and backend needed to verify an sdist-built
wheel without downloading build dependencies during the test:

```bash
python -m pytest tests/test_distribution.py -q
```

This check builds from a fresh source copy, verifies metadata and runtime assets,
and imports the resulting package outside the checkout. Use clean source trees
for release builds so stale local build files cannot enter a distribution.

From the repository root, install the test dependencies and run the local smoke checks:

```bash
python -m pip install -e ".[test]"
python -m pytest tests/test_version.py tests/test_config.py tests/test_cli.py -q
```

Add regression tests in `tests/` for changed behavior and run the relevant test
modules. The full suite includes downloads and external integrations; the developer
overview explains test isolation and service requirements. Use existing code style
and **NumPy-style docstrings** for public APIs.

Live Zenodo uploads are opt-in: tests marked `zenodo_upload` require both
`H5RDMTOOLBOX_RUN_ZENODO_UPLOAD_TESTS=1` and `ZENODO_SANDBOX_API_TOKEN`.
Ordinary test runs skip these remote writes, even if a token is available.
See the [testing guide](./tests/README.md) for commands and skip reasons.

CI runs on pushes, pull requests, and manual dispatch with read-only repository
permissions and without Zenodo write credentials or automatic account cleanup.
Manual Sandbox cleanup requires an explicit list of deposition IDs; it never
enumerates or deletes all drafts in an account.

Server validation has its own CI job using `.[test-server]`, which includes HTTPX
for FastAPI's TestClient. Missing optional dependencies are reported as a module
skip in ordinary test runs, but fail when `H5RDMTOOLBOX_REQUIRE_SERVER_TESTS=1`.
Broken installed dependencies always fail visibly. CI checks the JUnit report
for actual passing test cases and rejects skips or an empty run. Workflow actions
use Node 24-compatible versions; self-hosted runners need version 2.327.1 or newer.

Catalog tests generate their HDF5 files and local indexes in per-test temporary
directories, keeping checked-in sample data read-only and avoiding shared state
between test cases.

Keep this README, API documentation, affected guides, and developer instructions
up to date in the same commit as the change. Add or update notebooks when they help
explain user-facing behavior. See the [documentation guide](./docs/README.md) for
build requirements. For prose-only changes, a build without notebook execution is:

```bash
python -m sphinx -W -b html -D nb_execution_mode=off docs docs/_build/html
```

For notebook behavior changes, omit `-D nb_execution_mode=off` to use the configured
execution policy.


