Metadata-Version: 2.4
Name: dioptrin-io
Version: 0.1.0
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: 3.15
Classifier: Programming Language :: Rust
Classifier: Topic :: Scientific/Engineering
Requires-Dist: numpy>=2,<3
Requires-Dist: pytest>=8 ; extra == 'test'
Requires-Dist: h5py>=3.11 ; extra == 'test'
Requires-Dist: tifffile>=2024.1.30 ; extra == 'test'
Provides-Extra: test
License-File: LICENSE
License-File: licenses/hdf5/Bitshuffle.txt
License-File: licenses/hdf5/HDF5.txt
License-File: licenses/hdf5/LZ4.txt
License-File: licenses/hdf5/LZF.txt
License-File: licenses/hdf5/libz-sys-MIT.txt
License-File: licenses/hdf5/zlib.txt
License-File: licenses/native-readers/CIF2-UNICODE-NOTICES.md
License-File: licenses/native-readers/FabIO-2026.6.0-copyright.txt
License-File: licenses/native-readers/FabIO-Esperanto-AGI-MIT.txt
License-File: licenses/native-readers/FabIO-GE-Pixi-MIT.txt
License-File: licenses/native-readers/FabIO-MAR345-MIT.txt
License-File: licenses/native-readers/FabIO-Xcalibur-MIT.txt
License-File: licenses/native-readers/FabIO-discovery-MIT.txt
License-File: licenses/native-readers/FabIO-series-MIT.txt
License-File: licenses/native-readers/FabIO-sparse-MIT.txt
License-File: licenses/native-readers/IMAGE-CODECS-NOTICES.md
License-File: licenses/native-readers/LibTIFF-OJPEG.txt
License-File: licenses/native-readers/Pillow-13.0.0-LICENSE.txt
License-File: licenses/native-readers/bzip2-0.6.1-LICENSE-MIT.txt
License-File: licenses/native-readers/caseless-0.2.2-LICENSE.txt
License-File: licenses/native-readers/data-encoding-LICENSE-MIT.txt
License-File: licenses/native-readers/dxtbx-TY5-BSD-3-Clause.txt
License-File: licenses/native-readers/fabio-xsdimage-LICENSE.txt
License-File: licenses/native-readers/fax-0.2.7-LICENSE
License-File: licenses/native-readers/jpeg-decoder-0.3.2-LICENSE-APACHE
License-File: licenses/native-readers/jpeg-decoder-0.3.2-LICENSE-MIT
License-File: licenses/native-readers/jpeg2k-0.10.1-LICENSE
License-File: licenses/native-readers/legacy-layouts-MIT.txt
License-File: licenses/native-readers/libbz2-rs-sys-0.2.6-LICENSE.txt
License-File: licenses/native-readers/libtiff-4.7.2-LICENSE.txt
License-File: licenses/native-readers/lz4_flex-0.14.0-LICENSE.txt
License-File: licenses/native-readers/mrc-0.8.0-LICENSE.txt
License-File: licenses/native-readers/openjpeg-sys-1.0.12-LICENSE
License-File: licenses/native-readers/roxmltree-0.21.2-LICENSE-APACHE.txt
License-File: licenses/native-readers/roxmltree-0.21.2-LICENSE-MIT.txt
License-File: licenses/native-readers/roxmltree-LICENSE-MIT.txt
License-File: licenses/native-readers/tempfile-3.27.0-LICENSE-MIT.txt
License-File: licenses/native-readers/tiff-0.11.3-LICENSE.txt
License-File: licenses/native-readers/tinyvec-1.13.3-LICENSE-APACHE.md
License-File: licenses/native-readers/tinyvec-1.13.3-LICENSE-MIT.md
License-File: licenses/native-readers/tinyvec-1.13.3-LICENSE-ZLIB.md
License-File: licenses/native-readers/turbojpeg-1.5.1-LICENSE
License-File: licenses/native-readers/turbojpeg-sys-1.2.0-LICENSE.md
License-File: licenses/native-readers/turbojpeg-sys-1.2.0-README.ijg
License-File: licenses/native-readers/unicode-normalization-0.1.24-COPYRIGHT.txt
License-File: licenses/native-readers/unicode-normalization-0.1.24-LICENSE-APACHE.txt
License-File: licenses/native-readers/unicode-normalization-0.1.24-LICENSE-MIT.txt
License-File: licenses/native-readers/weezl-0.1.12-LICENSE-APACHE
License-File: licenses/native-readers/weezl-0.1.12-LICENSE-MIT
License-File: licenses/native-readers/zune-core-0.5.3-LICENSE-APACHE
License-File: licenses/native-readers/zune-core-0.5.3-LICENSE-MIT
License-File: licenses/native-readers/zune-core-0.5.3-LICENSE-ZLIB
License-File: licenses/native-readers/zune-jpeg-0.5.15-LICENSE-APACHE
License-File: licenses/native-readers/zune-jpeg-0.5.15-LICENSE-MIT
License-File: licenses/native-readers/zune-jpeg-0.5.15-LICENSE-ZLIB
License-File: licenses/python/pyo3-LICENSE-APACHE.txt
License-File: licenses/python/pyo3-LICENSE-MIT.txt
License-File: licenses/python/rust-numpy-LICENSE.txt
License-File: vendor/fax-0.2.7/LICENSE
License-File: vendor/hdf5-bitshuffle-1.0.0/ext/bitshuffle/LICENSE
License-File: vendor/hdf5-bitshuffle-1.0.0/ext/bitshuffle/lz4/LICENSE
License-File: vendor/tiff-0.11.3/LICENSE
License-File: native/tiff-thunderscan/LICENSE.md
Summary: Native detector-file readers returning original counts as NumPy arrays
Author: Clemens Prescher
License-Expression: MIT
Requires-Python: >=3.10
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Documentation, https://dioptrin-io.readthedocs.io/en/latest/PYTHON_API.html
Project-URL: Issues, https://github.com/CPrescher/dioptrin-io/issues
Project-URL: Repository, https://github.com/CPrescher/dioptrin-io

# dioptrin-io for Python

Native detector-file readers with NumPy image arrays. All decoding uses the
existing Rust library; FabIO and h5py are not runtime dependencies.

Install from PyPI:

```sh
python -m pip install dioptrin-io
```

The release provides standard CPython 3.10–3.15 wheels for macOS Intel/Apple
Silicon, Windows x64, and manylinux 2.28+ x64/ARM64. Windows ARM64 wheels cover
CPython 3.13–3.15. Supported wheels need no Rust compiler, CMake, FabIO or h5py
installation; NumPy is the runtime Python dependency. macOS wheels target 11.0+.

From a source checkout (Rust >=1.89, a C/C++ compiler, CMake and Git required):

```sh
python -m pip install .
```

```python
import dioptrin_io

with dioptrin_io.open("image.edf") as source:
    print(source.nframes)  # Detected automatically, for single images and stacks
    image = source[0]  # Image object; equivalent to source.read(0)
    print(image.data.shape, image.data.dtype)
    header = image.header
    metadata = image.metadata

with dioptrin_io.open("scan.h5", dataset="/entry/data/data") as scan:
    print(scan.nframes)
    first = scan[0]
    if scan.nframes > 1:
        second = scan[1]
    for image in scan:  # Decodes one frame at a time
        print(image.data.shape, image.header, image.metadata)
```

`open()` is the single file-opening entry point. It inspects the file and exposes
`source.nframes` (also `len(source)`) without loading the complete image stack
into memory. Some formats require an indexing scan; gzip/bzip2 wrappers may be
decompressed into a temporary file on disk. `source[index]` or
`source.read(index)` decodes one selected frame as an `Image` containing `.data`
(a NumPy array), `.header` (the parsed source header dictionary) and `.metadata`
(additional type, layout and provenance information).
This works for both single-image files
(`nframes == 1`) and multiframe files, so you do not need to know beforehand
which kind of file you have. `source.read()` selects frame zero. The data,
header and metadata are captured together under the reader's source checks.
Each returned Image owns independent data, header and metadata snapshots; all
remain valid after the source is closed. Editing a returned array or dictionary
affects only that Image, not the source or later reads.

Iteration yields Image objects in stored frame order, loading a frame only when
`next()` requests it. Each iterator starts at frame zero and has its own position;
indexing and `read()` do not change it. Keep the reader open during iteration.
Read failures propagate, including source changes and a reader closed midway
through iteration. Exhausted iterators remain exhausted. Indexing accepts
nonnegative integer frame indices; negative indices and out-of-range indices
raise `IndexError`. Slices are not supported. `list(source)` explicitly collects
all returned Images in memory; ordinary iteration does not retain earlier Images.

Paths accept strings, bytes and `os.PathLike` objects. Content discovery, not the
extension, selects a reader. Eiger, LImA and Lambda HDF5 conventions can be
discovered; other HDF5 files need `dataset=`. This initial Python API rejects
external HDF5 links and does not expose VDS policies, raw layouts, partial-source
recovery, sparse reconstruction, file series or format-specific compatibility
options. Such inputs fail explicitly. Only paths, not Python file-like objects,
are accepted in this initial interface.

Arrays preserve stored axes, original counts and reserved values. Integer
widths stay integer; complex values stay complex. Color arrays use
`(rows, columns, channels)`. Fortran NPY storage is reordered into the same logical
axes. Arrays use native byte order and independent writable NumPy storage: one
copy from the decoded Rust buffer, without conversion through Python lists or
float64. Arrays remain valid after closing, reading another frame or a later
read error. No intensity/geometry corrections, flips or TIFF Orientation
transformations are applied. Existing reader limitations still apply, including
TIFF binary16 widening to float32.

NPY/HDF5 binary16 maps to `float16` without changing its bits, and Boolean arrays
retain original storage bytes (inspect with `.view("uint8")` when needed).
Integer complex MRC samples use a structured dtype with signed16 `real` and
`imag` fields, preserving exact pairs rather than casting to floating complex.

## Header fields and additional metadata

`image.header` contains the fields exposed by the existing native reader. Values
retain their parsed source types: textual numbers remain strings, binary numbers
become Python integers/floats, byte fields stay bytes, and arrays stay lists or
tuples. No units, corrections or orientations are applied. Field spelling and
case follow each native reader (EDF keys, for example, are uppercase). Original
header bytes remain separately available in `image.metadata["raw_header"]` for
formats with retained contiguous header bytes.

| Format | `.header` dictionary |
|---|---|
| NPY | `descr`, `fortran_order`, original `shape` tuple |
| EDF, d*TREK/HiPiC and text detector headers | Native parsed assignment maps; EDF includes inherited general fields |
| TIFF/BigTIFF | Original integer tag IDs, including private tags; rational values are exact `(numerator, denominator)` tuples, including zero denominators |
| Detector TIFF | TIFF tags plus an `acquisition` dictionary from the detector header |
| CBF | Separate `cif`, `binary` and `loops` dictionaries/lists; Pilatus text assignments under `pilatus` when available |
| HDF5 | Attribute dictionaries keyed by absolute object path: `/` and the selected dataset's ancestors and dataset; no other datasets are loaded |
| Bruker, Oxford, R-AXIS, FIT2D, GE, MPA | Native parsed header fields, retaining text, numeric and byte values |
| MAR345 | Native text assignments plus the original `binary_words`; no unit conversion through derived scalar helpers |
| Esperanto | Original grouped text assignments |
| SPE | Native source fields; fabricated defaults and reformatted timestamps are excluded |
| MRC | Typed MRC header fields, with original label, stamp and reserved byte blocks |
| JPEG | Original marker `segments` and the existing parser's `jfif` density records |
| JPEG2000 | Original box type/payload records and `codestream_header` bytes |
| DM3 | Source tag records with paths, names, descriptors, small raw values and explicit offsets/lengths for large values; image arrays are not loaded as header fields |
| Pixi | Validated `record_size`; the remaining opaque bytes stay in metadata |
| XSD | Native parsed dtype, shape, size, coding and optional checksum declarations |

Uninterpreted data is kept explicit rather than guessed: TIFF opaque tags are
retained under `metadata["opaque_tags"]`; CBF source order, data-block names and
parser diagnostics remain under `cif_entries`, `cif_data_blocks` and
`cif_diagnostics`. For header-only inspection, `scan.header(index)` returns the
same dictionary without decoding image pixels.

HDF5 attributes preserve scalar/array shape, integer values, UTF-8 strings and
ASCII bytes. Canonical Boolean codes become Python booleans; other stored codes
stay integers. Empty arrays become empty lists and null dataspaces become `None`.
Attribute dtype/byte-order descriptions and original shapes are retained in
`metadata["header_attribute_types"]` and `metadata["header_attribute_shapes"]`.
Compound, enum, reference and other unsupported attribute types are explicitly
listed in `metadata["header_unsupported"]` and are not substituted with values.
Inspection is bounded to 64 ancestry/rank levels, 4096 attributes and a 16 MiB
metadata budget, including Python list structure. Variable-length string sizes
are checked after the HDF5 library decodes them, as with existing native text
metadata reads. This is scoped attribute access, not a scan of the entire NeXus
hierarchy or an interpretation of neighboring datasets as header fields.

`Image.metadata` also contains `format` and `details`. The latter preserves
the Rust source metadata as diagnostic text and is **not a stable serialization
schema**. Retained original contiguous header bytes are named `raw_header`;
native/EDF/HDF5 provides `source_shape`, native/EDF/TIFF/HDF5 provides
`little_endian`, and HDF5 provides `type_descriptor`. Consult original metadata
for source dtype declarations and format-specific storage interpretations.
`Reader.metadata(frame=0)` remains available for metadata-only inspection
without decoding image pixels; it returns the same dictionary structure.

Readers capture immutable source snapshots and check for changed files on every
read. Close and reopen after acquisition/file changes. Reads release the Python
GIL; calls on one reader are serialized. Negative/out-of-range frame indices
raise `IndexError`, closed readers raise `ValueError`, and decoder/source/bounds
errors raise `OSError`. The Rust readers' default resource limits apply (normally
256 MiB per decoded frame); this initial Python API does not override them.

The default Python build enables HDF5 and the available dense image formats.
Rust consumers retain empty default features and no Python dependency. A smaller
Python build can use `maturin build --no-default-features`, retaining
NPY/d*TREK/HiPiC/PNM, TIFF, CBF, EDF and outer gzip/bzip2 wrappers.

## Development and packaging

```sh
python -m pip install maturin numpy pytest h5py tifffile
maturin develop
python -m pytest
cargo fmt --manifest-path python/Cargo.toml --check
cargo clippy --manifest-path python/Cargo.toml --all-targets --all-features -- -D warnings
maturin build --release --out dist
maturin sdist --out dist
```

The source distribution includes the core path dependency and vendored native
backends. Wheels include native-reader/codec license notices. Building a wheel
locally does not publish it to PyPI. The `Python release wheels` workflow builds
and tests every release wheel. `Publish Python package` validates the complete
artifact matrix and tested commit before uploading through PyPI Trusted
Publishing. Existing Rust tag `v0.1.0` retains its original source release;
Python releases use separate `python-vVERSION` tags.

h5py is an optional independent test encoder. Where its CPython 3.15 encoder
wheel is unavailable, those encoder-generated tests are explicitly skipped;
the pre-generated licensed HDF5 corpus is still compared bit-for-bit through
the Python/NumPy interface on every release interpreter.

