Metadata-Version: 2.4
Name: iatro-base-iac
Version: 0.1.3
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Dist: pyarrow
Requires-Dist: numpy
Requires-Dist: brotli ; extra == 'dev'
Requires-Dist: imagecodecs ; extra == 'dev'
Requires-Dist: pytest ; extra == 'dev'
Provides-Extra: dev
License-File: LICENSE
License-File: THIRD_PARTY_NOTICES.md
Summary: IatroCache (.iac): a lightweight medical data cache format
License-Expression: MIT
Requires-Python: >=3.9
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Homepage, https://github.com/iatrode/iatro-base-iac
Project-URL: Repository, https://github.com/iatrode/iatro-base-iac

# IatroCache

[![CI](https://github.com/iatrode/iatro-base-iac/actions/workflows/ci.yml/badge.svg)](https://github.com/iatrode/iatro-base-iac/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/iatro-base-iac.svg)](https://pypi.org/project/iatro-base-iac/)

IatroCache (`.iac`) is a lightweight, high-throughput binary cache format for
multimodal medical datasets. It combines Arrow metadata with directly
addressable payload bytes in one immutable file, supporting image tiles,
feature vectors, clinical text, DICOM instances, and project-defined schemas.

The repository contains two public packages:

- `iatro-base-iac`: the v2 container, native reader/writer, record packs, and
  codecs.
- `iatro-iac-adapters`: reusable schemas for tiles, WSI images, teacher
  features, clinical text, paired text, and patient-level DICOM.

## Why IatroCache

- Explicit record boundaries through metadata rather than stream scanning.
- Owned batch reads and an explicit mmap-backed zero-copy API.
- Variable-length records and fixed-width dense matrices in one format.
- Arrow tables for searchable, schema-rich metadata.
- Native checked addressing, I/O, and built-in codecs, with modality policy
  kept in Python adapters.
- Independent per-record compression for random access and data-loader
  concurrency.
- Native Raw, Brotli, Zstandard, JPEG XL, JPEG, JPEG 2000/HTJ2K, and JPEG-LS
  support for clinical text, radiology, and pathology payloads.

## Installation

Install the general container API:

```bash
pip install iatro-base-iac
```

Install the reusable medical-data schemas and matching core dependency:

```bash
pip install iatro-iac-adapters
```

## Quick start

```python
import pyarrow as pa

from iatro.iac import Codec, PackReader, build_pack

slides = pa.table({
    "slide_idx": pa.array([0], type=pa.uint8()),
    "slide_id": ["slide-001"],
    "patient_id": ["patient-001"],
})
items = pa.table({"item_id": ["a", "b"]})

build_pack(
    "example.iac",
    {"payload_type": "raw_bytes", "codec": Codec.NONE},
    slides,
    items,
    [b"first", b"second"],
)

reader = PackReader("example.iac")
try:
    payload = reader.read_payload(1)
    batch = reader.read_payloads([1, 0, 1])
    metadata = reader.index_table
finally:
    reader.close()
```

The writer adds `offset`, `length`, and `crc32` columns for variable records.
Batch reads preserve requested order and duplicates and return owned data.
`read_payload_views()` is the separate, explicit zero-copy interface.

## Documentation

The [IatroCache Wiki](docs/index.md) covers:

- [design and architecture](docs/design.md);
- [the v2 file format](docs/file-format.md);
- [the complete core API](docs/core-api.md) and [codecs](docs/codecs.md);
- [domain adapters](docs/adapters.md) for
  [pathology data](docs/modalities/pathology.md),
  [clinical text](docs/modalities/clinical-text.md), and
  [DICOM](docs/modalities/dicom.md);
- [validation](docs/validation.md),
  [performance and concurrency](docs/performance.md), and
  [contributing](docs/contributing.md).

## Format at a glance

```text
[ fixed 64 KiB header ] magic + version + JSON layout and schema fields
[ slide table         ] Arrow IPC stream
[ index table         ] Arrow IPC stream
[ data segment        ] variable payloads or a fixed-width matrix
```

The current package version is `0.1.3`; the on-disk format version is `2`.
Those version domains evolve independently.

## License

IatroCache is released under the MIT License.

