Metadata-Version: 2.5
Name: clearscale
Version: 0.9.0rc2
Summary: Clear multiscale image metadata manipulation
Project-URL: Documentation, https://clearscale.readthedocs.io/
Project-URL: Source, https://github.com/ilastik/clearscale
Author-email: "Benedikt T. Best" <benedikt.best@embl.de>
Maintainer-email: "Benedikt T. Best" <benedikt.best@embl.de>, ilastik team <team@ilastik.org>
License-Expression: MIT OR Apache-2.0
License-File: LICENSE
License-File: LICENSE-APACHE
License-File: LICENSE-MIT
Keywords: image-metadata,multiscale,ngff,ngff-zarr,ome-ngff,ome-zarr,precomputed
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Image Processing
Requires-Python: >=3.10
Provides-Extra: dev
Requires-Dist: pre-commit; extra == 'dev'
Requires-Dist: pyright; extra == 'dev'
Requires-Dist: pytest; extra == 'dev'
Provides-Extra: docs
Requires-Dist: mkdocstrings-python==2.0.10; extra == 'docs'
Requires-Dist: zensical==0.0.74; extra == 'docs'
Description-Content-Type: text/markdown

# clearscale

`clearscale` is a small dependency-free Python package for clear multiscale image metadata manipulation.

## Features

* Zero dependencies, pure Python
* Compatible with Python 3.10+
* Read OME-Zarr metadata and even Neuroglancer Precomputed (very basic support)
* Read all OME-Zarr versions (0.1 to 0.6) using one code path - no version branching, future versions come for free
* Write OME-Zarr versions 0.4, 0.5 and 0.6
* Virtually no knowledge about OME-Zarr required
* Create, manipulate, save and load multiscale metadata
* Make sure metadata are correct for any scaling method
* Find out how exactly your scaling method resamples with [characterization](docs/characterization.md)
* Read and write multiscale transformations into alternate [coordinate systems](docs/space.md)
* Read and write transformations between multiscales using [scenes](docs/scene.md)

## Minimal OME-Zarr example

The absolute minimum requirements for writing OME-Zarr:
* axis keys (`"zyx"`)
* writing the data to a zarr *array* inside a zarr *group* (and recording the array sub-path)
* `zarr_format=3` for OME-Zarr `version="0.5"`

```python
# Assuming you have numpy and zarr-python 3.* installed
import numpy as np
import zarr
import clearscale as cs

# You have some array
image = np.random.random((128, 1024, 1024)).astype(np.float32)

# 1. Data: Make a group and write the array into it
# (no clearscale involved)
group = zarr.open_group(
    "demo-output/example1.ome.zarr", 
    mode="w", 
    zarr_format=3
)
array_path = "s0"
group.create_array(array_path, data=image)

# 2. Metadata: Scale -> Multiscale -> Group -> attrs
group.attrs.update(
    cs.OmeZarrGroup
    .from_single(
        cs.Multiscale.from_single(
            scale=cs.Scale.from_lists("zyx"), 
            scale_key=array_path
        )
    )
    .to_attrs(version="0.5")
)
```

## Zero dependencies

Being dependency-free means:

* `clearscale` fits in any Python environment. Installing or uninstalling it will not modify any other installed packages.
* `clearscale` does *not* interact with data (numpy/zarr arrays). You implement data processing according to your needs, and use `clearscale` to generate metadata that describes your processing.

This provides full flexibility for data processing, but it comes at the risk of desynchronization between data and metadata.

You can mitigate this by coupling data processing to the metadata, for example by using the generated metadata as the parameter for image scaling and data writing:

```python
import clearscale as cs

# 1. Define operation plan as metadata
shape = cs.Shape(zip("tzyx", my_data.shape))
scaling_blueprint = cs.BlueprintShapes.downscale_powers_of_2_xyz(
    base_shape=shape,
    rounding="ceil",
    limit_all=cs.Shape(z=8, y=128, x=128),
)

# 2. Scale data according to the blueprint.
# Metadata as parameter for data processing.
for scale_key, target_shape in scaling_blueprint.items():
    # Target shape from blueprint
    scaled_data = do_my_scaling(my_data, target_shape.values_tuple())
    # Target array path from blueprint
    zarr_group.create_array(scale_key, data=scaled_data)

# 3. Write metadata: Scale -> Multiscale -> Group -> attrs
pixel_size = cs.PixelSize(t=5.0, z=260.0, y=0.53, x=0.53)
unit       = cs.Unit(t="s", z="micrometer", y="micrometer", x="micrometer")
base       = cs.Scale(shape, pixel_size, unit)

ms         = cs.Multiscale.from_single(base, blueprint=scaling_blueprint)
group_meta = cs.OmeZarrGroup.from_single(ms).to_attrs(version="0.6")
zarr_group.attrs.update(group_meta)
```

For example, if you use `numpy`, `scikit-image` and `zarr-python`, the placeholders above could look like:

```
import os, numpy, skimage, zarr

my_data = numpy.random.rand(3, 12, 512, 512)
do_my_scaling = skimage.transform.resize
zarr_group = zarr.open_group(os.path.expanduser("demo-output/sweet.ome.zarr"), mode="w")

# Now you can actually run the example snippet above and produce a valid `sweet.ome.zarr`
```

## Install

`clearscale` is considered in "beta" status for a short time, with 1.0 release intended soon(TM), when we have some feedback from first users.

Please try it and let us know if you encounter anything unclear!

Install the 0.9.0 release candidate from PyPI:

```bash
python -m pip install clearscale==0.9.0rc1
```

Or add to your conda `env.yaml`:

```yaml
dependencies:
  - pip
  - pip:
      - clearscale==0.9.0rc1
```

Note that until we publish 1.0, the API is considered subject to change so we can ensure it "solidifies" with as much convenience and robustness as possible.
Though we hope there won't be too many changes necessary at this point :)

## Documentation

Rendered guides (OME-Zarr how-to, core concepts, metadata scaling, ...) and automatically generated API Reference are in the [clearscale documentation](https://clearscale.readthedocs.io/).

The handwritten guides are also directly under [`docs/`](docs/index.md) in this repository.

## End-to-end examples

### Downsample a numpy array and save it as OME-Zarr

The previous example used a "metadata-first" approach (first compute metadata blueprint, then scale data according to it).

This example works the other way round (first scale data, then record what was done in blueprint).

```python
# This example assumes numpy, scikit-image and zarr-python 3.* are installed
import numpy as np
import zarr
from skimage.transform import pyramid_gaussian

import clearscale as cs

# 1. Generate a pyramid
image = np.random.random((128, 1024, 1024)).astype(np.float32)
pyramid = [level.astype(np.float32) for level in pyramid_gaussian(image, downscale=2)]

# 2. Write arrays to zarr on disk and record scaled shapes
group = zarr.open_group("demo-output/example2.ome.zarr", mode="w")
scaled_shapes = []
for i, level in enumerate(pyramid):
    scale_key = f"s{i}"
    group.create_array(scale_key, data=level)
    scaled_shapes.append(
        (
            scale_key,
            cs.Shape(zip("zyx", level.shape))
        )
    )

# 3. Describe the full-resolution image
base = cs.Scale.from_lists(
    keys="zyx",
    shape=image.shape,
    pixel_size=[25, 240, 240],
    unit=["micrometer", "nanometer", "nanometer"],
)

# 4. Use the recorded scale shapes as a blueprint to expand a Multiscale
blueprint = cs.BlueprintShapes(scaled_shapes)
multiscale = cs.Multiscale.from_single(base, blueprint=blueprint)

# 5. Save OME-Zarr metadata
group_meta = cs.OmeZarrGroup.from_single(multiscale).to_attrs(version="0.5")
group.attrs.update(group_meta)
```

### Download a single scale of a public OME-Zarr as a valid local OME-Zarr

```python
from pathlib import Path
import zarr  # This example assumes zarr-python 3.* is installed (plus fsspec, requests and aiohttp for https reading)
import clearscale as cs

URL = "https://s3.embl.de/i2k-2020/platy-raw.ome.zarr"
SCALE_KEY = "s6"
LOCAL_PATH = Path(f"demo-output/platy-raw-{SCALE_KEY}.ome.zarr")

# 1. Discover what's on the server
remote_group = zarr.open_group(URL)

remote_meta = cs.OmeZarrGroup.from_group(remote_group)
assert remote_meta.kind is cs.GroupKind.MULTISCALE, "This tells us there is exactly one Multiscale in this group"
assert remote_meta.version == "0.4", "OME-Zarr version is a property of the group, not of individual Multiscales"

remote_multiscale = remote_meta.multiscales[0]
assert SCALE_KEY in remote_multiscale, "Multiscale is dict-like, with the sub-paths to arrays as keys"
assert remote_multiscale[SCALE_KEY].shape == {"z": 357, "y": 405, "x": 430}

# 2. Create the local target and download the data
source_array = remote_group[SCALE_KEY]
local_group = zarr.open_group(str(LOCAL_PATH), mode="w", zarr_format=2)  # OME-Zarr v0.4 must use zarr-format v2
print(f"Downloading {SCALE_KEY} data...")
local_array = local_group.create_array(SCALE_KEY, data=source_array)

# 3. Derive a new Multiscale from the chosen Scale entry in the existing Multiscale
local_multiscale = remote_multiscale.derive(SCALE_KEY)

# 4. Write the new metadata to the downloaded store
local_group_meta = cs.OmeZarrGroup.from_single(local_multiscale)
local_group.attrs.update(local_group_meta.to_attrs(version="0.4"))
```

## Why clearscale?

The motivation behind clearscale is to make upgrading from

> My tool/script does **zarr**

to

> My tool/script does **OME-Zarr**

as easy as possible.

### Why not use one of the existing libraries?

You *should* probably use one of the existing libraries, if they work for you :)

Check out the [NGFF Tools](https://ngff.openmicroscopy.org/resources/tools/index.html) page for some examples.
Python libraries include e.g.
[bioio](https://github.com/bioio-devs/bioio),
[ngff-zarr](https://github.com/thewtex/ngff-zarr),
[ngio](https://biovisioncenter.github.io/ngio/stable/),
[ome-zarr-py](https://github.com/ome/ome-zarr-py), and
[yaozarrs](https://github.com/tlambert03/yaozarrs).
Several of these handle not only the metadata, but also data manipulation all in one.

### Dependencies are hard

But maybe you can't, or don't want to, use any of the above.

* Technical reasons (dependency conflicts)
* Legal reasons (licensing conflicts, commercial use)
* You already have data processing in place that you don't want to reimplement or risk adapting inefficiently.
* You just want to minimise bloat.

clearscale is tiny, dependency-free, permissively licensed, and works around your existing data processing.

### Zarr is complex enough

There
[are](https://github.com/zarr-developers/zarr-python)
[several](https://github.com/canpute/simplezarr)
[backend](https://github.com/google/tensorstore)
[packages](https://github.com/constantinpape/z5)
your application or script might use for reading and writing zarr data.
Implementing data processing in an efficient, chunk-wise (shard-wise) manner is complex on its own.
You shouldn't need to learn the OME metadata specification on top to make your datasets accessible to others, or interoperable with other tools.

With clearscale, no matter how you handle zarr *data*, handling multiscale *metadata* like OME-Zarr looks the same.

### Custom applications

There are thousands of ways to scale or otherwise transform an image.
Maybe none of the existing libraries that can write OME-Zarr implements quite what you need, so you have to roll your own.

clearscale isn't tied to data processing.
Whatever you're doing, clearscale can simplify metadata manipulation.

### Should I use `clearscale` in my project right now?

If you're willing to adapt to future changes in the API: Yes, please do try it!

If you are looking for a stable API, please wait until the first formal release to PyPI.
After that point, API changes will follow semver and deprecations will be avoided or give appropriate notice through DeprecationWarnings.

## Contributing / Feature scope
Any form of feedback, issues, bug reports, feature requests, pull requests and other contributions are welcome!
See [CONTRIBUTING.md](CONTRIBUTING.md).

## License

Licensed under either the [MIT license](LICENSE-MIT) or the
[Apache License, Version 2.0](LICENSE-APACHE), at your option.
