Metadata-Version: 2.5
Name: chunkmirage
Version: 0.1.0a1
Summary: Spoof chunked array formats (zarr, n5, neuroglancer precomputed) over HTTP with on-the-fly processing, caching, and pluggable ops.
Project-URL: Homepage, https://janeliascicomp.github.io/chunkmirage/
Project-URL: Documentation, https://janeliascicomp.github.io/chunkmirage/
Project-URL: Source, https://github.com/JaneliaSciComp/chunkmirage
Project-URL: Issues, https://github.com/JaneliaSciComp/chunkmirage/issues
Project-URL: Changelog, https://github.com/JaneliaSciComp/chunkmirage/blob/main/CHANGELOG.md
Author: David Ackerman (@davidackerman), Yurii Zubov (@yuriyzubov)
License: BSD 3-Clause License
        
        Copyright (c) 2026, Howard Hughes Medical Institute
        
        Redistribution and use in source and binary forms, with or without
        modification, are permitted provided that the following conditions are met:
        
        1. Redistributions of source code must retain the above copyright notice, this
           list of conditions and the following disclaimer.
        
        2. Redistributions in binary form must reproduce the above copyright notice,
           this list of conditions and the following disclaimer in the documentation
           and/or other materials provided with the distribution.
        
        3. Neither the name of the copyright holder nor the names of its
           contributors may be used to endorse or promote products derived from
           this software without specific prior written permission.
        
        THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
        AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
        IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
        DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
        FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
        DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
        SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
        CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
        OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
        OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
License-File: LICENSE
Keywords: chunked,imaging,n5,neuroglancer,tensorstore,virtual,zarr
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: BSD License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Image Processing
Classifier: Typing :: Typed
Requires-Python: >=3.11
Requires-Dist: hypercorn[h2]>=0.17
Requires-Dist: numcodecs>=0.12
Requires-Dist: numpy>=1.26
Requires-Dist: pydantic>=2.6
Requires-Dist: starlette>=0.37
Requires-Dist: tensorstore>=0.1.60
Requires-Dist: typer>=0.12
Requires-Dist: uvicorn[standard]>=0.29
Provides-Extra: all
Requires-Dist: cryptography>=42; extra == 'all'
Requires-Dist: dracopy>=1.4; extra == 'all'
Requires-Dist: h5py>=3.10; extra == 'all'
Requires-Dist: imagecodecs>=2024.1; extra == 'all'
Requires-Dist: mcp>=1.0; extra == 'all'
Requires-Dist: neuroglancer>=2.40; extra == 'all'
Requires-Dist: scikit-image>=0.22; extra == 'all'
Requires-Dist: scipy>=1.12; extra == 'all'
Requires-Dist: tifffile>=2024.1; extra == 'all'
Provides-Extra: cpu
Requires-Dist: scipy>=1.12; extra == 'cpu'
Requires-Dist: torch>=2.6; extra == 'cpu'
Provides-Extra: gpu
Requires-Dist: scipy>=1.12; extra == 'gpu'
Requires-Dist: torch>=2.6; extra == 'gpu'
Provides-Extra: hdf5
Requires-Dist: h5py>=3.10; extra == 'hdf5'
Provides-Extra: https
Requires-Dist: cryptography>=42; extra == 'https'
Provides-Extra: mcp
Requires-Dist: mcp>=1.0; extra == 'mcp'
Provides-Extra: mesh
Requires-Dist: dracopy>=1.4; extra == 'mesh'
Requires-Dist: scikit-image>=0.22; extra == 'mesh'
Provides-Extra: ops
Requires-Dist: scipy>=1.12; extra == 'ops'
Provides-Extra: tiff
Requires-Dist: imagecodecs>=2024.1; extra == 'tiff'
Requires-Dist: tifffile>=2024.1; extra == 'tiff'
Provides-Extra: viewer
Requires-Dist: neuroglancer>=2.40; extra == 'viewer'
Description-Content-Type: text/markdown

# chunkmirage

**Spoof chunked array formats over HTTP with on-the-fly processing.**

chunkmirage serves *virtual* datasets that look, to any HTTP-capable viewer or library
(Neuroglancer, BigDataViewer/Fiji, vizarr, napari, webKnossos, zarr-python, dask,
tensorstore, ...), like ordinary Zarr v2, Zarr v3, N5 or Neuroglancer precomputed volumes.
Nothing exists on disk. Each chunk is computed when it is requested: read from a real
source (zarr, N5, precomputed, HDF5, GeoTIFF; local, S3, GCS or HTTP) or generated, pushed
through a pipeline of ops, encoded in the format the client asked for, and cached per
stage, so changing a parameter downstream never recomputes what comes before it.

**▶ [Try the demos in your browser](https://janeliascicomp.github.io/chunkmirage/browser/)**,
nothing to install. Every chunk on screen is computed in the page by chunkmirage's own
Python, from public data, as the viewer asks for it.

<p>
<a href="https://janeliascicomp.github.io/chunkmirage/browser/pipeline.html?card=fires"><img src="https://raw.githubusercontent.com/JaneliaSciComp/chunkmirage/main/web/public/cards/fires.jpg" alt="Los Angeles fires: burn severity, read by Neuroglancer, a web map or GDAL" title="Los Angeles fires: burn severity, read by Neuroglancer, a web map or GDAL" width="32%"></a>
<a href="https://janeliascicomp.github.io/chunkmirage/browser/pipeline.html?card=mandelbulb"><img src="https://raw.githubusercontent.com/JaneliaSciComp/chunkmirage/main/web/public/cards/mandelbulb.jpg" alt="A 3-D fractal 2^28 voxels across" title="A 3-D fractal 2^28 voxels across" width="32%"></a>
<a href="https://janeliascicomp.github.io/chunkmirage/browser/pipeline.html?card=fronts"><img src="https://raw.githubusercontent.com/JaneliaSciComp/chunkmirage/main/web/public/cards/fronts.jpg" alt="The Gulf Stream's fronts" title="The Gulf Stream's fronts" width="32%"></a>
</p>

They include burn severity of the Los Angeles fires, the Gulf Stream's fronts and
hurricanes' cold wakes, landing slopes at the Moon's south pole, a 3-D fractal, microscope
tiles stitched by RANSAC, two fly brains registered on your GPU, organelle contact sites,
mRNA spots and nuclei tracked through a colony. Each shows the `chunkmirage serve` command
that serves the same from Python. The [demos page](https://janeliascicomp.github.io/chunkmirage/demos/) lists them all, with the
Python examples.

## How it works

```
viewer  --HTTP-->  chunkmirage  --tensorstore/h5py-->  real data (zarr/n5/precomputed/hdf5, file/s3/gcs/http)
                      |
                      +-- pipeline: source -> [op, op, ...] -> encoded chunk
                      +-- per-stage chunk cache keyed by pipeline hash
                      +-- frontends: n5 | zarr (v2) | zarr3 | precomputed, all served at once
                      +-- REST API for live pipeline edits, and plugins for ops, sources and routes
```

It was inspired by [example-virtual-n5](https://github.com/stuarteberg/example-virtual-n5)
and [cellmap-flow](https://github.com/janelia-cellmap/cellmap-flow), which grew out of it,
and makes their trick general: any format, any source, any per-chunk computation, any
client.

## Quick start

```bash
uv sync --extra all --group dev            # or: pip install -e ".[all]"
chunkmirage serve /path/to/data.zarr/em/fibsem-uint8 --op threshold:low=120
```

With no data at all, a generated 4096³ volume, segmented live:

```bash
uv run chunkmirage serve "synthetic://blobs+noise?shape=4096,4096,4096" \
    --op gaussian:sigma=1.5 --op threshold:low=110 \
    --op morphology:operation=open,radius=2 --op label:min_size=200 --python-viewer
```

Open the printed control page (`http://<your-ip>:8000/ui`) and drag the `threshold` slider:
only the changed stage recomputes. Or point any viewer at one of these:

| viewer source URL                                        | format                   |
| -------------------------------------------------------- | ------------------------ |
| `n5://http://localhost:8000/<name>/n5`                   | N5                       |
| `zarr://http://localhost:8000/<name>/zarr`               | Zarr v2 (+ OME-NGFF 0.4) |
| `zarr3://http://localhost:8000/<name>/zarr3`             | Zarr v3 (+ OME-NGFF 0.5) |
| `precomputed://http://localhost:8000/<name>/precomputed` | Neuroglancer precomputed |

Edit the pipeline without restarting:

```bash
curl -X PUT localhost:8000/api/datasets/<name> -H 'content-type: application/json' \
  -d '{"source": "/path/to/data.zarr/em/fibsem-uint8", "ops": [{"op": "threshold", "low": 150}]}'
```

More in [getting started](https://janeliascicomp.github.io/chunkmirage/getting-started/), including viewing from another machine.

## As a library

```python
from chunkmirage import Pipeline, open_source, create_app
from chunkmirage.ops import Op, Threshold

class MyModel(Op):
    name = "my_model"
    halo = 16          # voxels of context read on every side
    cache = True       # keep this stage's output chunks

    def apply(self, block):
        return run_my_network(block)

pipe = Pipeline(open_source("s3://bucket/data.zarr/em/s0"), ops=[MyModel(), Threshold(low=120)])
app = create_app({"em": pipe})                   # a Starlette ASGI app
```

Ops, source schemes and HTTP routes from other packages register through entry points
(`chunkmirage.ops`, `chunkmirage.sources`, `chunkmirage.routes`), and the app can be mounted
inside another one. See [pipelines](https://janeliascicomp.github.io/chunkmirage/concepts/pipelines/) and the
[REST API](https://janeliascicomp.github.io/chunkmirage/reference/api/).

## What it does today

* **Sources:** zarr v2/v3, N5, precomputed and HDF5 on file, S3, GCS or HTTP; xarray arrays
  and GeoTIFFs; computed `synthetic://` volumes, `scene://` resampling through OME-Zarr 0.6
  transformations, `register://` deformable registration solved on a GPU, `stitch://`
  BigStitcher tiles stitched and fused as read, and `warp://`, `stack://` and `flip://`.
* **Frontends:** N5, Zarr v2, Zarr v3 and precomputed, all at once, and meshes made when
  fetched.
* **Ops:** pointwise, filters, morphology, connected components, spots, contacts,
  downsampling, slope and hillshade, with halos handled for you.
* **Serving:** per-stage cache, live REST edits, a control page, work ordered by what clients
  ask for and dropped when they stop waiting.
* **In the browser:** the same ops run in Pyodide, and registration on WebGPU.

Not yet: GPU ops other than registration, an MCP server. See the [roadmap](https://janeliascicomp.github.io/chunkmirage/roadmap/).

## Documentation

**https://janeliascicomp.github.io/chunkmirage/** (built from `docs/`; `uv run mkdocs serve`
locally). [the design page](https://janeliascicomp.github.io/chunkmirage/design/) explains the architecture and the choices behind
it.

## License

BSD 3-Clause, Howard Hughes Medical Institute. Authors: TBD (collaborative project).
