Metadata-Version: 2.4
Name: drb-chunk-netcdf
Version: 0.1.0
Summary: Physical chunk model for the drb:netcdf / drb:hdf classes
Author: GAEL Systems
Author-email: drb-python@gael.fr
License: LGPLv3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Environment :: Plugins
Classifier: License :: OSI Approved :: GNU Lesser General Public License v3 (LGPLv3)
Requires-Python: <3.14,>=3.11
Description-Content-Type: text/markdown
License-File: LICENCE.txt
Requires-Dist: drb-chunk<1,>=0.6.1
Requires-Dist: netCDF4
Requires-Dist: numpy
Dynamic: license-file

# drb-chunk-netcdf

netCDF/HDF5-backed physical chunk model for the `drb:netcdf` / `drb:hdf`
classes, built on top of the [drb-chunk](../../core/README.md) add-on.

## What it is

`drb-chunk-netcdf` teaches `drb-chunk` how to read a netCDF-4/HDF5 variable
on its **native chunk boundaries** instead of on whatever grid a product
descriptor happens to declare. It targets the generic `drb:netcdf`/`drb:hdf`
topic classes, not a single product family — a *format* chunk add-on,
sibling to `drb-chunk-zarr` and `drb-chunk-image` (GDAL rasters).

Concretely, it registers a `PhysicalTilingProvider` (`netcdf-chunks`) that
probes a variable's native chunking via `netCDF4`, without loading the full
variable. Classic netCDF-3 variables (and any HDF5 variable stored
contiguously) have no native chunk grid; the provider falls back to the
whole-variable shape as a single physical chunk.

## How it self-activates

Installing the package is enough; there is no product-side wiring:

- It registers `NetcdfChunksProvider` under the `drb.chunk.physical`
  entry-point group
  (`drb.addons.chunk.netcdf.provider:NetcdfChunksProvider`), the seam the
  `drb-chunk` core uses to discover physical-tiling providers lazily, and a
  matching `netcdf-var` physical reader via `register_physical_reader`.
- It ships a `cortex.ttl` carrying two triples —
  `<http://www.gael.fr/drb#netcdf> drb:physicalTiling "netcdf-chunks"` and
  `<http://www.gael.fr/drb#hdf> drb:physicalTiling "netcdf-chunks"` —
  declared under the `drb.topic` entry-point group. The core's `ManagerDao`
  merges every installed `drb.topic` descriptor into one RDF graph, so both
  triples land on the *existing* `drb:netcdf` and `drb:hdf` topics without
  touching the driver or any other package. Both topics share the same
  `drb-driver-netcdf` implementation, so one provider serves both.

Once both are installed, any chunk descriptor whose `drb:source` resolves
to a netCDF/HDF5 variable node automatically gets a physical chunk model:
the core resolves the source's format topic, reads `drb:physicalTiling`,
looks up `netcdf-chunks`, and calls `probe()`.

A netCDF/HDF5 variable node (`DrbNetcdfVariableNode`) does not itself
classify as `drb:netcdf`/`drb:hdf` — the format topic matches the
top-level file node, an ancestor. The core's `_physical_model()` walks
`source_node.parent` up from the variable until an ancestor resolves to a
format topic declaring `drb:physicalTiling`, so this works with the
variable node obtained through the driver's normal navigation — no
special-casing needed in this add-on.

## The two-layer model

`drb-chunk` separates the **physical** chunk model — the source's native
chunking, discovered (never declared), the unit of I/O this add-on
provides — from the **logical** chunk model — the output representation a
product chunk declares (a regular N-D grid today; `burst`, and later
`healpix`, for other formats). A chunk descriptor with no declared logical
geometry (`drb:chunkShape`/`drb:transformer`) simply *adopts* the physical
grid as its logical grid: `chunk.array.transformer` becomes a `RegularGrid`
whose `chunk_shape` is the variable's native chunking. A descriptor that
declares its own grid (e.g. a coarser or finer regular tiling than the
native chunks) keeps that logical grid, but reads are still issued on the
*physical*, native-chunk-aligned boundaries and remapped into the
requested logical window — so adjacent logical tiles that share a native
chunk only pay for that chunk once (coalesced, cached per source node).
`probe()` also accepts a `multiplier` (default `1`), read from a chunk's
`drb:nativeMultiplier`, to request an exact multiple of the native chunk
as the physical unit.

### netCDF-3 / contiguous fallback

Classic netCDF-3 files (and HDF5 variables explicitly stored contiguously)
have no native chunk grid at all: `netCDF4.Variable.chunking()` returns
`None` (or the string `"contiguous"`) rather than a shape tuple. In that
case `probe()` falls back to the **whole variable** as a single physical
chunk (`chunk_shape == array_shape`) — correct, if coarse: any read
touching the variable pulls the whole thing in once, cached for
subsequent windows, rather than failing or guessing a tiling that isn't
actually there.

## Installation

```bash
pip install drb-chunk-netcdf
```

Requires `drb-chunk>=0.6.1,<1`, `netCDF4` and `numpy` (installed as
dependencies), and `drb-driver-netcdf` to resolve `drb:netcdf`/`drb:hdf`
sources.

## Usage

No API of its own to call — once installed alongside `drb-chunk`, any
product add-on's `Chunk.select(...)` over a netCDF/HDF5-backed source picks
up the physical model transparently:

```python
from drb.topics import resolver
from drb.addons.addon import AddonManager
from drb.chunk.selection import WindowSelection

topic, node = resolver.resolve("/path/to/product.nc")
addon = AddonManager().get_addon("chunk")

chunk = addon.apply(node, chunk_name="band_04")   # e.g. a product add-on
window = WindowSelection(x=0, y=0, w=128, h=128)
array = chunk.select(window).get_impl(numpy.ndarray)
```

Reads above are issued on the variable's native chunk boundaries (e.g.
1x64x64), cached per source node, and remapped into the requested window —
not read once per requested tile.

## N-D note

netCDF/HDF5 chunking is natively N-D (`Variable.chunking()`/`.shape` cover
every dimension, not just two spatial axes); the core's `RemapEngine` and
`aligned_window`/coalescing logic are already N-D-general, so this
provider handles variables of any rank as-is — a leading `time`/`band`
axis is just another dimension in the chunk grid, not a special case.

## Limitations

- Read-only probing: this add-on discovers the native chunking, it does
  not write or re-chunk files.
- `drb:source` must resolve/navigate to the variable node itself (the node
  whose `get_impl(netCDF4.Variable)` yields the variable) — the same
  convention `drb-chunk-zarr`/`drb-chunk-image` use for the array/raster.
- The netCDF-3/contiguous fallback treats the whole variable as one
  physical chunk; there is no partial native tiling to discover for those
  formats.

## Reference

See [`docs/dev/writing-a-physical-tiling-provider.md`](../../core/docs/dev/writing-a-physical-tiling-provider.md)
in `drb-chunk` for the `PhysicalTilingProvider` contract; `drb-chunk-image`
is the reference implementation, `drb-chunk-zarr` the second, this add-on
the third.
