drb-chunk-zarr 0.1.0

Physical chunk model for the drb:zarr class

drb-chunk-zarr

Zarr-backed physical chunk model for the drb:zarr class, built on top of the drb-chunk add-on.

What it is

drb-chunk-zarr teaches drb-chunk how to read a Zarr array on its native chunk boundaries instead of on whatever grid a product descriptor happens to declare. It targets the generic drb:zarr topic class, not a single product family — a format chunk add-on, sibling to drb-chunk-image (GDAL rasters).

Concretely, it registers a PhysicalTilingProvider (zarr-chunks) that probes an array's native .chunks layout via zarr, without loading the full array.

How it self-activates

Installing the package is enough; there is no product-side wiring:

• It registers ZarrChunksProvider under the drb.chunk.physical entry-point group (drb.addons.chunk.zarr.provider:ZarrChunksProvider), the seam the drb-chunk core uses to discover physical-tiling providers lazily, and a matching zarr-array physical reader via register_physical_reader.
• It ships a cortex.ttl carrying a single triple — <http://www.gael.fr/drb#zarr> drb:physicalTiling "zarr-chunks" — declared under the drb.topic entry-point group. The core's ManagerDao merges every installed drb.topic descriptor into one RDF graph, so this triple lands on the existing drb:zarr topic without touching the driver or any other package.

Once both are installed, any chunk descriptor whose drb:source resolves to a Zarr array node automatically gets a physical chunk model: the core resolves the source's format topic, reads drb:physicalTiling, looks up zarr-chunks, and calls probe().

The two-layer model

drb-chunk separates the physical chunk model — the source's native chunking, discovered (never declared), the unit of I/O this add-on provides — from the logical chunk model — the output representation a product chunk declares (a regular N-D grid today; burst, and later healpix, for other formats). A chunk descriptor with no declared logical geometry (drb:chunkShape/drb:transformer) simply adopts the physical grid as its logical grid: chunk.array.transformer becomes a RegularGrid whose chunk_shape is the array's native .chunks. A descriptor that declares its own grid (e.g. a coarser or finer regular tiling than the native chunks) keeps that logical grid, but reads are still issued on the physical, native-chunk-aligned boundaries and remapped into the requested logical window — so adjacent logical tiles that share a native chunk only pay for that chunk once (coalesced, cached per source node). probe() also accepts a multiplier (default 1), read from a chunk's drb:nativeMultiplier, to request an exact multiple of the native chunk as the physical unit.

Installation

pip install drb-chunk-zarr

Requires drb-chunk>=0.6.0,<1, zarr and numpy (installed as dependencies), and drb-driver-zarr to resolve drb:zarr sources.

Usage

No API of its own to call — once installed alongside drb-chunk, any product add-on's Chunk.select(...) over a Zarr-backed source picks up the physical model transparently:

from drb.topics import resolver
from drb.addons.addon import AddonManager
from drb.chunk.selection import WindowSelection

topic, node = resolver.resolve("/path/to/product.zarr")
addon = AddonManager().get_addon("chunk")

chunk = addon.apply(node, chunk_name="band_04")   # e.g. a product add-on
window = WindowSelection(x=0, y=0, w=512, h=512)
array = chunk.select(window).get_impl(numpy.ndarray)

Reads above are issued on the array's native chunk boundaries (e.g. 256×256), cached per source node, and remapped into the requested 512×512 window — not read once per 512×512 tile.

N-D note

Zarr chunking is natively N-D (arr.chunks/arr.shape cover every dimension, not just two spatial axes); the core's RemapEngine and aligned_window/coalescing logic are already N-D-general, so this provider handles arrays of any rank as-is — a leading band/time axis is just another dimension in the chunk grid, not a special case.

Limitations

• Read-only probing: this add-on discovers the native chunking, it does not write or re-chunk stores.
• drb:source must resolve/navigate to the array node itself (the node whose get_impl(zarr.core.Array) yields the array) — the same convention drb-chunk-image uses for the raster file.

Reference

See docs/dev/writing-a-physical-tiling-provider.md in drb-chunk for the PhysicalTilingProvider contract; drb-chunk-image is the reference implementation, this add-on the second.