zarr-indexing

Composable, lazy coordinate transforms for Zarr array indexing.

Documentation: https://zarr-indexing.readthedocs.io/

This package implements TensorStore-inspired index transforms. The core idea: each supported indexing operation (slicing, fancy indexing, etc.) produces a coordinate mapping from user space to storage space. These mappings compose without reading selected source values. LazyArray materializes them on request; the transform algebra itself performs no source I/O.

Key types:

• LazyArray — wraps a system-memory/basic-indexing source with lazy indexing: LazyArray.from_numpy(numpy_array)[10:50, ::2].oindex[[3, 1, 1], :]. The key's type picks the frame: NumPy keys are positions relative to the view, while an IndexDomain or IndexTransform key addresses the view's absolute domain, which every view keeps. composes a transform and returns a new view without reading data, and result() materializes it into owned system memory. LazyArray(source) uses the basic reader; from_numpy selects numpy_reader, which currently uses the same slab-and-gather implementation. Device sources that refuse NumPy conversion need a custom reader to transfer values into the output buffer.
• Reader — the explicit backend execution boundary: transforms say which values belong in the result, while readers say how a backend obtains them
• IndexDomain — a rectangular region of integer coordinates
• IndexTransform — maps input coordinates to storage coordinates
• ChunkPlan and ChunkProjection — lazily partition a selection over a caller-selected grid and pair each chunk-local transform with its placement in the request, without binding a storage backend or scheduler
• GridPartition — the plan's factored, columnar form: one table per axis (StridedSet, IndexedSet) plus joint_sets for connected index-array components, from which projections are derived on demand
• ConstantMap, DimensionMap, ArrayMap — the three ways a single output dimension can depend on the input
• compose — chain two transforms into one

The base package depends on NumPy and the standard library; its optional testing module also requires Hypothesis. The package does not import zarr. It is developed in the zarr-python repository, and its examples include reading Zarr arrays through Dask. Installing it does not replace Zarr's indexing implementation.

Installation

pip install zarr-indexing

Examples

• Lazy indexing a NumPy array
• Lazy indexing with Dask

Contributing

Development commands, the test suite and the docs build are described in CONTRIBUTING.md in the repository. Issues and pull requests go to zarr-developers/zarr-python.