Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

czarr

Status Build Tests Documentation codecov prek zizmor
Meta Hatch project Ruff ty uv Conventional Commits Commitizen friendly License
Package PyPI PyPI

czarr reads and writes Zarr v3 arrays on NVIDIA GPUs. Compression runs through nvCOMP, file I/O runs through cuFile (GPUDirect Storage where the system supports it), and reads return cupy.ndarray.

Documentation: https://czarr.readthedocs.io/

Installation

The NVIDIA wheels are CUDA-version-specific, so the CUDA build is an explicit choice:

pip install "czarr[cu12]"     # CUDA 12
pip install "czarr[cu13]"     # CUDA 13

pip install czarr without an extra installs no CUDA wheels and fails at import. Linux only.

Usage

Through zarr, after one setup call:

import czarr, zarr

czarr.configure_gpu()
arr = zarr.open_array("data.zarr")     # any zstd/lz4/gzip/zlib/blosc store,
out = arr[:]                           # including CPU-written ones; decodes on GPU

Or through the explicit API, which parses metadata once and holds no global state:

from czarr.core import Array

arr = Array.open("data.zarr")
out = arr[0:4, :, 8:24]

Both return cupy.ndarray. Writing goes through zarr (czarr.create_cuda_array or zarr.create_array with czarr codecs). See the documentation for the how-to guides and API reference.

Codec selection

Situation Codec
Default ANS (lowest scratch, scales past 4 GiB)
Small arrays Bitcomp (fastest below 4 GiB; OOMs above on A40)
Interoperable bitstream Zstd, LZ4, Gzip, Zlib (bit-identical with the CPU libraries)
Existing blosc stores decode-only via czarr.Blosc; write new data as [Shuffle, Zstd]

GPU decode pays off from roughly 1 MiB chunks upward; below 256 KiB, multi-threaded CPU Blosc is faster.

Chunk shapes for TCZYX microscopy

Access pattern Chunks
Whole timepoints, arr[t] (1, 1, Z, Y, X)
Z-planes, arr[t, c, z] (1, 1, 1, Y, X)
XY tiles (1, 1, 1, tile_y, tile_x)
Full timeseries, arr[:, c, z] (T, 1, 1, Y, X)

Match the chunk to the access pattern so each read decodes only the chunks it touches. Sharded stores are handled: czarr replaces zarr's sharding_indexed codec with a variant that coalesces partial-shard reads.

Limitations

  • The lowlevel read path covers zstd, blosc, and shuffle. Other codec chains fall back to the zarr pipeline, GPU-decoded where a czarr codec exists.
  • Blosc is decode-only.
  • Real GPUDirect Storage requires the nvidia_fs kernel module; without it cuFile stages through a pinned host bounce. cuFile cannot read tmpfs.
  • cupy is a hard dependency; zarr's GPU buffer prototype is hard-coded to cupy.ndarray.

An experiment in AI-assisted development

As you can probably tell, most of this has been generated with AI tooling, this is just an experiment to see if this even works, and it has helped me resolve the issue of i/o bottlenecks in my own work.

License

MIT.

Metadata

Release files for czarr 0.0.1a0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for czarr 0.0.1a0
File Size Uploaded Provenance Actions
czarr-0.0.1a0.tar.gz 307.2 kB With provenance Details for czarr-0.0.1a0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for czarr 0.0.1a0
File Interpreter ABI Platform Actions
czarr-0.0.1a0-py3-none-any.whl Python 3 none any Details for czarr-0.0.1a0-py3-none-any.whl

Total release size: 411.9 kB

Release files / czarr-0.0.1a0.tar.gz

Download URL czarr-0.0.1a0.tar.gz
Size 307.2 kB
Tags Source
SHA-256 checksum
How to use checksums
b3ef7b9a1f414bd87140e3b955516987e82905096009b43590debc1fe07af2b3
BLAKE2b-256 checksum
How to use checksums
fa3fa419a480b6472965ebc05bc02660eb293f6d6fb14cb5526aadd253571b9e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.13

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jul 7, 2026.

Transparency log

Release files / czarr-0.0.1a0-py3-none-any.whl

Download URL czarr-0.0.1a0-py3-none-any.whl
Size 104.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
f062bd15d75d76c255e3ec9b5dc3febd62028f06f8313e09f27a48a88be9328f
BLAKE2b-256 checksum
How to use checksums
66df3aa00230f2a88591f6a90ee7a5b2aa4b01019a7515eeb7b43f1b8928c8b8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.13

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jul 7, 2026.

Transparency log

Security for czarr 0.0.1a0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for czarr-0.0.1a0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jul 7, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release
Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page