This release is a pre-release and may not be stable for production use.
czarr
czarr reads and writes Zarr v3 arrays on NVIDIA GPUs. Compression runs through nvCOMP, file I/O runs through cuFile (GPUDirect Storage where the system supports it), and reads return cupy.ndarray.
Documentation: https://czarr.readthedocs.io/
Installation
The NVIDIA wheels are CUDA-version-specific, so the CUDA build is an explicit choice:
pip install "czarr[cu12]" # CUDA 12
pip install "czarr[cu13]" # CUDA 13
pip install czarr without an extra installs no CUDA wheels and fails at import. Linux only.
Usage
Through zarr, after one setup call:
import czarr, zarr
czarr.configure_gpu()
arr = zarr.open_array("data.zarr") # any zstd/lz4/gzip/zlib/blosc store,
out = arr[:] # including CPU-written ones; decodes on GPU
Or through the explicit API, which parses metadata once and holds no global state:
from czarr.core import Array
arr = Array.open("data.zarr")
out = arr[0:4, :, 8:24]
Both return cupy.ndarray. Writing goes through zarr (czarr.create_cuda_array or zarr.create_array with czarr codecs). See the documentation for the how-to guides and API reference.
Codec selection
| Situation | Codec |
|---|---|
| Default | ANS (lowest scratch, scales past 4 GiB) |
| Small arrays | Bitcomp (fastest below 4 GiB; OOMs above on A40) |
| Interoperable bitstream | Zstd, LZ4, Gzip, Zlib (bit-identical with the CPU libraries) |
| Existing blosc stores | decode-only via czarr.Blosc; write new data as [Shuffle, Zstd] |
GPU decode pays off from roughly 1 MiB chunks upward; below 256 KiB, multi-threaded CPU Blosc is faster.
Chunk shapes for TCZYX microscopy
| Access pattern | Chunks |
|---|---|
Whole timepoints, arr[t] |
(1, 1, Z, Y, X) |
Z-planes, arr[t, c, z] |
(1, 1, 1, Y, X) |
| XY tiles | (1, 1, 1, tile_y, tile_x) |
Full timeseries, arr[:, c, z] |
(T, 1, 1, Y, X) |
Match the chunk to the access pattern so each read decodes only the chunks it touches. Sharded stores are handled: czarr replaces zarr's sharding_indexed codec with a variant that coalesces partial-shard reads.
Limitations
- The lowlevel read path covers zstd, blosc, and shuffle. Other codec chains fall back to the zarr pipeline, GPU-decoded where a czarr codec exists.
Bloscis decode-only.- Real GPUDirect Storage requires the
nvidia_fskernel module; without it cuFile stages through a pinned host bounce. cuFile cannot read tmpfs. cupyis a hard dependency; zarr's GPU buffer prototype is hard-coded tocupy.ndarray.
An experiment in AI-assisted development
As you can probably tell, most of this has been generated with AI tooling, this is just an experiment to see if this even works, and it has helped me resolve the issue of i/o bottlenecks in my own work.
License
MIT.
Metadata
Release files for czarr 0.0.1a0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| czarr-0.0.1a0.tar.gz | 307.2 kB | With provenance | Details for czarr-0.0.1a0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| czarr-0.0.1a0-py3-none-any.whl | Python 3 | none | any | Details for czarr-0.0.1a0-py3-none-any.whl |
Total release size: 411.9 kB
Release files / czarr-0.0.1a0.tar.gz
| Download URL | czarr-0.0.1a0.tar.gz |
|---|---|
| Size | 307.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
b3ef7b9a1f414bd87140e3b955516987e82905096009b43590debc1fe07af2b3
|
|
BLAKE2b-256 checksum How to use checksums |
fa3fa419a480b6472965ebc05bc02660eb293f6d6fb14cb5526aadd253571b9e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Jul 7, 2026.
Transparency logRelease files / czarr-0.0.1a0-py3-none-any.whl
| Download URL | czarr-0.0.1a0-py3-none-any.whl |
|---|---|
| Size | 104.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
f062bd15d75d76c255e3ec9b5dc3febd62028f06f8313e09f27a48a88be9328f
|
|
BLAKE2b-256 checksum How to use checksums |
66df3aa00230f2a88591f6a90ee7a5b2aa4b01019a7515eeb7b43f1b8928c8b8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Jul 7, 2026.
Transparency logSecurity for czarr 0.0.1a0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for czarr-0.0.1a0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Jul 7, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.