CFT Zarr Custom Codecs

Custom codecs for Zarr v3 optimized for CFT (Cryo-Fluorescence Tomography) data storage.

Codecs

This package provides custom Zarr v3 codecs:

• cft_zarr.jpeg_compressor: JPEG compressor for RGB images (BytesBytesCodec - supports incremental updates)
• cft_zarr.shift12jls_compressor: JPEG-LS compressor for 12-bit fluorescent data (BytesBytesCodec - supports incremental updates)
• cft_zarr.jpeg: JPEG codec for RGB images (ArrayBytesCodec)
• cft_zarr.shift12jls: JPEG-LS codec for 12-bit fluorescent data (ArrayBytesCodec)
• cft_zarr.jls16: JPEG-LS codec for uint16 data without shifting; can also store scaled uint32 stack sums
• cft_zarr.jpegxl: JPEG XL codec (ArrayBytesCodec)

Installation

pip install cft-zarr

Usage

Reading Zarr Files

Important: You must import cft_zarr before opening Zarr files that use these codecs. This registers the codecs with Zarr.

import cft_zarr  # This registers the codecs
import zarr

# Now you can open Zarr files that use custom codecs
arr = zarr.open('rgb.zarr', mode='r')

Using with napari

When opening Zarr files in napari, cft_zarr can be auto-registered via the napari plugin system (installed in the same environment as napari). If you still have trouble, import cft_zarr first:

import cft_zarr  # Register codecs before opening files
import napari

# Now napari can read Zarr files with custom codecs
viewer = napari.Viewer()
viewer.open('path/to/file.zarr')  # Will work with custom codecs

Or in a Python script before launching napari:

import cft_zarr  # Must import before opening Zarr files
import napari

viewer = napari.Viewer()
viewer.open('rgb.zarr')
napari.run()

Creating Zarr Arrays with Custom Codecs

import cft_zarr
from cft_zarr import JPEGCompressor, JLS16Codec, Shift12JLSCompressor
import zarr

# Create RGB array with JPEG compression
rgb_array = zarr.create(
    shape=(100, 512, 512, 3),
    chunks=(4, 512, 512, 3),
    dtype='uint8',
    compressors=[JPEGCompressor(level=85)]
)

# Create fluorescent array with Shift12JLS compression
fl_array = zarr.create(
    shape=(100, 512, 512),
    chunks=(4, 512, 512),
    dtype='uint16',
    compressors=[Shift12JLSCompressor()]
)

# Create a uint16 fluorescent array with direct JPEG-LS compression
jls16_array = zarr.create(
    shape=(100, 512, 512),
    chunks=(4, 512, 512),
    dtype='uint16',
    serializer=JLS16Codec(),
    compressors=[]
)

# Store 32x 12-bit stack sums as uint32 with scale=2.
# The encoded stream stores round(value / 2) as uint16 JPEG-LS and decodes
# back to value * 2, giving at most +/-1 count error from the scaling step.
stack32_array = zarr.create(
    shape=(100, 512, 512),
    chunks=(4, 512, 512),
    dtype='uint32',
    serializer=JLS16Codec(scale=2),
    compressors=[]
)

Decode concurrency

The cft_zarr.jpeg and cft_zarr.shift12jls array codecs decode chunks on a shared worker pool instead of blocking Zarr's IO event loop. The pool starts threads on demand, with one worker per available logical CPU, shared across RGB and fluorescence arrays. Python 3.13+ uses the process-aware CPU count; Python 3.11–3.12 uses the system CPU count. If the count is unavailable, the pool uses one worker. No extra dependency or configuration is required. Codec metadata, pixel values and encoding are unchanged.

Each worker still decodes a complete physical chunk. Decoded-chunk caching in the viewer remains important for adjacent slices. Cancelling a read can cancel queued work; a native decode already running finishes before its worker is reused. Active native calls cannot be interrupted. The concurrency cap bounds simultaneous decodes, not total dataset or viewer cache memory.