micro-reader-codecs

Native image decoders for micro-reader.

micro-reader-codecs holds the Rust decoders that micro-reader uses to read compressed microscopy data. It is a separate package so that micro-reader itself stays pure Python.

You normally don't need to install it yourself. It is installed automatically with:

pip install micro-reader

It has no Python dependencies.

How it works

• Every decoder writes straight into a buffer you provide, such as a NumPy array.
• Decoders run without holding Python's lock, so micro-reader's thread pool can decode on all cores.
• JPEG 2000 and JPEG XL decode on one thread each. micro-reader already decodes many tiles at once, and threads inside the codec would compete for the same cores.

Decoders

Reading large blocks in parts

Some compressed blocks are too large to decode as a whole. These decoders read only part of a block:

• jpeg2k_decode_area_into and jxl_decode_area_into decode only the rectangle you ask for.
• StreamDecoder decodes one compressed block of a file as a stream and keeps only the bytes you ask for. It supports zlib, deflate, LZW, zstd, PackBits, PNG rows and uncompressed data. Each read continues where the last one stopped.
• BlockIndex keeps checkpoints of a deflate block. Several StreamDecoders can share one, so jumping back in the block resumes from the nearest checkpoint instead of the start.

Not included yet

JPEG XR is not included, because the jpegxr crate needs libclang to build. Until then, micro-reader decodes JPEG XR through imagecodecs:

pip install "micro-reader[codecs]"

Building

The Rust toolchain is pinned in rust-toolchain.toml. To build and install into the active virtual environment:

maturin develop --release

Tests

The tests live in the repository's tests/ folder:

• test_codecs.py
• test_czi_codecs.py
• test_tiff_codecs.py
• test_memory_limit.py

They compare these decoders with independent reference decoders, bit for bit.

License

MIT

The bundled crates and C libraries (libzstd, libdeflate, OpenJPEG) are MIT, Apache-2.0, BSD or Zlib licensed. Their notices ship in every wheel as THIRD_PARTY_LICENSES.txt.

Whenever Cargo.lock changes, regenerate it with:

python scripts/third_party_licenses.py