jix 0.0.1

Multi-dimensional array library with block-compressed, lazy-evaluated storage

jix (Python)

Python bindings for the jix Rust library. Built with PyO3.

Block-based compression. An array is split into a grid of fixed-size nd-blocks, each compressed independently. Only the blocks that overlap a read request are decompressed, so random access into large arrays is cheap.

Lazy operation chains. Every operation - arithmetic, shape change, reduction, type cast - returns a new view that wraps the input(s) and records the transformation, nothing is computed until data is explicitly requested. The full pipeline runs in a single decompression pass the moment you ask for output.

import jix
import numpy as np

# Compress a NumPy array into block-compressed storage.
a = jix.compact(np.arange(1_000_000, dtype=np.float32).reshape(1000, 1000))

# Build a lazy pipeline - no decompression happens yet.
result = (a - a.mean(axis=0)) / a.std(axis=0)

# Materialize the pipeline into a NumPy array.
out = result.numpy()

# Or write straight to disk - blocks are decompressed, transformed,
# and re-compressed one at a time without materializing the full result.
result.write_to("normalized.jix")

# Load back; use mmap=True for zero-copy access to large files.
b = jix.read_array("normalized.jix", mmap=True)
print(b.shape, b.dtype)   # (1000, 1000) float32