fastmri

Near-lossless compression of NYU fastMRI raw k-space: a validated 16-bit GGD-CDF companding quantizer followed by lossless JPEG-LS, packaged as mricompression, plus the tooling to download the fastMRI collections and the Hugging Face dataset of compressed k-space built from them for training neural codecs.

• src/mricompression/ — the installable package (below); mc1.py — the driver that drew and encoded the first dataset (round mc1, runs/mricompression.md).
• NEAR_LOSSLESS.md — what "near-lossless" means here and how it is measured.
• runs/universal_quant.md — how the quantizer was fitted (round uq1); runs/final_validation.md — the held-out validation that froze it (round fv1); runs/ggdlpc_rate.md and runs/ggdlpc_layout.md — the entropy-coder study that chose JPEG-LS over GGDLPC.
• download.py --list — the available archives; data is stored outside the repo (see fastmri_paths.py).
• experiments/ — the research scripts and notebooks behind the reports in runs/, kept as a historical record (they ran from the repo root).

The mricompression package

python3 -m build --wheel && pip install dist/mricompression-*.whl

import h5py
from mricompression.fastmri import to_6d, from_6d   # h5 kspace <-> (reps, avgs, slices, coils, H, W)
from mricompression.codec import encode, decode     # 6-D complex64 <-> container bytes
from mricompression.recon import rss                # 6-D k-space -> (reps, slices, Hc, Wc) images

k = h5py.File('file1000101.h5')['kspace'][:]        # knee/brain (S, C, H, W); prostate T2 (3, S, C, H, W); diffusion (50, S, C, H, W)
data = encode(to_6d(k))                             # uint16 codes -> one lossless JPEG-LS image per (re/im, rep, avg, slice, coil) plane
k6 = decode(data)                                   # complex64, within the validated +-1 gray level of k
images = rss(k6, crop=320)                          # zero-filled root-sum-of-squares reconstruction

The 6-D layout inserts size-1 axes: knee/brain multicoil have reps = avgs = 1, prostate T2's leading axis is the 3 signal averages, prostate diffusion's the 50 diffusion repetitions (told apart by shape, or with to_6d(k, kind=...)). The container is self-describing (magic, version, the six shape integers, then length-prefixed JPEG-LS streams in C order over (channel, rep, avg, slice, coil)); codec.decode_codes returns the uint16 codes bit-exactly, codec.header the shape without decoding. mricompression.quant is the frozen quantizer copied unmodified (the LUT ships as package data and is hash-checked by python -m mricompression.selftest <file.h5>). recon.rss reproduces the knee/brain reconstruction_rss to float precision (max relative error ~2e-7); the prostate release reconstructions add GRAPPA and regridding and are not reproduced.

Measured rates (bits per uint16 sample, JPEG-LS on the frozen codes): knee MC ~11.3-11.7, brain MC ~9-13 by sequence, prostate T2 ~11.8, prostate diffusion ~12.2 — about 1.35x below the 16-bit codes and 2.7x below complex64.

Quantizer

ggd_quant.py + ggd_quant_lut.npy (frozen 2026-08-31, round fv1). The map sends each real and imaginary k-space sample through the CDF of a zero-mean generalized Gaussian (ALPHA = 7.724098728030807e-09 — the 40-volume Sharifi–Leon-Garcia fit scaled by A_FACTOR = 1385 — BETA = 0.162663616159092), truncated to [-0.7, 0.7] and renormalized, then to a 16-bit code; codes decode through a published 65,536-entry table of fit-set centroids. The truncation limit 0.7 is ~10x the largest magnitude observed in a 4,782-volume census, so no sample of any evaluated collection clips. Code 32768 decodes to exactly 0.0, so the huge population of structural zeros — and with it the sampling mask — round-trips exactly.

Contract: quantize_kspace(k) maps a complex64 array of any shape to uint16 with a leading (real, imag) axis; dequantize_kspace(q) inverts it (quantize/dequantize are the scalar float↔uint16 pair). Elementwise and identical for every sample of every volume: no side information, no per-volume state, 32 bits per complex sample before entropy coding.

Validation (round fv1, runs/final_validation.md): 564 held-out volumes — 200 knee multicoil, 200 brain multicoil, 104 prostate T2, 60 prostate diffusion, drawn disjoint from every fit/dev/eval volume of the design rounds — reconstructed through each collection's verified reference pipeline. Every volume is within ±1 displayed 8-bit gray level under all four display normalizations (volmax, slicemax, p995, dark); the acceptance gates (≤3 levels; ≤1 % of volumes beyond) pass with zero offending volumes. Image PSNR ≥ 102.35 dB on every volume; 0 clipped samples.

Exclusions: the breast collection is excluded from the analysis — its release GRASP/ADMM reconstruction is non-deterministic at the 3–24 gray-level scale, so no displayed-image criterion is measurable through it (runs/breast_pipeline.md); breast k-space SNR under this quantizer was 75.24/72.57 dB (mean/min, 20 volumes, 0 clipped). Knee singlecoil is dropped from the planned dataset as redundant: it is NYU's emulated single-coil reduction of the same scans as knee multicoil, and its shared scan IDs would force joint patient-disjoint splitting.