Metadata-Version: 2.4
Name: mricompression
Version: 0.1.0
Summary: Near-lossless compression of fastMRI raw k-space: GGD-CDF 16-bit quantizer + JPEG-LS plane container, with RSS reconstruction
Author-email: Dan Jacobellis <danjacobellis@utexas.edu>
License-Expression: LicenseRef-Proprietary
Project-URL: Homepage, https://danjacobellis.net
Project-URL: Repository, https://github.com/danjacobellis/fastmri
Project-URL: Issues, https://github.com/danjacobellis/fastmri/issues
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy
Requires-Dist: scipy
Requires-Dist: pillow
Requires-Dist: pillow-jpls
Requires-Dist: h5py
Dynamic: license-file

# fastmri

Near-lossless compression of NYU fastMRI raw k-space: a validated 16-bit GGD-CDF companding quantizer followed by lossless JPEG-LS, packaged as `mricompression`, plus the tooling to download the fastMRI collections and the Hugging Face dataset of compressed k-space built from them for training neural codecs.

- `src/mricompression/` — the installable package (below); `mc1.py` — the driver that drew and encoded the first dataset (round `mc1`, `runs/mricompression.md`).
- `NEAR_LOSSLESS.md` — what "near-lossless" means here and how it is measured.
- `runs/universal_quant.md` — how the quantizer was fitted (round `uq1`); `runs/final_validation.md` — the held-out validation that froze it (round `fv1`); `runs/ggdlpc_rate.md` and `runs/ggdlpc_layout.md` — the entropy-coder study that chose JPEG-LS over GGDLPC.
- `download.py --list` — the available archives; data is stored outside the repo (see `fastmri_paths.py`).
- `experiments/` — the research scripts and notebooks behind the reports in `runs/`, kept as a historical record (they ran from the repo root).

## The `mricompression` package

```
python3 -m build --wheel && pip install dist/mricompression-*.whl
```

```python
import h5py
from mricompression.fastmri import to_6d, from_6d   # h5 kspace <-> (reps, avgs, slices, coils, H, W)
from mricompression.codec import encode, decode     # 6-D complex64 <-> container bytes
from mricompression.recon import rss                # 6-D k-space -> (reps, slices, Hc, Wc) images

k = h5py.File('file1000101.h5')['kspace'][:]        # knee/brain (S, C, H, W); prostate T2 (3, S, C, H, W); diffusion (50, S, C, H, W)
data = encode(to_6d(k))                             # uint16 codes -> one lossless JPEG-LS image per (re/im, rep, avg, slice, coil) plane
k6 = decode(data)                                   # complex64, within the validated +-1 gray level of k
images = rss(k6, crop=320)                          # zero-filled root-sum-of-squares reconstruction
```

The 6-D layout inserts size-1 axes: knee/brain multicoil have reps = avgs = 1, prostate T2's leading axis is the 3 signal averages, prostate diffusion's the 50 diffusion repetitions (told apart by shape, or with `to_6d(k, kind=...)`). The container is self-describing (magic, version, the six shape integers, then length-prefixed JPEG-LS streams in C order over (channel, rep, avg, slice, coil)); `codec.decode_codes` returns the uint16 codes bit-exactly, `codec.header` the shape without decoding. `mricompression.quant` is the frozen quantizer copied unmodified (the LUT ships as package data and is hash-checked by `python -m mricompression.selftest <file.h5>`). `recon.rss` reproduces the knee/brain `reconstruction_rss` to float precision (max relative error ~2e-7); the prostate release reconstructions add GRAPPA and regridding and are not reproduced.

Measured rates (bits per uint16 sample, JPEG-LS on the frozen codes): knee MC ~11.3-11.7, brain MC ~9-13 by sequence, prostate T2 ~11.8, prostate diffusion ~12.2 — about 1.35x below the 16-bit codes and 2.7x below complex64.

## Quantizer

`ggd_quant.py` + `ggd_quant_lut.npy` (frozen 2026-08-31, round `fv1`). The map sends each real and imaginary k-space sample through the CDF of a zero-mean generalized Gaussian (ALPHA = 7.724098728030807e-09 — the 40-volume Sharifi–Leon-Garcia fit scaled by A_FACTOR = 1385 — BETA = 0.162663616159092), truncated to [-0.7, 0.7] and renormalized, then to a 16-bit code; codes decode through a published 65,536-entry table of fit-set centroids. The truncation limit 0.7 is ~10x the largest magnitude observed in a 4,782-volume census, so no sample of any evaluated collection clips. Code 32768 decodes to exactly 0.0, so the huge population of structural zeros — and with it the sampling mask — round-trips exactly.

Contract: `quantize_kspace(k)` maps a complex64 array of any shape to uint16 with a leading (real, imag) axis; `dequantize_kspace(q)` inverts it (`quantize`/`dequantize` are the scalar float↔uint16 pair). Elementwise and identical for every sample of every volume: no side information, no per-volume state, 32 bits per complex sample before entropy coding.

Validation (round `fv1`, `runs/final_validation.md`): 564 held-out volumes — 200 knee multicoil, 200 brain multicoil, 104 prostate T2, 60 prostate diffusion, drawn disjoint from every fit/dev/eval volume of the design rounds — reconstructed through each collection's verified reference pipeline. Every volume is within ±1 displayed 8-bit gray level under all four display normalizations (`volmax`, `slicemax`, `p995`, `dark`); the acceptance gates (≤3 levels; ≤1 % of volumes beyond) pass with zero offending volumes. Image PSNR ≥ 102.35 dB on every volume; 0 clipped samples.

Exclusions: the breast collection is excluded from the analysis — its release GRASP/ADMM reconstruction is non-deterministic at the 3–24 gray-level scale, so no displayed-image criterion is measurable through it (`runs/breast_pipeline.md`); breast k-space SNR under this quantizer was 75.24/72.57 dB (mean/min, 20 volumes, 0 clipped). Knee singlecoil is dropped from the planned dataset as redundant: it is NYU's emulated single-coil reduction of the same scans as knee multicoil, and its shared scan IDs would force joint patient-disjoint splitting.
