mar-eval 0.3.1

Task-based evaluation toolkit for Metal Artifact Reduction (MAR) in CT imaging (Annex GG Framework)

mar-eval

Toolkit for the Objective Evaluation of Metal Artifact Reduction (MAR) in CT Imaging

---

What is this?

mar-eval is an open-source Python toolkit that implements the analysis framework described in Annex GG of the proposed IEC 60601-2-44 Ed. 4.
It enables objective evaluation of Metal Artifact Reduction (MAR) in CT imaging using the Channelized Hotelling Observer (CHO), AUC-based detectability metrics, and bias assessment between MAR and Filtered-Back Projection (FBP) reconstructions.

The toolkit supports the type test use case — premarket evaluation on representative CT systems or realistic simulation outputs.

---

Example Notebook

A full Annex GG demo is provided at:

notebooks/annex_gg_full_demo.ipynb

This notebook runs the entire workflow end-to-end: from synthetic image generation and ROI extraction to CHO analysis, AUC computation, paired t-tests, and bias evaluation.
It uses detectability curves (AUC vs. Dose, AUC vs. Contrast) rather than heatmaps, matching Vaishnav et al., Med. Phys. 2020.

---

Install

pip install mar-eval
# for notebook demo & plots:
pip install matplotlib jupyterlab pyyaml

Python ≥ 3.9 is supported.
GitHub Actions tests run on 3.9 – 3.12.

---

Quick Start (synthetic)

from mareval import (
    generate_synthetic_study, extract_roi, batch_extract_rois,
    build_pca_channels, cho_template, cho_decision_values,
    compute_auc_ci
)

cfg = dict(image_shape=(64,64), doses=3, contrasts=3, realizations=4, rng_seed=0)
study = generate_synthetic_study(cfg)
images = study["images"]
centers = [(32, 24)]
roi_size = (17,17)

train_imgs = [images[(0,0,0,'FBP','absent')], images[(2,2,0,'MAR','present')]]
train_rois = batch_extract_rois(train_imgs, centers, roi_size)
U = build_pca_channels(train_rois, n_channels=8)

pos = [extract_roi(images[(0,0,r,'FBP','present')], centers[0], roi_size).ravel() for r in range(cfg['realizations'])]
neg = [extract_roi(images[(0,0,r,'FBP','absent')], centers[0], roi_size).ravel() for r in range(cfg['realizations'])]

import numpy as np
pos = np.asarray(pos); neg = np.asarray(neg)
ch_pos = (pos - pos.mean(0)) @ U
ch_neg = (neg - neg.mean(0)) @ U
w = cho_template(ch_pos, ch_neg, lambda_reg=1e-3)

s = np.concatenate([ch_pos @ w, ch_neg @ w])
y = np.array([1]*len(ch_pos) + [0]*len(ch_neg))
res = compute_auc_ci(s, y, n_bootstrap=1000)
print(res)

---

Annex GG “adult chest / spinal rod” example

• Config: configs/adult_chest_spinal_rod.yaml
• Notebook: notebooks/annex_gg_full_demo.ipynb

The notebook executes the full flow:

1. Load config and generate synthetic images approximating an adult chest with a titanium spinal rod and an adjacent low-contrast lesion.
2. Preview example slices and ROI placement.
3. ROI extraction and channel learning (PCA).
4. CHO template and AUC computation for each (dose, contrast) cell, both FBP and MAR.
5. Paired one-tailed t-test and ΔAUC bias assessment.
6. Detectability curves for AUC vs Dose and AUC vs Contrast.
7. CSV output tables in outputs/.

The synthetic generator produces realistic streak-like artifacts and dose-dependent noise; MAR reduces streak amplitude and adds mild smoothing.

---

API Highlights

from mareval import (
  build_pca_channels, cho_template, cho_decision_values,
  compute_auc, compute_auc_ci, paired_ttest_one_tailed, delta_auc_bias_assessment,
  generate_synthetic_study, extract_roi, batch_extract_rois,
  make_parameter_grid, save_auc_table_csv, save_delta_auc_table_csv
)

• CHO
  • build_pca_channels(rois, n_channels=16, whiten=True)
  • cho_template(ch_pos, ch_neg, lambda_reg=1e-3)
  • cho_decision_values(ch_samples, w)
• AUC & Stats
  • compute_auc(values, labels)
  • compute_auc_ci(values, labels, n_bootstrap=2000)
  • paired_ttest_one_tailed(a, b)
  • delta_auc_bias_assessment(auc_fb, auc_mar)
• Simulator flow
  • generate_synthetic_study(cfg_dict)
  • extract_roi(img, center, size) / batch_extract_rois([...], centers, size)
  • make_parameter_grid(...)
  • save_auc_table_csv(...), save_delta_auc_table_csv(...)

---

Citation

If you use mar-eval in your research, please cite:

C.D. Cocchiaraley, Annex GG — Objective evaluation of Metal Artifact Reduction algorithms in CT imaging, Proposed addition to IEC 60601-2-44 Ed. 4 (2025).

---

Versioning

• v0.3.0 introduces the simulator workflow, unified Annex GG notebook, and detectability-curve outputs.
• Version tags align releases with document snapshots (e.g., Annex GG draft refs).

---

License

MIT