Medical Imaging Segmentation Toolkit

MIST is a simple, scalable, end-to-end framework for 3D medical image segmentation. It handles everything from raw NIfTI files to trained models and evaluated predictions, with sensible defaults that work well out of the box and a configuration file for when you need more control.

[!TIP] Try MIST in your browser — no setup required. The end-to-end Colab demo runs the whole pipeline — analyze → preprocess → train → evaluate → predict → postprocess → rank → visualize — on a heart MRI dataset using a free Colab T4 GPU.

Installation

Training (NVIDIA GPU required):

pip install "mist-medical[train]"

Inference only (CPU-compatible):

pip install mist-medical

Development:

git clone https://github.com/mist-medical/MIST.git
cd MIST
pip install -e ".[train]"    # full
pip install -e .             # inference only

Docker (NVIDIA GPU required, driver ≥ 525.x):

docker pull mistmedical/mist:latest

Quick Start

1. Prepare a dataset JSON describing your data:

{
    "task":        "brats2023",
    "modality":    "mr",
    "train-data":  "/full/path/to/raw/data/train",
    "test-data":   "/full/path/to/raw/data/validation",
    "mask":        ["seg.nii.gz"],
    "images":      {"t1": ["t1n.nii.gz"],
                    "t2": ["t2w.nii.gz"],
                    "tc": ["t1c.nii.gz"],
                    "fl": ["t2f.nii.gz"]},
    "labels":      [0, 1, 2, 3],
    "final_classes": {"WT": [1, 2, 3],
                      "TC": [1, 3],
                      "ET": [3]}
}

2. Run the full pipeline (analyze → preprocess → train → evaluate):

mist_run_all --data dataset.json \
             --numpy /path/to/numpy \
             --results /path/to/results

3. Run inference on new data:

mist_predict --models-dir /path/to/results/models \
             --config /path/to/results/config.json \
             --paths-csv /path/to/test.csv \
             --output /path/to/predictions

Key Features

• Automatic configuration — analysis step determines target spacing, patch size, normalization, and foreground cropping from your data
• Five-fold cross-validation by default, with custom fold assignment support
• Multi-GPU training via PyTorch DDP; uses all visible GPUs automatically
• GPU-accelerated data loading via NVIDIA DALI during training
• Sliding window inference with configurable overlap and patch blending
• Test-time augmentation and multi-model ensembling at inference
• Postprocessing with learnable, per-class morphological strategies
• CPU inference — mist_predict runs on any machine, including Macs

Supported Architectures

Change the architecture in config.json or pass --model <key> at the command line.

Supported Loss Functions

Composite losses blend a region-based term with a boundary/distance term, weighted by a scheduled alpha. Schedules: constant, linear, cosine.

Pipeline Commands

Documentation

Full documentation, including configuration reference and advanced topics, is at mist-medical.readthedocs.io.

What's New

• July 2026 — Interactive Colab demo — run the full MIST pipeline end-to-end on a free Colab GPU with the heart segmentation notebook: analysis, training, evaluation, inference, postprocessing, and BraTS-style ranking, all in the browser with no local setup.
• June 2026 — Official Docker image — mistmedical/mist:latest ships a CUDA 12.8 build. Requires NVIDIA driver ≥ 525.x.
• June 2026 — 2.0.1 release candidate — BF16 automatic mixed precision replaces FP16 throughout training and inference, reducing memory use and eliminating gradient loss scaling. Sliding-window inference gains a tunable sw_batch_size parameter. mist_rank adds pairwise Wilcoxon significance testing via --significance-csv. Several targeted memory-reduction fixes land across the inference stack.
• June 2026 — Multi-model ensembling — mist_ensemble combines discrete NIfTI predictions from two or more separately trained models into a single consensus segmentation via STAPLE (--ensemble-backend staple, default) or majority vote (--ensemble-backend majority_vote). Works for single-class and multi-class label maps.
• May 2026 — 2.0.0 release candidate — BraTS-style multi-strategy ranking (mist_rank), a structured postprocessing transform registry with LLM-readable metadata (describe_transforms), and full pathlib + PEP 585/604 modernization across the codebase.
• April 2026 — CPU inference support — mist_predict now runs on any machine, including Macs and laptops without an NVIDIA GPU. Install with pip install mist-medical (no GPU required).
• March 2026 — Resume training — interrupted runs can be continued from the last checkpoint with --resume, with atomic checkpointing to prevent corruption.
• March 2026 — GPU-aware automatic patch size — the analysis step now derives the patch size from available GPU memory, so the default configuration is hardware-appropriate without manual tuning.
• March 2026 — Transfer learning — initialize encoders from pretrained weights with --pretrained-weights, and average model weights across folds with mist_average_weights.
• March 2026 — Better training defaults — AdamW optimizer and gradient clipping are now the defaults, with the clipping threshold exposed via grad_clip_norm in config.json.
• September 2025 — BraTS 2025 adult glioma challenge @ MICCAI 2025 — MIST takes 3rd place (repeat).
• November 2024 — MedNeXt models — small, base, medium, and large variants added (mednext-small, mednext-base, mednext-medium, mednext-large).
• October 2024 — BraTS 2024 adult glioma challenge @ MICCAI 2024 — MIST takes 3rd place.

Citation

If you use MIST in your work, please cite:

@article{celaya2024mist,
  title   = {MIST: A Simple and Scalable End-To-End 3D Medical Imaging Segmentation Framework},
  author  = {Celaya, Adrian and others},
  journal = {arXiv preprint arXiv:2407.21343},
  year    = {2024}
}

@article{celaya2022pocketnet,
  title   = {PocketNet: A Smaller Neural Network For Medical Image Analysis},
  author  = {Celaya, Adrian and others},
  journal = {IEEE Transactions on Medical Imaging},
  doi     = {10.1109/TMI.2022.3224873},
  year    = {2022}
}