MONAI Physio

A collection of methods, workflows, tutorials, and CLI tools for creating personalized physiological digital twins.

MONAI Physio typically begins with a 3D medical image of a subject, extracts anatomic models from that image, and then uses AI surrogates to estimate the subject's physiological processes - initially focusing on cardiac and respiratory motion, and expanding to electrophysiology, blood flow, and organ perfusion. The package provides methods for forming these physiological AI surrogates and for finetuning the segmentation and registration AI methods that power them, with special emphasis on statistical shape models: they capture subject-specific characteristics that help determine subject-specific physiological function, and establish correspondence across subjects to aid AI surrogate generalization and simplify the application of traditional solvers.

MONAI Physio is not validated for clinical use. It is a research and visualization toolkit, not a medical device, and must not be used for diagnosis, treatment planning, or clinical decision-making.

Documentation

https://project-monai.github.io/monai-physio/ is the primary entry point for users and contributors. Key sections:

• Installation and Quickstart
• Tutorials - runnable end-to-end workflows and their datasets. Most numbered tutorials ship multiple organ/dataset variant scripts (heart, lung, duke_heart) that share the same workflow class, so you can see how to adapt a workflow to your own anatomy and data
• CLI & Scripts Guide - command-line tools for conversion, segmentation, registration, and USD workflows
• API Reference - workflow, registration, segmentation, and USD classes
• Developer Guides - architecture, extension points, and implementation conventions
• Contributing and Testing
• Roadmap - proposed priorities and bounded ways to contribute
• FAQ and Troubleshooting
• Issues and Discussions

Highlights

• Personalized digital twins: build subject-specific anatomic models and physiological AI surrogates from 3D/4D medical images
• Statistical shape models: capture subject-specific anatomy and establish cross-subject correspondence, aiding AI surrogate generalization and simplifying traditional solver setup
• Simplified workflows on industry-leading open-source tools: ICON and Greedy for registration; MONAI with TotalSegmentator and Simpleware for segmentation; scikit-learn for statistical shape modeling; ITK for image processing; PyVista and OpenUSD/Omniverse for geometry manipulation; CuPy for accelerated computing; and PhysicsNeMo for AI surrogates
• Extensible class hierarchy: add new segmentation and registration methods, and extend to new data types, organs, and physiological processes, without reworking the workflow layer
• Physiological motion: cardiac and respiratory motion today, expanding to electrophysiology, blood flow, and organ perfusion
• NVIDIA Omniverse as the simulation hub: the end goal for simulation - a simulation-information hub and gateway to other engines (e.g., Ansys solvers), interactive simulations for treatment planning (e.g., Isaac Sim, Newton), visualization systems (e.g., AR/VR devices), and physical systems (e.g., robots via ROS)
• CLI and Python API: installed command-line tools and workflow classes for repeatable, scriptable pipelines

Quick Start

Install

uv pip install "monai-physio[cuda12]" --torch-backend=cu128 -f https://data.pyg.org/whl/torch-2.11.0+cu128.html

Run it inside a virtual environment that has uv (pip install uv). It installs entirely from prebuilt wheels and needs an NVIDIA GPU with a CUDA 12.8 or newer driver. --torch-backend=cu128 selects the CUDA 12.8 PyTorch build, and -f points uv at the PyG wheel index that hosts torch-scatter; both are required. See the installation guide for CUDA 13, pip, and source installs.

Download Tutorials

The tutorials are not installed by pip. They live in this repository. Clone it to run them:

git clone https://github.com/Project-MONAI/monai-physio.git

Download Tutorial Data

Tutorial 1 (heart) runs on the public Slicer-Heart 4D CT sample. We provide automated download for multiple datasets via a CLI, including the lung tutorials' key dataset, TCIA-4DLung (a converted tutorial subset - the full TCIA 4D-Lung collection remains a manual download), see data/TCIA-4DLung/README.md.

IMPORTANT: Run the download from the top level of the clone. The tutorials resolve their inputs against the repository root, so downloading elsewhere puts the data where they will not find it.

cd monai-physio
monai-physio-download-data Slicer-Heart-CT --directory data/Slicer-Heart-CT

Run Tutorial 01: Gated CT to USD

python tutorials/tutorial_01_convert_to_usd_slicer_heart_4d_ct.py

Explore the Code

That tutorial builds the same workflow the Python API exposes:

import itk
from pathlib import Path

from monai_physio import (
    RegisterImagesICON,
    SegmentChestTotalSegmentatorWithContrast,
    WorkflowConvertImageToUSD,
)

frame_files = sorted(Path("data/Slicer-Heart-CT").glob("slice_???.mha"))
time_series_images = [itk.imread(str(path)) for path in frame_files]

workflow = WorkflowConvertImageToUSD(
    time_series_images=time_series_images,
    reference_image=time_series_images[int(0.7 * len(time_series_images))],
    output_directory="./results",
    usd_project_name="cardiac_model",
    registration_method=RegisterImagesICON(),  # or RegisterImagesGreedy()
    segmentation_method=SegmentChestTotalSegmentatorWithContrast(),
)
results = workflow.process()

Explore the CLIs

The Tutorial 01 workflow and many of the workflows in this toolkit are also available as command-line tools but CLIs provide fewer options for customization:

monai-physio-convert-image-to-usd cardiac_4d.nrrd --contrast --output-dir ./results

Next Steps

See the quickstart and tutorials for full walkthroughs covering segmentation, registration, statistical shape modeling, and USD export.

Contributing

See CONTRIBUTING.md (also published at project-monai.github.io/monai-physio/contributing.html) for code style, testing, IDE setup, and pull request conventions.

License

This project is licensed under the Apache 2.0 License - see the LICENSE file for details.

Additionally, NVIDIA Omniverse is distributed under its own custom license, which makes it free for academic and commercial use. https://docs.omniverse.nvidia.com/ov/latest/common/NVIDIA_Omniverse_License_Agreement.html

Non-commercial Licenses (optional)

• NVIDIA Segment CT MRI AI weights (used in the SegmentNVSegmentCTMRI class, are restricted from commercial use. https://github.com/NVIDIA-Medtech/NV-Segment-CTMR
• NVIDIA Segment CT AI weights (used in the SegmentNVSegmentCT class) are released under the NVIDIA Open Model License Agreement, research use only, not for clinical use. https://huggingface.co/nvidia/NV-Segment-CT
• TotalSegmentator includes the optional use of some of their research-only models. Using those models assumes that you have the appropriate license key install, otherwise an error occurs. Those models can be disabled by calling set_has_academic_license(False) member function of the SegmentChestTotalSegmentator class.