COMP3888_Capstone

Quickstart

curl -LsSf \
  -H "Authorization: Bearer $(gh auth token)" \
  -H "Accept: application/vnd.github.raw" \
  "https://api.github.com/repos/C-O-R-N-H/COMP3888_TU13_01_P35/contents/install.sh?ref=main" | sh

The repository is private, so this needs the GitHub CLI logged in (gh auth login). A plain curl of the raw URL returns 404.

Once that finishes, run inference with:

pansegai scan.nii.gz .prediction-artifacts
pansegai scan.nii.gz ./predictions --checkpoint mri/mri_final_ckpt_c2b_annealing_aggressive.pth

Prerequisites

Please ensure you have uv installed.

curl -LsSf https://astral.sh/uv/install.sh | sh

Installation

From the repository root, download the pancreas segmentation datasets (about 40 GiB). The script writes into datasets/ regardless of the current directory, and skips anything already extracted:

./datasets/download_datasets

Running an example

uv run --extra cuda python phase_1/totalseg_demo.py

Choose exactly one PyTorch backend extra: cuda for the standard CUDA wheel, or xpu for Intel Arc. uv rejects selecting both, preventing one wheel from silently replacing the other.

Running Evaluations

To run the baseline evaluations for this project, please see the Makefile in the root of this repository.

You can run a variety of available evaluations by simply running:

make {name of eval make target}
# example below
make eval-mri-mrseg-amos-pancreas

Alternatively, it is also possible to run the CLI script with relevant arguments directly:

uv run python panseg/cli.py eval \
		--parquet $(MRI_PARQUET) \
		--modality mri \
		--model mrseg \
		--dataset amos-pancreas

This will run the evaluation for the MRI MRSEG model on the AMOS pancreas dataset. Errors are thrown for missing or invalid arguments.

pansegai CLI

pansegai runs the same Hugging Face checkpoint inference as panseg/cli.py infer. Modality is inferred from voxel intensities unless you pass --modality. --checkpoint is a file path inside dillondesilva/comp3888-capstone-panseg-models; omit it to use the default CT or MRI annealing checkpoint. install.sh installs uv when it is missing, then installs the tool from this git repository (PANSEG_REF defaults to main). MRSegmentator is an optional extra (uv sync --extra mrsegmentator) because its Torch pin conflicts with this project's Torch 2.13 requirement; pansegai does not need it.

From a checkout, without the install script:

uv tool install --editable ".[nnunet]"

The same run without installing the tool:

uv run --extra nnunet python panseg/cli.py infer \
  --input scan.nii.gz \
  --artifacts-dir .prediction-artifacts

Eval artifact viewer

After an infer dump exists, compare models against the original image and ground truth:

uv run --extra viz streamlit run apps/eval_viewer.py

The sidebar picks a folder under .eval-artifacts/. Toggle Ground truth (green) and Predicted (red) to overlay both on the same original slice.

MONAI BasicUNet

The first in-project MONAI model is optional, so install its dependency extra before importing it:

uv sync --extra monai --extra cuda

MonaiUNetModel is a 3D, single-channel pancreas-segmentation network. It accepts a floating-point tensor shaped [batch, depth, height, width] and returns a float32 binary mask of the same shape. It adds MONAI's channel dimension internally.

import torch
from panseg.models.monai_unet import MonaiUNetModel

model = MonaiUNetModel(modality="ct", checkpoint_path="models/panseg_unet.pt")
model.eval()
mask = model(torch.randn(1, 32, 32, 32))

checkpoint_path must point to a local raw PyTorch state dictionary produced by the same MonaiUNetModel architecture. Without one, the model has random weights and is useful only for integration checks. Training, NIfTI preprocessing, and sliding-window full-volume inference are not part of this initial integration.

Run unit tests with:

make test                 # all tests under tests/ (with coverage)
make test-ci              # same shape as GitHub Actions (CPU + monai + coverage)
make test-metrics         # HD95 / CC / eval-summary contracts
make test-code            # application modules (skips MONAI-only cases)
make test-monai           # MONAI DiNTS adapter tests (needs --extra monai)

Or directly:

uv run --group dev pytest tests/
uv run --extra monai --group dev pytest tests/code/test_monai_dints.py

Continuous integration

GitHub Actions (.github/workflows/ci.yml) runs on every push to main and on pull requests. The job uses Linux CPU wheels only (no CUDA/XPU), installs the monai extra so optional DiNTS tests execute, and does not download the ~40 GiB image datasets — unit tests rely on temporary fixtures and mocks. Coverage XML/HTML reports are uploaded as workflow artifacts.

Run the MONAI patch demo with a compatible checkpoint:

uv run --extra monai --extra cuda python phase_1/monai_demo.py --checkpoint models/panseg_unet.pt

The demo runs a centre crop rather than full-volume inference. Full-volume sliding-window inference will be added with the MONAI training workflow.

Pretrained MONAI pancreas model

MonaiPancreasDiNTSModel uses MONAI Model Zoo's CT-only pancreas/tumour DiNTS bundle. Like TotalSegModel, it downloads its pretrained bundle automatically on first use and caches it in ~/.cache/panseg/monai-bundles (override with bundle_dir). Its run() method accepts the same batched 3D tensor and optional NIfTI affine as TotalSegModel; it returns only the pancreas label as a binary mask. The bundle supplies the CT preprocessing and full-volume sliding-window inference.

For inference, the adapter selects CUDA first, then PyTorch's Intel XPU backend (for Intel Arc), then CPU. XPU uses one sliding-window patch at a time by default to keep memory use conservative.

All PanSeg models accept accelerator="auto" by default. Auto selection uses CUDA, then XPU, then MPS, then CPU, but only chooses accelerators supported by the individual adapter. For example, the MONAI adapters support XPU, while TotalSegmentator currently supports CUDA, MPS, and CPU only. Request a specific supported backend with accelerator="xpu" or accelerator="cpu".

import nibabel as nib
import torch

from panseg.models.monai_dints import MonaiPancreasDiNTSModel

source = nib.load("scan.nii.gz")
volume = torch.from_numpy(source.get_fdata()).unsqueeze(0).float()
model = MonaiPancreasDiNTSModel(modality="ct", accelerator="auto")
mask = model.run(volume, affine=source.affine)

Run the matching demo with:

uv run --extra monai python phase_1/monai_dints_demo.py

If you have activated a separate environment containing PyTorch's Intel XPU wheel (for example .venv-xpu), use the tracked xpu extra. It selects PyTorch's XPU wheel index and prevents uv from replacing it with a CUDA build:

uv sync --extra monai --extra xpu
uv run --extra monai --extra xpu python phase_1/monai_dints_demo.py