pentachrome-plugin

Napari plugin for the Pentachrome histology pipeline. The public plugin exposes a single clinician-facing widget, Guided Analysis, that walks through the whole workflow in one pane:

1. Extract — pull tissue-region TIFFs out of Olympus .vsi files.
2. Detect — run the trained nnUNet Epithelium / MultiStructure models on those images and load colorized masks back into the viewer.
3. Measure — per-region statistics (thickness, composition, cell densities), with CSV export.

Each phase also exists as a separate advanced widget (VsiExtractorWidget, NnUnetInferenceWidget, AnalysisWidget) used during development. These are not registered in the public menu, see From source (development).

Source and issues: https://github.com/dtsilis7/Pentrachrome-Pipeline

Requirements: Windows, Python 3.10, napari >= 0.4.18, and the nnUNet model weights. VSI reading uses bioio-bioformats (a plugin dependency) which auto-downloads a JDK on first use — no Java install, no javabridge. Only the model weights are installed separately (see below).

Installation (Windows, PowerShell)

A working install has three parts, in this order:

1. A conda environment (Python 3.10) with NumPy<2
2. The plugin itself (pulls in the JDK-free bioio-bioformats VSI reader)
3. The nnUNet model weights (downloaded separately)

⚠️ pip install pentachrome-plugin gives you the UI and VSI extraction, but inference still needs the model weights (Step 3) in the same env. Installing through napari's plugin manager only does the pip part.

Step 1 — environment

Create a Python 3.10 env pinned to numpy<2 (the bioio-bioformats VSI reader is pinned <2, and nnU-Net / the imaging stack share the 1.x ABI).

conda create -n pentachrome python=3.10 "numpy<2" -y
conda activate pentachrome

No JDK or javabridge to install. VSI reading goes through bioio-bioformats (pulled in by Step 2), which provisions the Bio-Formats JVM via scyjava + cjdk and auto-downloads a JDK the first time you extract (cached afterward).

Step 2 — the plugin

pip install pentachrome-plugin   # also installs the bioio-bioformats VSI reader
pip install nnunetv2             # required for the Detect step

You can also install the plugin through napari's Plugins -> Install/Uninstall Plugins dialog (search "pentachrome"), but that only covers this step, you still need Step 1 and Step 3 in the same environment.

Already installed? To pull a newer release into the same env, upgrade just the plugin (leaves PyTorch, nnU-Net, and weights untouched); restart napari afterwards:

pip install --upgrade pentachrome-plugin
pip show pentachrome-plugin        # check the "Version:" line

Step 3 — model weights

Download the nnUNet weights and point the widget at them — see Model weights below.

Step 4 — GPU (CUDA) acceleration (optional, NVIDIA only)

⚠️ pip install nnunetv2 installs the CPU-only PyTorch wheel. On a machine with an NVIDIA GPU, the Guided Analysis Check GPU button will still report "No CUDA GPU available to PyTorch" and inference runs on the (much slower) CPU, because the default PyTorch wheel from PyPI (torch ...+cpu) contains no CUDA at all. Installing the NVIDIA CUDA toolkit system-wide does not fix this — the PyTorch wheel bundles its own CUDA runtime.

To use the GPU, reinstall PyTorch from the matching CUDA index after nnU-Net, so the CUDA wheel wins:

conda activate pentachrome
# Do this AFTER `pip install nnunetv2` — otherwise nnU-Net drags the +cpu wheel back in.
# Name torch AND torchvision: they must come from the same index as a matched pair.
pip install --force-reinstall torch torchvision "numpy<2" --index-url https://download.pytorch.org/whl/cu124

⚠️ Never install torch on its own. torchvision comes in transitively with nnU-Net (dynamic-network-architectures → timm → torchvision) and its compiled C++ ops are built against one exact torch version. Reinstalling torch alone strands the old torchvision, and inference fails with RuntimeError: Couldn't load custom C++ ops or operator torchvision::nms does not exist. Fix it by rerunning the command above with both names, not by pinning a torchvision version by hand.

Pick the CUDA line your driver supports (run nvidia-smi; the top-right "CUDA Version" is the max your driver allows — a newer driver runs older wheels):

Verify (in the same env):

python -c "import torch, torchvision; print(torch.__version__, torch.version.cuda, torch.cuda.is_available()); print('torchvision', torchvision.__version__); print(torch.ops.torchvision.nms)"

You want a version ending in +cuXXX (not +cpu), a CUDA number, and True — and the last line must print a function rather than raising, which is what proves torch and torchvision still match. Then Check GPU in the widget turns green. If you ever reinstall or upgrade nnunetv2, redo this step — it can pull the +cpu wheel back in and desync torchvision at the same time.

On managed/locked-down Windows machines, inference is launched through the env's python.exe (not the generated nnUNetv2_predict.exe), so AppLocker / group-policy .exe restrictions don't block it (fixed in 0.7.1).

Verify

python -m napari

In napari, open Plugins -> Guided Analysis — the widget should load and show its Extract / Analyze / Statistics steps.

From source (development)

For working on the plugin itself, do Step 1 above, then install editable from a checkout instead of from PyPI (cd into the plugin directory first, or pass the absolute path):

conda activate pentachrome
cd "...\pentachrome_plugin"
pip install -e .

The three phase widgets are de-registered from the public menu. To open them standalone during development, run the repo's dev_widgets.py — it docks the Extractor / Inference / Statistics widgets as tabs:

python dev_widgets.py

Launch

conda activate pentachrome   # or whichever env you installed into
python -m napari

In napari: Plugins -> Guided Analysis.

The per-phase sections below (nnUNet Inference, Mask Statistics, etc.) describe the underlying widgets, which the Guided Analysis pane drives end-to-end. Their Plugins -> ... menu references apply only to the dev widgets opened via dev_widgets.py; end users reach the same functionality through Guided Analysis.

Model weights

The nnUNet weights (~900 MB) aren't bundled in the PyPI package. Download [nnunet_results.zip] from the releases page, unzip it, and point the inference widget's nnUNet results field at the extracted folder (the one containing Dataset001_Epithelium and Dataset002_MultiStructure).

nnUNet Inference (Phase 2)

Requires nnunetv2 installed in the same environment (the nnUNetv2_predict CLI must be on PATH) — this is covered by Step 2 of Installation above.

Workflow:

1. Load TIFFs into napari (e.g. via Phase 1's auto-load checkbox, or drag-and-drop).
2. Open Plugins -> nnUNet Inference.
3. Select one or more image layers in the list.
4. Tick Epithelium, MultiStructure, or both.
5. Set Output folder (where raw + colorized masks go) and nnUNet results (folder containing Dataset001_Epithelium and Dataset002_MultiStructure). The results path auto-fills if nnUNet_Training/nnUNet_results/results is found.
6. Pick Device (cpu or cuda) and click Analyze.

Speed vs quality (important on CPU)

nnUNet inference on a laptop CPU is slow because every image goes through folds × mirror augmentations × sliding-window patches forward passes. With defaults that can be 20+ passes per image. The widget exposes three knobs in the Speed / quality group:

With all three defaults on a CPU laptop, one ROI tile should take a few minutes instead of 30+. Switch to All 5 folds + TTA on once you've moved to a GPU box.

Continuing from the extractor

The two widgets are linked through two small bridges, so you can run Extract -> Analyze in a single napari session without re-picking files:

• When the extractor auto-loads a TIFF as a viewer layer, it stashes the on-disk path on layer.metadata['source_tiff']. The inference widget reads that during staging and copies the original file into _staging_input/ rather than re-saving the in-memory array, important for 15k x 15k tiles.
• When an extraction completes, the inference widget's "Use last extractor output" button pre-fills the output folder to <extractor_output_root>/_inference, so masks land next to the per-VSI subfolders the extractor created.

Both bridges are in-process only (see _session.py); they reset when napari closes.

Outputs land in:

<output_folder>/
  _staging_input/            # nnUNet-named (_0000.tif) copies of the selected layers
  epithelium_raw/            # binary masks from Dataset001
  epithelium_colored/        # RGB colorized masks (red epithelium)
  multistructure_raw/        # 6-class masks from Dataset002
  multistructure_colored/    # RGB colorized masks (Elastin/Collagen/Nuclei/Mucins/Membrane/Goblets)

Colorized masks are added to the viewer as RGB image layers when the run finishes.

nnUNet inference architecture

Same subprocess pattern as Phase 1. The widget never imports torch or nnUNetv2 directly; it spawns _inference_worker.py which:

• sets nnUNet_results to the configured results dir,
• calls nnUNetv2_predict once per enabled model (folds 0-4 for Epithelium, fold 0 for MultiStructure, matching run_inference.py),
• colorizes the resulting integer masks with the palettes from colorize_masks.py / compare_grid.py,
• streams JSON-line events on stdout for the widget's progress bar and log.

How it works

• The widget itself never touches the JVM. When you click Extract ROIs, it spawns _vsi_worker_bioio.py as a separate Python process.
• That worker starts the Bio-Formats JVM through bioio-bioformats (scyjava + cjdk — no user JDK), loops over the VSI files using TileMaskStitcher (vsi_handler/tile_mask_stitcher_bioio.py, which drives loci.formats.ImageReader directly via vsi_handler/_bioio_reader.py), writes numbered TIFFs into <output_root>/<vsi_basename>/, and emits JSON-line progress events on stdout.
• The widget streams those events on a background thread and updates the progress bar / log without blocking the UI.
• When the worker exits, the JVM dies with it. The next extraction batch starts a fresh JVM in a fresh process - this avoids the "JVM cannot be restarted" pitfall during a long napari session.

Defaults

The parameter defaults mirror Processing_VSI_Files.py:

Layout

pentachrome_plugin/
  pyproject.toml
  README.md
  src/pentachrome_plugin/
    __init__.py
    napari.yaml             # napari manifest
    _session.py             # in-process cross-widget state (extractor -> inference -> analysis)
    _widget.py              # VsiExtractorWidget (Phase 1)
    _vsi_worker_bioio.py    # VSI subprocess entrypoint (bioio-bioformats)
    vsi_handler/            # bioio VSI reader (_bioio_reader.py) + tile/mask stitcher
    _inference_widget.py    # NnUnetInferenceWidget (Phase 2)
    _inference_worker.py    # nnUNet subprocess entrypoint
    _analysis_widget.py     # AnalysisWidget (Phase 3, in-process)

Phase 3 (Mask Statistics) lives alongside these and registers through napari.yaml.

Mask Statistics (Phase 3)

Pure in-process; no subprocess needed (no JVM, no torch). Reuses EpithelialAnalysis/Analyzers/ (Descriptors.py, Thickness.py), so the same metrics that fed the original region_summary.csv show up in the widget.

Workflow:

1. Run Phase 2 first so epithelium_raw/ and multistructure_raw/ exist.
2. Open Plugins -> Mask Statistics.
3. Select one or more image layers in the list (their names must match the mask filenames in epithelium_raw/ / multistructure_raw/; if the inference widget staged them, that's already true).
4. Click Use last inference output (or browse).
5. Tweak Pixel size, Region dilation, Min epithelium area if needed (defaults match Main.py).
6. Click Analyze.

For each detected epithelial region the widget reports:

A bold (all regions) row appended per image gives area-weighted means of the percentages / thickness and totals for the counts. Export CSV... saves the whole table (per-region rows + aggregate rows).

The exported CSV carries one extra column that the on-screen table deliberately hides: log2(E/C), the base-2 log of the Elastin/Collagen ratio (0 = balanced, +1 = elastin twice collagen, −1 = the reverse). It's symmetric and so averages and statistically tests correctly across regions/patients, where the raw ratio — bounded at 0, unbounded above — is skewed. It's kept out of the viewer because it's a stats aid, not an at-a-glance number, and it's a pure function of the raw column so nothing is lost. (Added in 0.7.3.)

The elastin organization score (ElastinAnalyzer.determine_organized_region) from Main.py is intentionally not yet exposed, it's much heavier (skan + shapely + ROI polygons) and will land as a separate toggle.

Class isolation

A "Class isolation" group at the top of the widget lets you view a single class (or a combination) without rerunning anything:

1. Pick a source layer (the original TIFF, not a colorized mask).
2. Tick one or more of Elastin, Collagen, Nuclei, Mucins, Cell Membrane, Goblets, Epithelium.
3. Click one of:
   • Show as mask — adds a new layer that's white everywhere except the ticked classes, colored with the same palette as the inference widget.
   • Show on original — adds a copy of the original image with all pixels outside the ticked classes turned white. Useful for sanity- checking the segmentation against the stain.
4. Clear isolated layers removes everything this panel added in one go.

Masks are read on demand from the inference output folder; the original layer's pixels are taken from the viewer.

License

This project is licensed under the MIT License, see the LICENSE file for details.