🧠🩻 MELAGE: Medical Imaging Software

Machine learning & analysis for next-generation neuroimaging and medical imaging research

Table of Contents

• Features
• Plugins & Dynamic Extensions
• Installation
• Usage
• AI Segmentation Plugins
• Python API & Scripting
• Dependencies
• Manual (collapsible)
• License
• Citation & Acknowledgements
• Releases

MELAGE is an open-source neuroimaging software designed for analysis, segmentation, and visualization of multimodal datasets.
It combines classical medical image processing with state-of-the-art deep learning support, making it useful for both researchers and practitioners.

🚀 What's new in v2.2.0:

• 🤖 Integrated AI segmentation sidebar — MedSAM, SAM 2 (with 3D/video propagation), and nnInteractive are now permanent sidebar panels, not floating dialogs. Click directly on the live GL canvas to segment.
• 🐍 Headless Python API & extended CLI — import melage from Jupyter or scripts; no GUI, no PyQt5 import needed.
• ✏️ New toolbar tools — Freehand drawing, Magic Wand (region grow by pixel similarity), and Image Info tool.
• 🎨 VS Code-style activity sidebar — collapsible plugin panels with a 48 px icon rail for faster navigation.
• 🎥 Real-Time Video Segmentation — full medical video loading and processing for Ultrasound loops and Cine-MRI.

🎥 Key Features

• ⚡ Real-Time Video Processing: Seamlessly load medical videos (e.g., Ultrasound, Cine-MRI) and perform segmentation with the same high speed and accuracy as static images.
• 🖼️ Multi-Modality Support: Comprehensive support for MRI, CT, X-Ray, and Ultrasound data in standard formats (DICOM, NIfTI, AVI, MP4).
• 🤖 AI Segmentation Sidebar: MedSAM, SAM 2, and nnInteractive built directly into the sidebar — click or draw bounding boxes on any slice to segment interactively.
• 🧠 Deep Learning Integration: Built-in support for PyTorch models, allowing you to deploy state-of-the-art AI for automated segmentation and classification.
• 🛠️ Advanced Preprocessing: Powerful tools for denoising, filtering, resampling, and harmonizing image data before analysis.
• 🎨 Interactive Visualization: 2D and 3D rendering capabilities for exploring anatomical structures and segmentation results in detail.
• 🔌 Dynamic Plugin System: easily extend functionality by dropping Python scripts into the plugins/ folder—MELAGE automatically generates the GUI for you.
• 💾 Flexible Export: Save your results, including video segmentation masks, into standard research-ready formats.
• 🐍 Python API & CLI: Call every processing step from Jupyter notebooks or shell scripts — no GUI required. Ideal for large-cohort pipelines and automated studies.

🧩 Plugins & Dynamic Extensions

MELAGE now features a powerful Dynamic Plugin System that allows you to integrate custom Deep Learning models or image processing algorithms without modifying the core source code.

How it works:

1. Create: Write your algorithm or model wrapper as a Python class inheriting from the MELAGE Plugin base class.
2. Drop-in: Place your script in the plugins/ directory.
3. Auto-Load: MELAGE automatically detects, loads, and generates a GUI widget for your tool upon launch.

This modular architecture supports:

• Deep Learning inference: Drag-and-drop integration for .pth or .onnx models.
• Custom Analysis: Add proprietary segmentation or quantification logic.
• Workflow Automation: Create macros for repetitive tasks.

🚀 How to Add a Plugin

1. Folder Structure: Organize your plugin in its own directory under the plugins/ folder. MELAGE recursively scans these folders to find valid plugins.

   plugins/
   ├── warpseg/
   │   ├── __init__.py
   │   ├── WarpSeg.py       <-- Contains the Plugin Class
   │   └── WarpSeg_schema.py <-- Contains the Plugin Scheme for GUI
   └── my_new_tool/
       └── ...
   

🚀 Installation

🐧 LINUX:

🐍 STEP 0: INSTALL CONDA (PREREQUISITE)

If you don't have Conda, install Miniconda (lightweight version).

1. Download installer

   wget [https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh](https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh)
   

2. Run installer (Type 'yes' to license and init)

   bash Miniconda3-latest-Linux-x86_64.sh
   

3. Refresh shell

   source ~/.bashrc
   

🛠️ STEP 1: CREATE ENVIRONMENT (RECOMMENDED)

# Create env (Python 3.10 is most stable with PyQt5)
conda create -n melage-gui python=3.10 -c conda-forge -y

# Activate the environment
conda activate melage-gui

# Install PyQt5 (includes Qt frameworks)
conda install -c conda-forge pyqt=5 -y

# Install melage inside the environment
pip install melage

# Verify which melage is being used (should point to this env)
which melage

# Run
melage

📦 STEP 2: INSTALL MELAGE (STANDALONE)

If skipping Conda (Not recommended for GUI apps):

From PyPI:

pip install melage

🚀 STEP 3: CREATE ONE-CLICK LAUNCHERS

Create a script file to automatically activate the environment and run the app.

1. Create a file named 'launch_melage.sh' with the following content: (Note: Adjust the 'source' path if your conda is installed elsewhere)

   #!/bin/bash
   # Initialize Conda (Adjust path based on 'conda info --base')
   source ~/miniconda3/etc/profile.d/conda.sh
   conda activate melage-gui
   melage
   

2. Make it executable:

   chmod +x launch_melage.sh
   

3. Run it:

   ./launch_melage.sh
   

4. (Optional) Create a Desktop Shortcut file named 'Melage.desktop': (Create this file in ~/.local/share/applications/ for Start Menu access OR on your ~/Desktop/ for a desktop icon).

   [Desktop Entry]
   Version=1.0
   Type=Application
   Name=Melage
   Comment=Melage GUI
   # IMPORTANT: Use absolute paths below (e.g., /home/user/...)
   Exec=/home/user/path/to/launch_melage.sh
   Icon=/home/user/path/to/your_icon.png
   Terminal=false
   Categories=Utility;
   

5. (Optional) If put on Desktop, right-click file -> "Allow Launching".

🍎 macOS:

🐍 STEP 0: INSTALL CONDA (PREREQUISITE)

1. Download installer (Intel)

   curl -O https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh
   

   ... OR ...

   Download installer (Apple M1/M2 Silicon)

   curl -O https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.sh
   

2. Run installer

   bash Miniconda3-latest-MacOSX-x86_64.sh
   # or
   bash Miniconda3-latest-MacOSX-arm64.sh
   

3. Refresh shell

   source ~/.zshrc
   

🛠️ STEP 1: CREATE ENVIRONMENT (RECOMMENDED)

# Create env (Python 3.10 is most stable with PyQt5)
conda create -n melage-gui python=3.10 -c conda-forge -y

# Activate the environment
conda activate melage-gui

# Install PyQt5 (includes Qt frameworks)
conda install -c conda-forge pyqt=5 -y

# Install melage inside the environment
pip install melage

# Verify which melage is being used (should point to this env)
which melage

# Run
melage

📦 STEP 2: INSTALL MELAGE (STANDALONE)

If skipping Conda (Not recommended for GUI apps):

From PyPI:

pip install melage

🚀 STEP 3: CREATE ONE-CLICK LAUNCHERS

Create a script file to automatically activate the environment and run the app.

1. Create a file named 'launch_melage.sh' with the following content: (Note: Adjust the 'source' path if your conda is installed elsewhere)

   #!/bin/bash
   # Initialize Conda (Adjust path based on 'conda info --base')
   source ~/miniconda3/etc/profile.d/conda.sh
   conda activate melage-gui
   melage
   

2. Make it executable:

   chmod +x launch_melage.sh
   

3. Run it:

   ./launch_melage.sh
   

Alternatively

1. Open "Automator" (Cmd + Space -> Type Automator).
2. Select "Application" -> Click "Choose".
3. Search for "Run Shell Script" and double-click it.
4. Paste the code below (Update the path using 'conda info --base'!):

source /Users/yourname/miniconda3/etc/profile.d/conda.sh
conda activate melage-gui
melage

5. Press Cmd+S to save. Name it "Melage" and save to Applications.

--- HOW TO CHANGE THE APP ICON ---

1. Copy your logo image (Open image -> Cmd + C).
2. Right-click your new "Melage.app" -> "Get Info".
3. Click the small icon in the top-left corner of the Info window.
4. Paste (Cmd + V).

🖥️ WINDOWS:

🐍 STEP 0: INSTALL CONDA (PREREQUISITE)

1. Download .exe from https://docs.conda.io/en/latest/miniconda.html
2. Run installer.
3. Open "Anaconda Prompt" from Start Menu for the steps below.

🛠️ STEP 1: CREATE ENVIRONMENT (RECOMMENDED)

# Create env
conda create -n melage-gui python=3.10 -c conda-forge -y


# Activate
conda activate melage-gui

# Install PyQt5
conda install -c conda-forge pyqt=5 -y
# Install antspyx (Optioanl) to avoid pip install melage failed for any reason)
conda install -c conda-forge antspyx

# Install melage
pip install melage

# Run
melage

📦 STEP 2: INSTALL MELAGE (STANDALONE)

If skipping Conda (Not recommended for GUI apps):

From PyPI:

pip install melage

🚀 STEP 3: CREATE ONE-CLICK LAUNCHERS

1. Create a file named 'launch_melage.bat' with the following content:

   call conda activate melage-gui
   melage
   pause
   

2. Double-click 'launch_melage.bat' to run the app.
3. (Optional) Right-click the .bat file -> "Send to" -> "Desktop (create shortcut)" to give it a custom icon.

🖥️✨ Usage

After installation and activating your virtual environment, you can launch MELAGE directly from the terminal:

conda activate melage-gui
melage

MELAGE graphical user interface in action.

🤖 AI Segmentation Plugins

Starting from v2.2.0, MELAGE ships three state-of-the-art interactive segmentation models as permanent sidebar panels. They draw directly on the live GL canvas — no separate window, no coordinate mapping needed.

Install the AI extras first:

pip install melage[ai]

🏥 MedSAM

MedSAM is a foundation model for medical image segmentation fine-tuned from SAM ViT-B.

How to use:

1. Open the MedSAM sidebar tab.
2. Select a model variant and axis (Axial / Coronal / Sagittal).
3. Draw a bounding box or click positive/negative points directly on the slice.
4. The result is written back into the active segmentation color.

Weights are downloaded automatically to ~/.melage/weights/ on first use.
Supports fp16 autocast for lower GPU memory usage.

🎬 SAM 2

SAM 2 extends SAM with a video-memory attention mechanism, making it uniquely suited for propagating segmentations through 3D volumes slice-by-slice.

How to use:

1. Open the SAM 2 sidebar tab.
2. Draw a bounding box or click points on one slice.
3. Press Propagate current label mask to let SAM 2's video predictor carry the segmentation through all adjacent slices automatically.
4. Switch to a new label index to segment the next structure.

Supports up to 8 simultaneous labels; each maps to a distinct integer in the segmentation volume.

🧠 nnInteractive

nnInteractive is a 3D-aware interactive segmentation framework from the MIC-DKFZ group. Unlike slice-based methods, it propagates each click through the full 3D volume immediately.

How to use:

1. Open the nnInteractive sidebar tab.
2. Click positive points (green) or negative points (red) on any slice.
3. The segmentation updates across the entire volume after each click — no manual propagation needed.
4. Press New Label → to start the next structure; previous labels remain.

Model weights are downloaded automatically from HuggingFace Hub (nnInteractive/nnInteractive) on first use.

Sidebar navigation

All three plugins appear as icon buttons in the VS Code-style activity bar on the left edge of the main window. Click an icon to expand or collapse its panel. The sidebar coexists with the existing plugin system — other dynamic plugins still appear in the Plugins menu.

🐍 Python API & Scripting

Starting from v2.2.0, MELAGE ships a headless Python API and an extended CLI.
No GUI, no display, no PyQt5 import needed — just import melage.
This is designed for:

• 📓 Jupyter notebooks — interactive exploration and visualisation of results
• 🔁 Shell pipelines — chain tools with a single melage run command
• 🏥 Large studies — loop over hundreds of subjects without touching the GUI

🔧 Quick start

import melage

# Load any supported format: NIfTI, NRRD, DICOM folder, GE Kretz .vol, video
vol = melage.load("brain.nii.gz")
print(vol)
# Volume(shape=(189,233,197), spacing=(1.00,1.00,1.00) mm, dtype=float64)

The Volume object is Jupyter-friendly — in a notebook it renders as a metadata table.

🛠️ Preprocessing

# N4 bias-field correction (SimpleITK)
vol = melage.preprocess.n4_bias(vol)
vol = melage.preprocess.n4_bias(vol, iterations=50, shrink_factor=2, use_otsu=True)

# Resample to a new voxel spacing
vol = melage.preprocess.resize(vol, spacing=1.0)          # isotropic 1 mm
vol = melage.preprocess.resize(vol, spacing=[0.8, 0.8, 1.2], method="linear")

# Percentile intensity normalisation
vol = melage.preprocess.normalize(vol)
vol = melage.preprocess.normalize(vol, percentile_low=1, percentile_high=99)

# Hard intensity window
vol = melage.preprocess.threshold(vol, low=100, high=3000)

# Keep only the largest connected component of a segmentation label
vol = melage.preprocess.largest_component(vol, label=1)

🔬 Segmentation

Results are stored in vol.segmentation (integer NumPy array).

# Brain Extraction Tool (BET) — based on Smith 2002
vol = melage.segment.bet(vol)
vol = melage.segment.bet(vol, fractional_threshold=0.4, thresholding=True)

# Fuzzy C-Means tissue segmentation
vol = melage.segment.fcm(vol, n_classes=3)
vol = melage.segment.fcm(vol, n_classes=3, method="PFCM", max_iter=200)

# Convenience: N4 bias correction → BET in one call
vol = melage.segment.preprocess_and_bet(vol)

💾 Saving results

melage.save(vol, "corrected.nii.gz")              # image data
melage.save(vol, "brain_mask.nii.gz", what="seg") # integer segmentation
melage.save(vol, "tissues.nii.gz",   what="seg")

🧊 3D visualization

MELAGE's interactive 3-D viewer (glScientific) needs a live Qt/OpenGL display, so the API ships a lightweight, headless counterpart in melage.visualize — built on scikit-image marching cubes + matplotlib — for quick surface previews, screenshots, and mesh export from scripts, servers, or notebooks.

vol = melage.load("brain.nii.gz")
vol = melage.segment.bet(vol)              # populates vol.segmentation

# Quick interactive-style 3-D plot (matplotlib figure)
fig = melage.visualize.render(vol, label=1, title="Brain surface")

# Render several labels together with custom colours
fig = melage.visualize.render(vol, label=[1, 2], color=[(0.8, 0.2, 0.2, 1), (0.2, 0.6, 0.9, 0.6)])

# Save a static screenshot directly to disk
melage.visualize.screenshot(vol, "brain_3d.png", label=1, elev=15, azim=45)

# Extract a surface mesh (trimesh.Trimesh) or export it (.stl, .obj, .ply, .glb, ...)
mesh = melage.visualize.mesh(vol, label=1, smooth=True)
melage.visualize.export_mesh(vol, "brain.stl", label=1, smooth=True)

📋 Image metadata

melage.info("brain.nii.gz")
# File   : brain.nii.gz
# Format : NIfTI
# Shape  : (189, 233, 197)
# Spacing: 1.000 × 1.000 × 1.000 mm
# Dtype  : int16

🔁 Config-driven pipelines

pipeline = [
    ("n4_bias",   {}),
    ("resize",    {"spacing": 1.0}),
    ("normalize", {}),
    ("bet",       {"fractional_threshold": 0.45}),
]

for subject in subject_list:
    vol = melage.load(subject)
    for tool, kwargs in pipeline:
        vol = melage.run(tool, vol, **kwargs)
    melage.save(vol, subject.replace(".nii.gz", "_bet.nii.gz"), what="seg")

melage.list_tools()   # → ['bet', 'fcm', 'n4', 'n4_bias', 'normalize', 'resize', ...]

📊 Jupyter notebook example

import melage
import matplotlib.pyplot as plt

vol = melage.load("brain.nii.gz")
vol                                         # renders metadata table in notebook

vol = melage.preprocess.n4_bias(vol, progress=False)
vol = melage.segment.bet(vol, progress=False)

mid = vol.shape[2] // 2
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(10, 4))
ax1.imshow(vol.data[:, :, mid],        cmap="gray");  ax1.set_title("Image")
ax2.imshow(vol.segmentation[:, :, mid], cmap="hot");  ax2.set_title("BET mask")
plt.tight_layout()

melage.visualize.render(vol, label=1, title="Brain surface")  # interactive 3-D plot

🖥️ CLI subcommands

# List all available tools
melage tools

# Inspect an image (no full load)
melage info brain.nii.gz

# Run a single tool — input → output
melage run n4_bias   brain.nii.gz   brain_n4.nii.gz
melage run resize    brain.nii.gz   brain_1mm.nii.gz  --spacing 1.0
melage run resize    brain.nii.gz   brain_ani.nii.gz  --spacing 0.8 0.8 1.2 --method linear
melage run normalize brain.nii.gz   brain_norm.nii.gz
melage run bet       brain.nii.gz   brain_mask.nii.gz
melage run fcm       brain.nii.gz   tissues.nii.gz    --n-classes 3

# Add --silent to suppress progress output (useful in scripts)
melage run n4_bias brain.nii.gz out.nii.gz --silent

The original --headless --tool syntax is still supported for backward compatibility.

📈 Progress control

Every function accepts a progress keyword:

from tqdm.auto import tqdm

bar = tqdm(total=100, desc="BET")
vol = melage.segment.bet(vol, progress=lambda pct, msg: bar.update(pct - bar.n))

📦 Dependencies

MELAGE relies on the following core libraries:

- NumPy, SciPy – numerical computing & scientific operations  
- scikit-image, Pillow, OpenCV – image processing & visualization  
- scikit-learn, numba, einops – machine learning & acceleration  
- nibabel, pydicom, pynrrd, SimpleITK – medical imaging formats (NIfTI, DICOM, NRRD)  
- PyQt5, QtPy, qtwidgets – GUI support  
- matplotlib, vtk, PyOpenGL – visualization & rendering  
- shapely, trimesh, rdp – geometry & 3D mesh processing  
- pyfftw – fast Fourier transforms  
- cryptography – security utilities  
- dominate – HTML generation  

Optional Extras

pip install melage[ai]      # AI segmentation plugins
pip install melage[bet]     # BET + 3D mesh export
pip install melage[all]     # everything

📜 License

For licensing inquiries, please contact:

• b.jafrasteh@gmail.com
• baj4003@med.cornell.edu

Protection & Registration

MELAGE is registered in the Electronic Register of Intellectual Property as software, under file FCAD-22002, by the Technology Transfer Office of the Andalusian Public Health System (OTT-SSPA).

• Identifier: 2211222681375
• View registration details on SafeCreative
  This legal protection ensures intellectual property rights are formally secured.

📖 Citation & Acknowledgements

If you use MELAGE in your research, please cite the following work:

Jafrasteh, B., Lubián-López, S. P., & Benavente-Fernández, I. (2023).
MELAGE: A purely Python-based Neuroimaging Software (Neonatal).
arXiv preprint arXiv:2309.07175

We would like to acknowledge all contributors and collaborators who have supported the development and testing of MELAGE.

🚀 Releases

Stable releases and updates of MELAGE are available on the GitHub Releases page.

• 🟢 Stable releases: Fully tested, recommended for production and research use.
• 🧪 Pre-releases / beta versions: For testing new features and providing feedback.

Changelog highlights

Stay updated by watching the repository for new release notifications.