EMCFsys

Towards foundation models for EM images analysis. emcfsys provides a comprehensive toolkit and a napari plugin for Electron Microscopy (EM) image restoration and segmentation, featuring EMCellFiner and EMCellFound—two foundation models pre-trained on a massive dataset of 4 million EM images.

✨ Key Features

1. Key Features for emcfsys

• Pre-trained Backbones: Optimized on 4M+ EM images for superior feature extraction: EMCellFound and EMCellFiner.
• Code-Free Train and Inference: Train the segmentation model using Pre-trained Backbones and inference in the GUI without Any Code.
• Code-Free and Training-Free Image restoration/super-resolution pipline: Training-Free pipline for EM image restoration/super-resolution.
• Code-Free GUI for Napari, and Code-based example tutorial in mmLab_backend_demo.ipynb. This notebook is built upon mmengine, mmseg, and mmdet to facilitate code‑driven training workflows.

2. Segmentation pipline using Foundation model

• Pre-trained Backbones:
  • EMCellFound Core: Our foundational backbones are built upon state-of-the-art ViT (Vision Transformer) and ConvNext architectures. These models are pre-trained using advanced self-supervised learning frameworks, specifically MAE (Masked Autoencoders) and DINOv3, ensuring robust feature representation for complex electron microscopy data.
  • Continuous Evolution: We are committed to the iterative refinement of our models. We periodically retrain EMCellFound using superior architectures, optimized algorithms, and larger-scale datasets to ensure the system consistently delivers peak performance.
  • Timm Library Integration: To provide maximum flexibility, the system fully supports a wide range of popular pre-trained models from the timm library, allowing users to select the most suitable backbone for their specific research needs.
• Segmentation Heads: Includes U-Net, PSPNet, DeepLabv3+, and UperNet.
• Finetune Models: We support to finetune the EMCellFound/Timm-model to make specialize segmentation pipline.
• Inference 2D/3D images: We support to load the Checkpoint and inference image in 2D and 3D.
• Tailored Training Strategies: Detailed specifications of our training configurations can be found in Functions notebook. Key components include:
  • Data Augmentation: Robust Dataset class with multiple transform strategies.
  • Loss Functions: Integrated CrossEntropy and Dice Loss (Focal Loss coming soon).
  • Metrics: Real-time evaluation using IoU, Accuracy, and F1-Score.
  • Smart Checkpoints: Automatically preserves the best-performing model (Best IoU) and prunes redundant files.

3. Image restoration/super-resolution pipline using Foundation model

• Retraining-free: We train the image restoration/super-resolution model EMCellFiner on 4M+ EM images, thus EMCellFiner has robust performance，can restore/super-resolution for most of EM images and make them finer.
• Single-image: We support restore/super-resolution in the GUI using GPU/CPU, and show in the GUI.
• Multi-image: We also support to restore/super-resolution the images in the folder, and output to another folder.

4. Tools

• Annotation Support: Built-in utility to convert Labelme JSON annotations to Semantic Segmentation masks.

📖Installation

To leverage the full potential of the EMCellFiner and EMCellFound foundation models, an NVIDIA GPU(We suggest > RTX3090) is highly recommended.

1. Environment Preparation

We recommend using Conda/miniconda to create an isolated environment. We suggest using python>=3.11 (We have successfully test the pipline in python version 3.8\3.9\3.10\3.11)

# Create a new environment named 'emcfsys' with Python> 3.11
conda create -n emcfsys python=3.11 -y
# Activate the environment
conda activate emcfsys

2. Install PyTorch with GPU Support

emcfsys requires PyTorch > 1.3. For optimal performance, we recommend PyTorch 2.0+. Choose the command matching your CUDA version:

# For Linux and Window:
pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu118

Or follow link to select the torch version https://pytorch.org/get-started/previous-versions/

3. Install napari and emcfsys

3.1 You can follow the napari GUI installation document:https://github.com/napari/napari Or use the follow pipline

pip install "napari[pyqt6, optional]"

Then you can install emcfsys via [pip]:

pip install emcfsys

You can also install emcfsys in the napari-plugin-store

At last, install necessary components: (Very important, don't forget to install these packages)

pip install labelme==5.9.1 numpy timm==1.0.22 opencv-python einops shapely==2.1.2 albumentations ninja Pillow seaborn safetensors huggingface-hub transformers imgviz==1.7.6

4. Code‑Based Fine‑Tuning with MMLab Backend

We provide model fine‑tuning implementations in both source‑code and Jupyter Notebook formats. This part can run independently of the GUI, but it requires additional installation of the MMLab‑series packages. Please refer to mmLab_backend_demo.ipynb.

📖 Quick Start

1. Use as a napari Plugin
2. Launch napari: napari
3. Navigate to: Plugins -> emcfsys
4. Load your image and select EMCellFiner for instant enhancement.

📖 Tutorial

All tutorials and feature descriptions can be found in the tutorial documentation (click me!).

License

Distributed under the terms of the [GNU GPL v3.0] license, "emcfsys" is free and open source software

Issues

If you encounter any problems, please [file an issue] along with a detailed description.

[]

Towards foundation models for EM images analysis. EMCellFiner and EMCellFound are two foundation models trained based on a 4 million EM images dataset.

If you use this project, please cite:

Yu Z, Guo J, Liu F, et al. EMCF ecosystem: Towards pretrained foundation model for electron microscopy image analysis[J]. bioRxiv, 2025: 2025.12. 09.693109.