Semantic Segmentation using Deep Learning ONNX models packaged as *.czann files
Semantic Segmentation of multidimensional images using Deep Learning ONNX models packaged as *.czann files.
Before installing, please setup a conda environment. If you have never worked with conda environments, go through this tutorial first.
You can then install
napari-czann-segment via pip:
pip install napari-czann-segment
What does the plugin do
The plugin allows you to:
- Use a *.czann file containing the Deep Neural Network (ONNX) for semantic segmentation and metadata
- Segmentation will be applied per 2D plane for all dimensions
- Processing larger multidimensional images it uses the cztile package to chunk the individual 2d arrays using a specific overlap.
- multidimensional images will be processed plane-by-plane
What does the plugin NOT do
Before one can actually use a model it needs to be trained, which is NOT done by this plugin.
There are two main ways hwo such a model can be created:
- Train the segmentation model fully automated on APEER and download the *.czann file
- Train your model in a Jupyter notebook etc. and package it using the czmodel python package as an *.czann
Using this plugin
A test image and a *.czann model file can be downloaded here.
In order to use this plugin the user has to do the following things:
- Open the image using "File - Open Files(s)" (requires napari-aicsimageio plugin).
- Click napari-czann-segment: Segment with CZANN model in the "Plugins" menu.
- Select a czann file to use the model for segmentation.
- metadata of the model will be shown (see example below)
|see: czmodel for details
|[1024, 1024, 1]
|tile dimensions of model input
|[1024, 1024, 3]
|tile dimensions of model output
|unique GUID for that model
|tile overlap used during training (for this model)
|['background', 'grains', 'inclusions']
|name of the model
- Adjust the minimum overlap for the tiling (optional, see cztile for details).
- Select the layer to be segmented.
- Toggle Use GPU for inference checkbox to enable / disable using a GPU (Nvidia) for the segmentation (experimental feature).
- Press Segment Selected Image Layer to run the segmentation.
A successful is obviously only the starting point for further image analysis steps to extract the desired numbers from the segmented image. Another example is shown below demonstrating a simple "Grain Size Analysis" using a deep-learning model trained on APEER used in napari
IMPORTANT: Currently the plugin only supports using models trained on a single channel image. Therefore, make sure that during the training on APEER or somewhere else the correct inputs images are used. It is quite simple to train a single RGB image, which actually has three channels, load this image in napari and notice only then that the model will not work, because the image will 3 channels inside napari.
- Only the CPU will be used for the inference using the ONNX runtime for the ONNX-CPU runtime
- GPUs are supported but require the ONNX-GPU runtime and the respective CUDA libraries.
- Please check the YAML for an example environment with GPU support.
- See also pytorch for instruction on how to install pytorch
Please clone this repository first using your favorite tool.
Feel free to create a new environment using the YAML file at your own risk:
conda env create --file conda_env_napari_czann_segment.yml
conda activate napari_czmodel
- Install the plugin locally
Please run the following command:
pip install -e .
To install latest development version:
pip install git+https://github.com/sebi06/napari_czann_segment.git
Contributions and Feedback are very welcome.
Distributed under the terms of the BSD-3 license, "napari-czann-segment" is free and open source software
If you encounter any problems, please file an issue along with a detailed description.
Release history Release notifications | RSS feed
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Hashes for napari-czann-segment-0.0.18.tar.gz
Hashes for napari_czann_segment-0.0.18-py3-none-any.whl