Skip to main content

Semantic Segmentation using Deep Learning ONNX models packaged as *.czann files

Project description

napari-czann-segment

License PyPI Python Version napari hub

Semantic Segmentation of multidimensional images using Deep Learning ONNX models packaged as *.czann files.


This napari plugin was generated with Cookiecutter using @napari's cookiecutter-napari-plugin template.

Train on APEER and use model in Napari

Installation

Before installing, please setup a conda environment. If you have never worked with conda environments, go through this tutorial first.

You can then install napari-czann-segment via pip:

pip install napari-czann-segment

What does the plugin do

The plugin allows you to:

  • Use a *.czann file containing the Deep Neural Network (ONNX) for semantic segmentation and metadata
  • Segmentation will be applied per 2D plane for all dimensions
  • Processing larger multidimensional images it uses the cztile package to chunk the individual 2d arrays using a specific overlap.
  • multidimensional images will be processed plane-by-plane

What does the plugin NOT do

Before one can actually use a model it needs to be trained, which is NOT done by this plugin.

There are two main ways hwo such a model can be created:

  • Train the segmentation model fully automated on APEER and download the *.czann file
  • Train your model in a Jupyter notebook etc. and package it using the czmodel python package as an *.czann

Using this plugin

Sample Data

A test image and a *.czann model file can be downloaded here.

  • PGC_20X.ome.tiff --> use PGC_20X_nucleus_detector.czann to segment

In order to use this plugin the user has to do the following things:

  • Open the image using "File - Open Files(s)" (requires napari-aicsimageio plugin).
  • Click napari-czann-segment: Segment with CZANN model in the "Plugins" menu.
  • Select a czann file to use the model for segmentation.
  • metadata of the model will be shown (see example below)
Parameter Value Explanation
model_type ModelType.SINGLE_CLASS_SEMANTIC_SEGMENTATION see: czmodel for details
input_shape [1024, 1024, 1] tile dimensions of model input
output_shape [1024, 1024, 3] tile dimensions of model output
model_id ba32bc6d-6bc9-4774-8b47-20646c7cb838 unique GUID for that model
min_overlap [128, 128] tile overlap used during training (for this model)
classes ['background', 'grains', 'inclusions'] available classes
model_name APEER-trained model name of the model

Napari - Image loaded and czann selected

  • Adjust the minimum overlap for the tiling (optional, see cztile for details).
  • Select the layer to be segmented.
  • Toggle Use GPU for inference checkbox to enable / disable using a GPU (Nvidia) for the segmentation (experimental feature).
  • Press Segment Selected Image Layer to run the segmentation.

Napari - Image successfully segmented

A successful is obviously only the starting point for further image analysis steps to extract the desired numbers from the segmented image. Another example is shown below demonstrating a simple "Grain Size Analysis" using a deep-learning model trained on APEER used in napari

Napari - Simple Grain Size Analysis

Remarks

IMPORTANT: Currently the plugin only supports using models trained on a single channel image. Therefore, make sure that during the training on APEER or somewhere else the correct inputs images are used. It is quite simple to train a single RGB image, which actually has three channels, load this image in napari and notice only then that the model will not work, because the image will 3 channels inside napari.

  • Only the CPU will be used for the inference using the ONNX runtime for the ONNX-CPU runtime
  • GPUs are supported but require the ONNX-GPU runtime and the respective CUDA libraries.
  • Please check the YAML for an example environment with GPU support.
  • See also pytorch for instruction on how to install pytorch

For developers

  • Please clone this repository first using your favorite tool.

  • Ideally one creates a new conda environment or use an existing environment that already contains Napari.

Feel free to create a new environment using the YAML file at your own risk:

cd the-github-repo-with-YAML-file
conda env create --file conda_env_napari_czann_segment.yml
conda activate napari_czmodel
  • Install the plugin locally

Please run the following command:

pip install -e .

To install latest development version:

pip install git+https://github.com/sebi06/napari_czann_segment.git

Contributing

Contributions and Feedback are very welcome.

License

Distributed under the terms of the BSD-3 license, "napari-czann-segment" is free and open source software

Issues

If you encounter any problems, please file an issue along with a detailed description.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

napari-czann-segment-0.0.18.tar.gz (11.8 MB view details)

Uploaded Source

Built Distribution

napari_czann_segment-0.0.18-py3-none-any.whl (28.1 kB view details)

Uploaded Python 3

File details

Details for the file napari-czann-segment-0.0.18.tar.gz.

File metadata

  • Download URL: napari-czann-segment-0.0.18.tar.gz
  • Upload date:
  • Size: 11.8 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.5

File hashes

Hashes for napari-czann-segment-0.0.18.tar.gz
Algorithm Hash digest
SHA256 aa8bf22f02c914dfbf43fa59785e8fdb77cb55d9a18639ef13b3275ab1e628da
MD5 b9a6564392028015256129f318a86b36
BLAKE2b-256 b8e21636415889b43d18e7e172f9088344b6ad5b146394ec8fd349ee7da54aad

See more details on using hashes here.

File details

Details for the file napari_czann_segment-0.0.18-py3-none-any.whl.

File metadata

File hashes

Hashes for napari_czann_segment-0.0.18-py3-none-any.whl
Algorithm Hash digest
SHA256 ac98eba002be4fba4a85ae60e222dd7d86090ac3720ffd4a1e883644ef2afd3e
MD5 f069bb2188bd95a788654243de1cedf4
BLAKE2b-256 14942c341d37b354c6132be10d5d5189d538055438d8d10afc8c03c8d56454a3

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page