SkeletonMetrics
Python package that evaluates the topological accuracy of a predicted neuron segmentation by comparing it to a set of ground truth skeletons (i.e. graphs). Topological errors (e.g. splits and merges) are detected by examining skeleton edges and checking if the corresponding nodes belong to the same object in the segmentation. Once the accuracy of each edge has been determined, several skeleton-based metrics are computed to quantify the topological accuracy.
Note: This repository is an implementation of the skeleton-based metrics described in High-Precision Automated Reconstruction of Neurons with Flood-filling Networks
Overview
The pipeline for computing skeleton metrics consists of three main steps:
1. Label Graphs: Nodes in ground truth graphs are labeled with segmentation IDs.
2. Error Detection: Compare labels of neighboring nodes to detect mistakes.
3. Compute Metrics: Update graph structure by removing omit nodes and compute skeleton-based metrics.
Figure: Visualization of skeleton metric computation pipeline, see Method section for description of each step.
Method
Step 1: Label Graphs
The process starts with a collection of ground truth graphs, each stored as an individual SWC file, where the "xyz" attribute represents voxel coordinates in an image. Each ground truth graph is loaded and represented as a custom NetworkX graph with these coordinates as a node-level attribute. The nodes of each graph are then labeled with their corresponding segment IDs from the predicted segmentation.
Figure: On the left, ground truth graphs are superimposed on a segmentation where colors represent segment IDs. On the right, the nodes of the graphs have been labeled with the corresponding segment IDs.
Step 2: Error Detection
Figure: From top to bottom: correct edge (nodes have same segment ID), omit edge (at least one node does not have a segment ID), split edge (nodes have different segment IDs), merged edge (segment intersects with multiple graphs).
Step 3: Compute Metrics
Lastly, we compute the following skeleton-based metrics:
- # Splits: Number of connected components (minus 1) in a graph after removing omit and split edges.
- # Merges: Number of merge mistakes.
- % Split Edges: Percentage of split edges.
- % Omit Edges: Percentage of omit edges.
- % Merged Edges: Percentage of merged edges.
- Expected Run Length (ERL): Expected run length of graph after removing omit, split, and merged edges.
- Normalized ERL: ERL normalized by the total run length of the graph.
- Edge Accuracy: Percentage of edges that are correct.
- Split Rate: Run length of the graph divided by number of splits.
- Merge Rate: Run length of the graph divided by number of merges.
Installation
To use the software, in the root directory, run
pip install -e .
Usage
Here is a simple example of evaluating a predicted segmentation.
from segmentation_skeleton_metrics.evaluate import evaluate
from segmentation_skeleton_metrics.utils.img_util import TiffImage
# Initializations
output_dir = "./"
segmentation_path = "path-to-predicted-segmentation"
fragments_pointer = "path-to-predicted-skeletons"
groundtruth_pointer = "path-to-groundtruth-skeletons"
# Run
segmentation = TiffImage(segmentation_path)
evaluate(
groundtruth_pointer,
segmentation,
output_dir,
fragments_pointer=fragments_pointer,
)
Figure: Example of printouts generated after running evaluation.
Note: this package can also be used to evaluate a segmentation in which split mistakes have been corrected.
Contact Information
For any inquiries, feedback, or contributions, please do not hesitate to contact us. You can reach us via email at anna.grim@alleninstitute.org or connect on LinkedIn.
License
segmentation-skeleton-metrics is licensed under the MIT License.
1 maintainer
Release files for segmentation-skeleton-metrics 5.9.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| segmentation_skeleton_metrics-5.9.5.tar.gz | 730.2 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| segmentation_skeleton_metrics-5.9.5-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 768.7 kB
Release files / segmentation_skeleton_metrics-5.9.5.tar.gz
| Download URL | segmentation_skeleton_metrics-5.9.5.tar.gz |
|---|---|
| Size | 730.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
df4ed9a831ba2eb456f5aed98d0bb3f2e7bd85a7b5427547eca0c67e1361d4f6
|
|
BLAKE2b-256 checksum How to use checksums |
10fe254627911fa5eccc55cad2e300968ae1c412d77c4500ec254f1945e3078d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Release files / segmentation_skeleton_metrics-5.9.5-py3-none-any.whl
| Download URL | segmentation_skeleton_metrics-5.9.5-py3-none-any.whl |
|---|---|
| Size | 38.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
37d43938b4e4de689c889e7c7aa758ef0ff10eb224cb461c697d1acbfa0ecdd4
|
|
BLAKE2b-256 checksum How to use checksums |
6970548e559e0b702b7fc849d8b2b8a4a9997457a06e6b6b82aae4032890f097
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
