Skip to main content

Numpy-based implementation of common performance metrics for semantic image segmentation

Project description

UKIS ukis-metrics

ukis-metrics codecov Upload Python Package PyPI version GitHub license Code Style

A pure Numpy-based implementation of the most common performance metrics for semantic image segmentation.

Installation

pip install ukis_metrics
>>> import ukis_metrics
>>> ukis_metrics.__version__
'0.1.3'

Why?

Simply because we wanted a lightweight and fast alternative to scikit learn for tracking during training.

execution_time.py compares the execution time of ukis-metrics with sklearn. Here's an example output:

Shape of array: (256, 256, 2)

                                        ### Metrics execution time [s] ###

                ukis-metrics                            sklearn metrics         speed gain
acc             0.001900                                0.007627                4.01
rec             0.001716                                0.024509                14.28
pre             0.001815                                0.025021                13.79
f1              0.001798                                0.024770                13.78
iou             0.001797                                0.024247                13.49
kap             0.001824                                0.034577                18.96

Workings and included metrics

In a first step the true positives tp, true negatives tn, false positives fp, false negatives fn and the number of valid pixels n_valid_pixels are computed. These values are then used to compute the following metrics:

  • Accuracy [1]:
    acc = \frac{tp + tn}{tp + fn + fp + tn}
    
  • Recall [1]:
    rec = \frac{tp}{tp + fn}
    
  • Precision [1]:
    prec = \frac{tp}{tp + fp}
    
  • F1-score [2]:
    F1 = \frac{2 * prec * rec}{prec + rec}
    
  • IoU [3]:
    IoU = \frac{tp}{tp + fp + fn}
    
  • Kappa: The computation of the Kappa-score incorporates several steps. Please refer to [4] for the full documentation

How to use

Simply pass a Numpy ndarray to get a dict containing the tpfptnfn and pass the dict to segmentation_metrics(tpfptnfn):

import ukis_metrics.seg_metrics as segm
import numpy as np
# ndarray containing the reference data, e.g.
shape = (256, 256, 1)
y_true = np.ones(shape)
# ndarray containing the model predicions, e.g.
y_pred = np.ones(shape)
# get tp, fp, tn, fn an n_valid_pixel
tpfptnfn = segm.tpfptnfn(y_true, y_pred, None)
metrics = segm.segmentation_metrics(tpfptnfn)

So far these metrics were only used for binary classification, although one should be able to use them for multiclass segmentation too, if the slices for a given class are provided individually.

There is a jupyter notebook showing how ukis-metrics can be used and how it could be extended for multiclass problems.

References

Contributors

The UKIS team creates and adapts libraries which simplify the usage of satellite data. Our team includes (in alphabetical order):

  • Fichtner, Florian
  • Helleis, Max
  • Martinis, Sandro
  • Wieland, Marc

German Aerospace Center (DLR)

Licenses

This software is licensed under the Apache 2.0 License.

Copyright (c) 2021 German Aerospace Center (DLR) * German Remote Sensing Data Center * Department: Geo-Risks and Civil Security

Changelog

See changelog.

What is UKIS?

The DLR project Environmental and Crisis Information System (the German abbreviation is UKIS, standing for Umwelt- und Kriseninformationssysteme aims at harmonizing the development of information systems at the German Remote Sensing Data Center (DFD) and setting up a framework of modularized and generalized software components.

UKIS is intended to ease and standardize the process of setting up specific information systems and thus bridging the gap from EO product generation and information fusion to the delivery of products and information to end users.

Furthermore, the intention is to save and broaden know-how that was and is invested and earned in the development of information systems and components in several ongoing and future DFD projects.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ukis_metrics-0.1.3.tar.gz (9.0 kB view hashes)

Uploaded Source

Built Distribution

ukis_metrics-0.1.3-py3-none-any.whl (9.2 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page