pytorch-ood 0.0.9

A Library for Out-of-Distribution Detection with PyTorch

---

Python library to accelerate research in Out-of-Distribution Detection, as well as related fields such as Open-Set Recognition, Novelty Detection, Confidence Estimation and Anomaly Detection based on Deep Neural Networks (with PyTorch).

This library provides

• Objective Functions

• OOD Detection Methods

• Datasets used in academic literature

• Neural Network Architectures used in academic literature, as well as pretrained weights

• Useful Utilities

and was created with the aim to speed up research and to facilitate reproducibility. It is designed such that it should integrate seamlessly with frameworks that enable the scaling of model training, like pytorch-lightning.

Installation

pip install pytorch-ood

Required Dependencies

• torch

• torchvision

• scipy

• torchmetrics

Optional Dependencies

• libmr for the OpenMax Detector, which is currently broken. You will have to install cython and libmr afterwards manually.

• pandas for the Cub200 Dataset

Quick Start

Load model pre-trained with energy regularization, and predict on some dataset data_loader using Energy-based outlier scores.

from pytorch_ood.model import WideResNet
from pytorch_ood import NegativeEnergy
from pytorch_ood.utils import OODMetrics

# create Neural Network
model = WideResNet(pretrained="er-cifar10-tune").eval().cuda()

# create detector
detector = NegativeEnergy(model)

# evaluate
metrics = OODMetrics()

for x, y in data_loader:
    metrics.update(detector(x.cuda()), y)

print(metrics.compute())

Citing

pytorch-ood was presented on the CVPR Workshop on Human-centered Intelligent Services: Safe and Trustworthy. If you use pytorch-ood in a scientific publication, please consider citing us:

@article{kirchheim2022pytorch,
  author = {Kirchheim, Konstantin and Filax, Marco and Ortmeier, Frank},
  journal = {CVPR Workshop on Human-centered Intelligent Services: Safe and Trustworthy},
  number = {},
  pages = {},
  publisher = {IEEE},
  title = {PyTorch-OOD: A Library for Out-of-Distribution Detection based on PyTorch},
  year = {2022}
  }

or:

Kirchheim, Konstantin and Filax, Marco and Ortmeier, Frank, 2022. PyTorch-OOD: A Library for Out-of-Distribution Detection based on PyTorch (IEEE)

Implemented Algorithms

Implemented Detectors :

Detector	Description	Year	Ref
OpenMax	Implementation of the OpenMax Layer as proposed in the paper Towards Open Set Deep Networks.	2016	[1]
ODIN	ODIN is a preprocessing method for inputs that aims to increase the discriminability of the softmax outputs for In- and Out-of-Distribution data.	2018	[2]
Mahalanobis	This method calculates a class center for each class, and a shared covariance matrix from the data.	2018	[3]
Monte Carlo Dropout	Implements the Monte Carlo Dropout for OOD detection.	2022	[4]
Softmax Thresholding	Implements the Softmax Baseline for OOD detection.	2022	[5]
Energy-Based OOD Detection	Implements the Energy Score of Energy-based Out-of-distribution Detection.	2020	[6]

Implemented Objective Functions:

Objective Function	Description	Year	Ref
Objectosphere	Implementation of the paper Reducing Network Agnostophobia.	2016	[7]
Outlier Exposure	Implementation of the paper Deep Anomaly Detection With Outlier Exposure.	2018	[8]
Deep SVDD	Implementation of the Deep Support Vector Data Description from the paper Deep One-Class Classification.	2018	[9]
II Loss	Implementation of II Loss function from Learning a neural network-based representation for open set recognition.	2022	[10]
CAC Loss	Class Anchor Clustering Loss from Class Anchor Clustering: a Distance-based Loss for Training Open Set Classifiers	2022	[11]
Energy Regularization	Adds a regularization term to the cross-entropy that aims to increase the energy gap between IN and OOD samples.	2020	[12]
Center Loss	Generalized version of the Center Loss from the Paper A Discriminative Feature Learning Approach for Deep Face Recognition.	2022	[13]

Contributing

We encourage everyone to contribute to this project by adding implementations of OOD Detection methods, datasets etc, or check the existing implementations for bugs.

License

The code is licensed under Apache 2.0. We have taken care to make sure any third party code included or adapted has compatible (permissive) licenses such as MIT, BSD, etc. The legal implications of using pre-trained models in commercial services are, to our knowledge, not fully understood.

---

Reference

[1]

OpenMax (2016) Towards open set deep networks, CVPR

[2]

ODIN (2018) Enhancing the reliability of out-of-distribution image detection in neural networks, ICLR

[3]

Mahalanobis (2018) A simple unified framework for detecting out-of-distribution samples and adversarial attacks, NeurIPS

[4]

Monte Carlo Droput

[5]

Softmax Paper

[6]

Energy-Based OOD (2020) Energy-based Out-of-distribution Detection, NeurIPS

[7]

Object Sphere paper

[8]

Outlier Exposure paper

[9]

SVDD paper

[10]

IILoss paper

[11]

CACLoss Paper

[12]

Energy Regegularization Paper

[13]

CenterLoss Paper