pytorch-ood 0.0.4

PyTorch Out-of-Distribution Detection

PyTorch Out-of-Distribution Detection

Python library to accelerate research in Out-of-Distribution Detection, as well as related fields such as Open-Set Recognition, Novelty Detection, Confidence Estimation and Anomaly Detection based on Deep Neural Networks (with PyTorch).

This library provides

• Objective Functions
• OOD Detection Methods
• Datasets used in academic literature
• Neural Network Architectures used in academic literature, as well as pretrained weights
• Useful Utilities

and was created with the aim to speed up research and to facilitate reproducibility. It is designed such that it should integrate seamlessly with frameworks that enable the scaling of model training, like pytorch-lightning.

Installation

pip install pytorch-ood

Required Dependencies

• torch
• torchvision
• scipy
• torchmetrics

Optional Dependencies

• libmr for OpenMax Detector
• pandas for the Cub200 Dataset

For OpenMax, you will have to install libmr, which is currently broken. You will have to install cython and libmr afterwards manually.

Quick Start

Load model pre-trained with energy regularization, and predict on some dataset data_loader using Energy-based outlier scores.

from pytorch_ood.model import WideResNet
from pytorch_ood.detector import NegativeEnergy
from pytorch_ood.utils import OODMetrics

# create Neural Network
model = WideResNet.from_pretrained("er-cifar10-tune").eval().cuda()

# create detector
detector = NegativeEnergy(model)

# evaluate
metrics = OODMetrics()

for x, y in data_loader:
    metrics.update(detector(x.cuda()), y)

print(metrics.compute())

Implemented Detectors

Detector	Reference
OpenMax	[1]
ODIN	[2]
Mahalanobis	[3]
Monte Carlo Dropout	[4]
Softmax Thresholding Baseline	[5]
Energy-Based OOD Detection	[6]

Implemented Objective Functions

Objective Function	Reference
Objectosphere	[7]
Outlier Exposure	[8]
Deep SVDD	[9]
II Loss	[10]
CAC Loss	[11]
Energy Regularization	[6]
Center Loss	[12]

Cite pytorch-ood

If you use this package in your research, please consider citing it. To appear in

@article{kirchheim2022pytorch,
	author = {Kirchheim, Konstantin and Filax, Marco and Ortmeier, Frank},
	journal = {CVPR Workshop for Human-centered Intelligent Services: Safety and Trustworthy},
	number = {},
	pages = {},
	publisher = {IEEE},
	title = {PyTorch-OOD: A Library for Out-of-Distribution Detection based on PyTorch},
	year = {2022}
}

Roadmap

• add additional OOD methods
• add more datasets, e.g. for audio and video
• implement additional tests
• migrate to DataPipes

Contributing

We encourage everyone to contribute to this project by adding implementations of OOD Detection methods, datasets etc, or check the existing implementations for bugs.

License

The code is licensed under Apache 2.0. We have taken care to make sure any third party code included or adapted has compatible (permissive) licenses such as MIT, BSD, etc. The legal implications of using pre-trained models in commercial services are, to our knowledge, not fully understood.

References

[1] OpenMax (2016) Towards open set deep networks, CVPR

[2] ODIN (2018) Enhancing the reliability of out-of-distribution image detection in neural networks, ICLR

[3] Mahalanobis (2018) A simple unified framework for detecting out-of-distribution samples and adversarial attacks, NEURIPS

[4] ...

[5] ...

[6] Energy-Based OOD (2020) Energy-based Out-of-distribution Detection, NEURIPS

[7] ...

[8] ...

[9] ...

[10] ...

[11] ...

[12] ...