PyTorch Out-of-Distribution Detection
Project description
PyTorch Out-of-Distribution Detection
Python library to accelerate research in Out-of-Distribution Detection, as well as related fields such as Open-Set Recognition, Novelty Detection, Confidence Estimation and Anomaly Detection based on Deep Neural Networks (with PyTorch).
This library provides
- Objective Functions
- OOD Detection Methods
- Datasets used in academic literature
- Neural Network Architectures used in academic literature, as well as pretrained weights
- Useful Utilities
and was created with the aim to speed up research and to facilitate reproducibility. It is designed such that it should integrate seamlessly with frameworks that enable the scaling of model training, like pytorch-lightning.
Installation
pip install pytorch-ood
Required Dependencies
torch
torchvision
scipy
torchmetrics
Optional Dependencies
libmr
for OpenMax Detectorpandas
for the Cub200 Dataset
For OpenMax, you will have to install libmr
, which is currently broken.
You will have to install cython
and libmr
afterwards manually.
Quick Start
Load model pre-trained with energy regularization, and predict on some dataset data_loader
using
Energy-based outlier scores.
from pytorch_ood.model import WideResNet
from pytorch_ood.detector import NegativeEnergy
from pytorch_ood.utils import OODMetrics
# create Neural Network
model = WideResNet.from_pretrained("er-cifar10-tune").eval().cuda()
# create detector
detector = NegativeEnergy(model)
# evaluate
metrics = OODMetrics()
for x, y in data_loader:
metrics.update(detector(x.cuda()), y)
print(metrics.compute())
Implemented Detectors
Detector | Reference |
---|---|
OpenMax | [1] |
ODIN | [2] |
Mahalanobis | [3] |
Monte Carlo Dropout | [4] |
Softmax Thresholding Baseline | [5] |
Energy-Based OOD Detection | [6] |
Implemented Objective Functions
Objective Function | Reference |
---|---|
Objectosphere | [7] |
Outlier Exposure | [8] |
Deep SVDD | [9] |
II Loss | [10] |
CAC Loss | [11] |
Energy Regularization | [6] |
Center Loss | [12] |
Cite pytorch-ood
If you use this package in your research, please consider citing it. To appear in
@article{kirchheim2022pytorch,
author = {Kirchheim, Konstantin and Filax, Marco and Ortmeier, Frank},
journal = {CVPR Workshop for Human-centered Intelligent Services: Safety and Trustworthy},
number = {},
pages = {},
publisher = {IEEE},
title = {PyTorch-OOD: A Library for Out-of-Distribution Detection based on PyTorch},
year = {2022}
}
Roadmap
- add additional OOD methods
- add more datasets, e.g. for audio and video
- implement additional tests
- migrate to DataPipes
Contributing
We encourage everyone to contribute to this project by adding implementations of OOD Detection methods, datasets etc, or check the existing implementations for bugs.
License
The code is licensed under Apache 2.0. We have taken care to make sure any third party code included or adapted has compatible (permissive) licenses such as MIT, BSD, etc. The legal implications of using pre-trained models in commercial services are, to our knowledge, not fully understood.
References
[1] OpenMax (2016) Towards open set deep networks, CVPR
[2] ODIN (2018) Enhancing the reliability of out-of-distribution image detection in neural networks, ICLR
[3] Mahalanobis (2018) A simple unified framework for detecting out-of-distribution samples and adversarial attacks, NEURIPS
[6] Energy-Based OOD (2020) Energy-based Out-of-distribution Detection, NEURIPS
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Hashes for pytorch_ood-0.0.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c805f696e04c34be25f684a8ee69c10b85bc548d6e1702c4f80e09a2136c1533 |
|
MD5 | 12dd4cc1404c86a09c44af51ac911c50 |
|
BLAKE2b-256 | 93e0e2cbc501fbe0e9da9ba0f41b98f45cad2fa9fa587f768392ce6dd4c8bf63 |