torch-uncertainty 0.13.0

TorchUncertainty is a package designed to help leverage uncertainty quantification techniques to make deep neural networks more reliable. It aims to be collaborative and to include as many methods as possible — so reach out to add yours!

TorchUncertainty is in active development: expect changes, but reach out and contribute if you are interested in the project! Please raise an issue if you have any bugs or difficulties and join the Discord server.

Our website and documentation are available at torch-uncertainty.github.io. :books:

TorchUncertainty contains the official implementations of multiple papers from major machine-learning and computer-vision conferences and was featured in tutorials at WACV 2024, HAICON 2024 and ECCV 2024.

Torch-Uncertainty was published at NeurIPS D&B 2025. Please consider citing the paper if the framework is helpful for your research.

This package provides a multi-level API, including:

• easy-to-use :zap: Lightning uncertainty-aware training & evaluation routines for 4 tasks: classification, probabilistic and pointwise regression, segmentation, and pixelwise regression (e.g., monocular depth estimation);
• fully automated evaluation of models with proper scores, calibration, selective classification, out-of-distribution detection and distribution-shift metrics — see the Evaluating Models page;
• layers, models, metrics, & losses ready to drop into your networks;
• scikit-learn-style post-processing methods such as temperature scaling and conformal prediction;
• transformations and augmentations, including a set of corruptions used to build "corrupted datasets" hosted on HuggingFace.

Have a look at the Reference page or the API reference for a more exhaustive list of the implemented methods, datasets, metrics, etc.

:gear: Installation

TorchUncertainty requires Python 3.10 or greater. Install the desired PyTorch version in your environment. Then, install the package from PyPI:

pip install torch-uncertainty

The installation procedure for contributors is different: have a look at the contribution page.

:whale: Docker image for contributors

For contributors running experiments on cloud GPU instances, we provide a pre-built Docker image with all the necessary dependencies and a Dockerfile to build your own custom images. This lets you launch an experiment-ready container with minimal setup. See DOCKER.md for further details.

:racehorse: Quickstart

We make a quickstart available at torch-uncertainty.github.io/quickstart.

:books: Implemented methods

TorchUncertainty currently supports classification, probabilistic and pointwise regression, segmentation and pixelwise regression (such as monocular depth estimation).

We also provide the following methods:

Uncertainty quantification models

To date, the following deep-learning uncertainty quantification methods have been implemented. Click :inbox_tray: on the methods for tutorials:

• Deep Ensembles, BatchEnsemble, Masksembles, & MIMO
• MC-Dropout
• Packed-Ensembles (see the blog post)
• Mixup Ensembles and MixupMP
• Variational Bayesian Neural Networks
• Checkpoint Ensembles & Snapshot Ensembles
• Stochastic Weight Averaging & Stochastic Weight Averaging Gaussian
• Deep Evidential Classification & Regression
• Regression with Beta Gaussian NLL Loss
• Quantile Regression with Pinball Loss
• Test-time adaptation with Zero

Augmentation methods

The following data augmentation methods have been implemented:

• Mixup, MixupIO, RegMixup, and WarpingMixup
• modernized corruptions to evaluate model performance under distribution shift

Post-processing methods

To date, the following post-processing methods have been implemented:

• Temperature, vector, and matrix scaling, & Dirichlet calibration
• Histogram binning, isotonic regression, and Bayesian binning into quantiles
• Conformal Predictions with APS and RAPS for classification and CQR for regression.
• DEUP (Direct Epistemic Uncertainty Prediction) — error-predictor post-hoc epistemic scores
• Monte Carlo Batch Normalization
• Laplace approximation through the Laplace library

Metrics

The following metrics have been implemented (see the Evaluating Models page for the full breakdown per task):

• proper scores: Brier score, Categorical NLL, distribution NLL
• out-of-distribution detection: AUROC, AUPR, FPR95
• selective classification with OOD: SCOD-AURC, SCOD-AUGRC, SCOD-CovAt5%Risk, SCOD-RiskAt80%Cov
• calibration: ECE with improved reliability diagrams, SmoothECE, Adaptive ECE, Classwise ECE, and QuantileCalibrationError for regression
• selective classification: AURC, AUGRC, CovAt5%Risk, RiskAt80%Cov
• conformal predictions: CoverageRate, SetSize
• prediction diversity: Disagreement, Entropy, MutualInformation, VariationRatio
• multiple pixelwise-regression and segmentation metrics

Official Implementations

It includes the official codes of the following papers:

• Packed-Ensembles for Efficient Uncertainty Estimation - ICLR 2023 - Tutorial
• LP-BNN: Encoding the latent posterior of Bayesian Neural Networks for uncertainty quantification - IEEE TPAMI 2023
• MUAD: Multiple Uncertainties for Autonomous Driving, a benchmark for multiple uncertainty types and tasks - BMVC 2022

Tutorials

Check out all our tutorials at torch-uncertainty.github.io/auto_tutorials.

Experiments

We provide some configuration files in the experiments folder. You may find more examples in other repositories related to the Torch-Uncertainty organization, such as the Packed-Ensembles reproducibility repo. or the Torch-Uncertainty reproducibility repo.

Experiment results are tracked with MLflow. After running an experiment, launch mlflow ui (or mlflow ui --backend-store-uri sqlite:///logs/<name> for the SQLite configs) and open http://localhost:5000 to compare runs, inspect metric curves, and browse logged artifacts. See the MLflow Guide for full details.

:telescope: Projects using TorchUncertainty

The following projects use TorchUncertainty:

• Towards Understanding and Quantifying Uncertainty for Text-to-Image Generation - CVPR 2025
• Towards Understanding Why Label Smoothing Degrades Selective Classification and How to Fix It - ICLR 2025
• A Symmetry-Aware Exploration of Bayesian Neural Network Posteriors - ICLR 2024

If you are using TorchUncertainty in your project, please let us know, and we will add your project to this list!

Citation

If you use this software, please cite its corresponding paper:

@inproceedings{lafage2025torch_uncertainty,
    title={Torch-Uncertainty: A Deep Learning Framework for Uncertainty Quantification},
    author={Lafage, Adrien and Laurent, Olivier and Gabetni, Firas and Franchi, Gianni},
    booktitle={NeurIPS Datasets and Benchmarks Track},
    year={2025}
}