Fiducio

Fiducio is a model-agnostic Python library of post-hoc calibrators for 2D and 3D semantic segmentation. Give it the logits or probabilities of any segmentation model and a labelled calibration set, and a calibrator returns better-calibrated probabilities without retraining the segmentation model. It implements temperature, vector, matrix and Dirichlet scaling, their translation-invariant variants, and the class-conditional (CDC, CMSap, CMSop) calibrators introduced in our paper Rethinking Post-Hoc Calibration in Semantic Segmentation, accepted at TMLR. Whether the segmentation is preserved depends on the calibrator; see the guarantees below.

It works with PyTorch U-Net, nnU-Net, SegFormer or any model that produces logits: Fiducio only ever sees (B, C, *spatial) tensors and never needs to know the architecture.

Top-1 reliability diagrams on held-out synthetic data (regenerate with examples/readme_figure.py).

Install

Fiducio is available on PyPI:

pip install fiducio

or, in a uv application:

uv add fiducio

Requires Python ≥ 3.10 and PyTorch ≥ 2.10. The core install depends only on numpy and torch. Optional extras: fiducio[plots] (matplotlib reliability diagrams), fiducio[docs], fiducio[dev], fiducio[all].

To install the development version from GitHub:

pip install "git+https://github.com/fiducio-ai/Fiducio.git"

For development from a checkout, use the committed uv lockfile:

uv sync --locked
uv run --no-sync pytest

This uses CPU PyTorch for development. See CONTRIBUTING.md for the pinned uv version, documentation and build commands.

Quick start

import torch
from fiducio import TemperatureScaling, load_calibrator

# From a held-out, labelled calibration set:
logits = torch.randn(8, 4, 64, 64)  # (B, C, H, W) model logits
labels = torch.randint(0, 4, (8, 64, 64))  # (B, H, W) labels

calibrator = TemperatureScaling(input_type="logits").fit(logits, labels)

# Apply to new predictions (returns calibrated probabilities):
probs = calibrator.transform(torch.randn(2, 4, 64, 64))

# Save and reload (loads on CPU by default):
calibrator.save("calibrator.pt")
calibrator = load_calibrator("calibrator.pt")

Every calibrator exposes the same API: fit(predictions, targets, mask=None), transform, predict_proba, fit_transform, save, and the top-level load_calibrator.

Accepted tensor shapes

The class axis is always dimension 1.

Inputs may be logits or probs (set input_type). Binary segmentation is two channels (C = 2). mask and ignore_index exclude voxels from fitting.

Calibrators

Translation-invariant means the output is unchanged when the same constant is added to every input logit of a voxel.

Every calibrator also exposes decision_function (calibrated logits) and a configurable optimizer ("adam" default, or "lbfgs").

Early stopping

Adam-fitted calibrators support the paper's recipe of early stopping on a validation set (with an optional learning-rate decay on plateaus):

from fiducio import OrderPreservingMatrixScaling

calibrator = OrderPreservingMatrixScaling(max_iter=2000, patience=20, lr_patience=10)
calibrator.fit(
    logits,
    labels,
    val_predictions=val_logits,
    val_targets=val_labels,  # held out from both
)

The iterate with the best validation NLL is kept. The default fixed budget (max_iter) is short, so it can underfit expressive calibrators; prefer a larger max_iter together with patience for CDC, CMSap and CMSop. This reproduces only the Adam + validation-NLL early-stopping recipe; the paper's case batching, class weighting, schedules, clipping and hyperparameter search are not part of the library (see the implementation guide).

Paper method mapping

CDC, CMSap and CMSop are class-conditional: they fit one affine map per uncalibrated top class, all experts optimized jointly by a single optimizer minimizing one cross-entropy loss over every voxel at once, with L2 regularization (lambda_reg / mu_reg) applied to the affine map induced in the common logit space rather than to the raw per-expert parameters — this keeps the regularization meaningful for CMSap/CMSop, whose parameters are non-negative margins/gaps rather than raw matrix entries. Pass independent_experts=True to instead fit each expert in its own optimization loop on only the voxels routed to it.

Calibration metrics are included: negative_log_likelihood, expected_calibration_error (ECE, population-weighted), average_calibration_error (ACE, unweighted over the same bins — doesn't let a sparsely populated bin get drowned out by a large one), brier_score, reliability_curve, plus an optional reliability-diagram plot (fiducio.plots.reliability_diagram, needs fiducio[plots]).

BA-ECE (boundary-aware ECE), per-case metric aggregation, the DSC/flip-rate audit and the paper's paired hierarchical-bootstrap tests are not included; the shipped metrics are top-1 pooled point estimates. average_calibration_error is the unweighted mean over non-empty uniform bins (not the adaptive ACE of other work) and is also available as unweighted_calibration_error.

Documentation

The paper project page presents the methods, interactive illustrations and published results.

The paper implementation guide records method mapping, numerical-reference coverage, fitting limits and the remaining steps needed to reproduce the paper's experiments. For ensemble inputs, see examples/ensemble_pooling.py.

Results in the paper

The paper compares these calibrators on real segmentation benchmarks. The reliability diagrams below (paper appendix) show the uncalibrated ensemble pooling (p0, p̄) against the best calibrator per dataset; the exact protocol, datasets and metrics are in the paper. This repository ships the calibrators, not the fitted models or benchmark pipelines, so it does not regenerate these figures (see the implementation guide).

Full guide and API reference: https://fiducio-ai.github.io/Fiducio/

• Installation
• Quickstart
• Concepts
• Calibrators
• Examples

Citation

Fiducio accompanies the paper Rethinking Post-Hoc Calibration in Semantic Segmentation (Kirscher et al., Transactions on Machine Learning Research, 2026; OpenReview, preprint: arXiv:2607.01902). If you use Fiducio in your research, please cite it using the metadata in CITATION.cff, or:

@article{kirscher2026rethinking,
  title   = {Rethinking Post-Hoc Calibration in Semantic Segmentation},
  author  = {Kirscher, Tristan and Kahl, Kim-Celine and Kovacs, Balint and
             Rokuss, Maximilian and Maier-Hein, Klaus and Coubez, Xavier and
             Meyer, Philippe and Faisan, Sylvain},
  journal = {Transactions on Machine Learning Research},
  issn    = {2835-8856},
  year    = {2026},
  url     = {https://openreview.net/forum?id=xwNoSNxgxV}
}

Status

Fiducio is beta (0.x): usable and tested, with an API that may still change before 1.0. It is not certified for safety-critical or clinical use.

License

Apache-2.0.