Predictive multiplicity for deep learning
Project description
multiplicity
Library for evaluating predictive multiplicity of deep leearning models.
Setup
pip install multiplicity
Quickstart
The library provides a method to estimate viable prediction intervals: prediction intervals that are robust to a small change in model's loss at training or evaluation time.
Import the library:
from multiplicity import torch as multiplicity
Suppose we have a trained torch binary classifier which outputs softmax probabilities:
model(x) # 0.75
Specify to the deviation of which metric we want to be robust to, and on which dataset:
robustness_criterion = multiplicity.ZeroOneLossCriterion(train_loader)
Then, we can compute the viable prediction range for a given example x like so:
lb, pred, ub = multiplicity.viable_prediction_range(
model=model,
target_example=x,
robustness_criterion=robustness_criterion,
criterion_thresholds=epsilon,
)
# lb=0.71, pred=0.75, ub=0.88
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
multiplicity-2.1.0.tar.gz
(4.6 kB
view hashes)
Built Distribution
Close
Hashes for multiplicity-2.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e9d352eb892861ddfa07c166fd2382f62cec9077bc87c7abca4fddbcfc5ae027 |
|
MD5 | dde026f608232c0590dd8ee8eecaeec2 |
|
BLAKE2b-256 | f0c54c2c65d2048f59bf32a40b73e207b4aa8f03ff87760f1b8bdc27e53ea20d |