Skip to main content

Quick access to uncertainty and confidence of Keras networks.

Project description

UNCERTAINTY WIZARD

Documentation Status PyPI

Uncertainty wizard is a plugin on top of tensorflow.keras, allowing to easily and efficiently create uncertainty-aware deep neural networks:

  • Plain Keras Syntax: Use the layers and APIs you know and love.
  • Conversion from keras: Convert existing keras models into uncertainty aware models.
  • Smart Randomness: Use the same model for point predictions and sampling based inference.
  • Fast ensembles: Train and evaluate deep ensembles lazily loaded and using parallel processing.
  • Super easy setup: Pip installable. Only tensorflow as dependency.

Installation

It's as easy as pip install uncertainty-wizard

Requirements

  • tensorflow >= 2.3.0
  • python 3.6* / 3.7 / 3.8

*python 3.6 requires to pip install dataclasses

Documentation

Our documentation is deployed to uncertainty-wizard.readthedocs.io. In addition, as uncertainty wizard has a 100% docstring coverage on public method and classes, your IDE will be able to provide you with a good amount of docs out of the box.

You may also want to check out the technical tool paper (preprint), describing uncertainty wizard functionality and api as of version 0.1.0.

Examples

A set of small and easy examples, perfect to get started can be found in the models user guide and the quantifiers user guide. Larger and examples are also provided - and you can run them in colab right away. You can find them here: List of jupyter examples.

Authors and Papers

Uncertainty wizard was developed by Michael Weiss and Paolo Tonella at USI (Lugano, Switzerland). If you use it for your research, please cite these papers:

@inproceedings{Weiss2021FailSafe,  
  title={Fail-Safe Execution of Deep Learning based Systems through Uncertainty Monitoring},  
  author={Weiss, Michael and Tonella, Paolo},  
  booktitle={2021 IEEE 14th International Conference on Software Testing, Validation and Verification (ICST)},  
  year={2021},  
  organization={IEEE},  
  note={forthcoming}  
}  

@inproceedings{Weiss2021UncertaintyWizard,  
  title={Uncertainty-Wizard: Fast and User-Friendly Neural Network Uncertainty Quantification},  
  author={Weiss, Michael and Tonella, Paolo},  
  booktitle={2021 IEEE 14th International Conference on Software Testing, Validation and Verification (ICST)},  
  year={2021},  
  organization={IEEE},  
  note={forthcoming}  
}  

The first paper (preprint) provides an empricial study comparing the approaches implemented in uncertainty wizard, and a list of lessons learned useful for reasearchers working with uncertainty wizard. The second paper (preprint) is a technical tool paper, providing a more detailed discussion of uncertainty wizards api and implementation.

References to the original work introducing the techniques implemented in uncertainty wizard are provided in the papers listed above.

Contributing

Issues and PRs are welcome! Before investing a lot of time for a PR, please open an issue first, describing your contribution. This way, we can make sure that the contribution fits well into this repository. We also mark issues which are great to start contributing as as good first issues. If you want to implement an existing issue, don't forget to comment on it s.t. everyone knows that you are working on it.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

uncertainty-wizard-0.1.1.tar.gz (36.2 kB view details)

Uploaded Source

Built Distribution

uncertainty_wizard-0.1.1-py3-none-any.whl (46.6 kB view details)

Uploaded Python 3

File details

Details for the file uncertainty-wizard-0.1.1.tar.gz.

File metadata

  • Download URL: uncertainty-wizard-0.1.1.tar.gz
  • Upload date:
  • Size: 36.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/45.2.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5

File hashes

Hashes for uncertainty-wizard-0.1.1.tar.gz
Algorithm Hash digest
SHA256 f35ba2285c000b1440afc3c837baa8e918bccdff7e902584ec4630737d922043
MD5 6b5b30091a887567a98e137697a81dc5
BLAKE2b-256 0ff7d14b7347b8b102bc11904319d55bf9c55971adee0e3243bc9a2bcaa2a0ad

See more details on using hashes here.

File details

Details for the file uncertainty_wizard-0.1.1-py3-none-any.whl.

File metadata

  • Download URL: uncertainty_wizard-0.1.1-py3-none-any.whl
  • Upload date:
  • Size: 46.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/45.2.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5

File hashes

Hashes for uncertainty_wizard-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 d0da460357216bf1a474c0c868f09cbaccb68ad5629cc1a8d84fed15ce7193cc
MD5 fef25ef3acfbb617df3bf3381d0f7ff9
BLAKE2b-256 76d573b2c5a724192c4fb1e48390f9b1d7153f1e2e1cfe8445a0d625d32f63eb

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page