Package for doing approximate Bayesian inference on deep neural networks [based in TF 2.0+]
Project description
deepbayes
deepbayes is a package that aims to make training Bayesian neural networks (BNNs) simple.
Implemented on top of TensorFlow 2.0 and working with tf.keras as a primary interface, one can simply declare a keras (or TF) model and pass it to the desired deepbayes optimizer and it will compile the model into an object that is ready to perform approximate inference.
deepbayes is actively under development, and location and names of packages may change (such changes will be noted at the bottom of this repo). jax support coming soon :)
Install:
pip install deepbayes
Supported Optimizers
BayesByBackprop
Implemented based on description in https://arxiv.org/abs/1505.05424
Hamiltonian (Hybrid) Monte Carlo
Implemented based on description in https://arxiv.org/abs/1206.1901
NoisyAdam (aka Vadam)
Implemented based on description in https://arxiv.org/abs/1712.02390
Variational Online Gauss Newton (aka VOGN)
Implemented based on description in https://arxiv.org/abs/1806.04854 and https://arxiv.org/abs/2002.10060
Stochastic Weight Averaging - Gaussian (aka SWAG)
Implemented bsed on description in https://arxiv.org/pdf/1902.02476.pdf
Stochastic Gradient Descent
Not sure what to reference you to here... a google will suffice
Supported Analysis
In addition to producing posterior distributions for BNNs, deepbayes also supports some basic analysis including
- Adversarial attacks for BNNs (FGSM, PGD, CW)
- Uncertainty quantification
- Statistical guarentees of robustness - https://arxiv.org/pdf/1903.01980.pdf
- Probabalistic safety estimates - https://arxiv.org/pdf/2004.10281.pdf
Dependancies:
tensorflow, tensorflow-probability, numpy, tqdm, statsmodels,
Future Support
Below we have a tentative to-do list of inference methods to impliment (7 to be exact) and other properties we want to tool to have.
- HTML documentation
- Non-Gaussian Posterior (requires extension of posterior representation as well)
- Stochastic Variational Inference (SVI)
- Probabilistic Backprop (PBP)
- Sequential Monte Carlo optimizer (SMC)
- Monte Carlo Dropout optimizer (MCD)
- Stochastic Gradient Langevin Dynamics optimizer (SGLD)
- Stochastic Gradient Markov Chain Monte Carlo (SGMCMC)
- Riemann manifold HMC (RMHMC)
- JAX autodif library instead of TF (deepbayes.jax.optimizers)
- Binary Bayesian neural networks
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for deepbayes-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c28f6efff9c8a65b3356225da4fa6a7674e61b24cc2eaa9781f7d73983a3865b |
|
MD5 | a013cb05e2ad967dd800b8788a0e9346 |
|
BLAKE2b-256 | 98ecb665adb0c46218b4f31bdfe9cee515df656eff7ff04802638ddc32be5218 |