Skip to main content

BackPACK: Packing more into backprop

Project description

BackPACK BackPACK: Packing more into backprop

Travis Coveralls Python 3.6+

BackPACK is built on top of PyTorch. It efficiently computes quantities other than the gradient.

Provided quantities include:

  • Individual gradients from a mini-batch
  • Estimates of the gradient variance or second moment
  • Approximate second-order information (diagonal and Kronecker approximations)

Motivation: Computation of most quantities is not necessarily expensive (often just a small modification of the existing backward pass where backpropagated information can be reused). But it is difficult to do in the current software environment.

Installation

pip install backpack-for-pytorch

Examples

Contributing

BackPACK is actively being developed. We are appreciating any help. If you are considering to contribute, do not hesitate to contact us. An overview of the development procedure is provided in the developer README.

How to cite

If you are using BackPACK, consider citing the paper

@inproceedings{dangel2020backpack,
    title     = {Back{PACK}: Packing more into Backprop},
    author    = {Felix Dangel and Frederik Kunstner and Philipp Hennig},
    booktitle = {International Conference on Learning Representations},
    year      = {2020},
    url       = {https://openreview.net/forum?id=BJlrF24twB}
}
BackPACK is not endorsed by or affiliated with Facebook, Inc. PyTorch, the PyTorch logo and any related marks are trademarks of Facebook, Inc.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for backpack-for-pytorch, version 1.3.0
Filename, size File type Python version Upload date Hashes
Filename, size backpack_for_pytorch-1.3.0-py3-none-any.whl (119.8 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size backpack-for-pytorch-1.3.0.tar.gz (1.1 MB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page