Skip to main content

scipy Linear operator implementations of the GGN and Hessian in PyTorch

Project description

Logo scipy linear operators of deep learning matrices in PyTorch

Python 3.7+ tests Coveralls

This library implements scipy.sparse.linalg.LinearOperators for deep learning matrices, such as

  • the Hessian
  • the Fisher/generalized Gauss-Newton (GGN)

Matrix-vector products are carried out in PyTorch, i.e. potentially on a GPU. The library supports defining these matrices not only on a mini-batch, but on data sets (looping over batches during a matvec operation).

You can plug these linear operators into scipy, while carrying out the heavy lifting (matrix-vector multiplies) in PyTorch on GPU. My favorite example for such a routine is scipy.sparse.linalg.eigsh that lets you compute a subset of eigenpairs.

Installation

pip install curvlinops-for-pytorch

Examples

Future ideas

Other features that could be supported in the future include:

  • Other matrices

  • Block-diagonal approximations (via param_groups)

  • Inverse matrix-vector products by solving a linear system via conjugate gradients

    • This could allow computing generalization metrics like the Takeuchi Information Criterion (TIC), using inverse matrix-vector products in combination with Hutchinson trace estimation
Logo mage credits

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

curvlinops-for-pytorch-1.0.0.tar.gz (43.4 kB view hashes)

Uploaded Source

Built Distribution

curvlinops_for_pytorch-1.0.0-py3-none-any.whl (11.7 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page