Skip to main content

Efficient Householder Transformation in PyTorch

Project description

This package implements the Householder transformation algorithm for calculating orthogonal matrices and orthonormal frames with differentiable bindings to PyTorch. In particular, the package provides an enhanced drop-in replacement for the torch.orgqr function, which was renamed into torch.linalg.householder_product as of PyTorch 1.9.

APIs for orthogonal transformations have been around since LAPACK; however, their support in the deep learning frameworks is lacking. Recently, orthogonal constraints have become popular in deep learning as a way to regularize models and improve training dynamics, and hence the need to backpropagate through orthogonal transformations arised.

PyTorch 1.7 implements matrix exponential function torch.matrix_exp, which can be repurposed to performing the orthogonal transformation when the input matrix is skew-symmetric. This is the baseline we use in Speed and Precision evaluation.

PyTorch 1.9 renamed torch.orgqr into torch.linalg.householder_product, and added support of autograd, batching, and GPU execution.

Compared to torch.matrix_exp, the Householder transformation implemented in this package has the following advantages:

  • Orders of magnitude lower memory footprint
  • Ability to transform non-square matrices (orthonormal frames)
  • A significant speed-up for non-square matrices
  • Better numerical precision for all matrix and batch sizes

Find more details and the most up-to-date information on the project webpage: https://www.github.com/toshas/torch-householder

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

torch_householder-1.0.1.tar.gz (457.4 kB view details)

Uploaded Source

File details

Details for the file torch_householder-1.0.1.tar.gz.

File metadata

  • Download URL: torch_householder-1.0.1.tar.gz
  • Upload date:
  • Size: 457.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.5.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.9.5

File hashes

Hashes for torch_householder-1.0.1.tar.gz
Algorithm Hash digest
SHA256 9a4b240c68947491c4e96a78771497562650f9a555001e062a0969fce206f786
MD5 b5282012729bcc983f7fd53852c22cb8
BLAKE2b-256 f37da87d4ea6c11f23d237fc81c094a6c18909486fdb9914599479cbeb5d089f

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page