Improve your neural networks with tensor methods

Tensor methods generalize matrix algebraic operations to higher-orders. Deep neural networks typically map between higher-order tensors. In fact, it is the ability of deep convolutional neural networks to preserve and leverage local structure that, along with large datasets and efficient hardware, made the current levels of performance possible. Tensor methods allow to further leverage and preserve that structure, for individual layers or whole networks.

In TensorLy-Torch, we provide convenient layers that do all the heavy lifting for you and provide the benefits tensor based layers wrapped in a nice, well documented and tested API.

For instance, convolution layers of any order (2D, 3D or more), can be efficiently parametrized using tensor decomposition. Using a CP decomposition results in a separable convolution and you can replace your original convolution with a series of small efficient ones:

These can be easily perform with FactorizedConv in TensorLy-Torch. We also have Tucker convolutions and new tensor-train convolutions! We also implement various other methods such as tensor regression and contraction layers, tensorized linear layers, tensor dropout and more!

Through pip

pip install tensorly-torch

From source

git clone https://github.com/tensorly/torch
cd torch
pip install -e .