An Efficient Unitary Neural Network implementation for PyTorch
This repository contains a simple PyTorch implementation of a Tunable Efficient Unitary Neural Network (EUNN) Cell.
The implementation is loosely based on the tunable EUNN presented in this paper: https://arxiv.org/abs/1612.05231.
pip install torch_eunn
from torch_eunn import EUNN # feed forward layer from torch_eunn import EURNN # recurrent unit
hidden_size and the
capacity of the EUNN need to be even, as explained in the section "Difference with original implementation".
- PyTorch >= 0.4.0:
conda install pytorch -c pytorch
Difference with original implementation
This implementation of the EUNN has a major difference with the original implementation proposed in https://arxiv.org/abs/1612.05231, which is outlined below.
In the original implementation, the first output of the top directional coupler
of a capacity-2 sublayer skips the second layer of directional couplers
(indicated with dots in the ascii figure below) to connect to the next
capacity-2 sublayer of the EUNN. The reverse happens at the bottom, where the
first layer of the capacity-2 sublayer is skipped. This way, a
(2*n+1)-dimensional unitary matrix representation is created, with
number of mixing units in each capacity-1 sublayer.
__ __...... \/ __/\____ __ \/ __ ____/\__ \/ __/\____ __ \/ ......__/\__
For each capacity-1 sublayer with
N=2*n+1 inputs (
N odd), we thus have
parameters (each mixing unit has 2 parameters). Thus to have a unitary matrix
representation that spans the full unitary space, one needs
N extra phases appended to the back of the capacity-
sublayer to bring the total number of parameters in the unitary-matrix
N**2 (the total number of independent parameters in a
In the implementation proposed here, the dots in each capacity-2 sublayer are
connected onto themselves (periodic boundaries). This has the implication that
for each capacity-1 sublayer with
n directional couplers, there are
inputs and as many independent parameters. This means that we just need
capacity-1 sublayers and no extra phases to span the full unitary space
This, however, has the implication that the
hidden_size = N = 2*n of the
unitary matrix should always be even. Also, because the forward pass is
defined per capacity-2 sublayer (as opposed per capacity-1 sublayer in the
original implementation) the capacity has to be even as well.
© Floris Laporte, 2018-2019.
Made available under the MIT license.
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size torch_eunn-0.2.0-py3-none-any.whl (6.8 kB)||File type Wheel||Python version py3||Upload date||Hashes View hashes|
|Filename, size torch_eunn-0.2.0.tar.gz (5.5 kB)||File type Source||Python version None||Upload date||Hashes View hashes|
Hashes for torch_eunn-0.2.0-py3-none-any.whl