torch-eunn 0.2.1

An Efficient Unitary Neural Network implementation for PyTorch

torch_eunn

This repository contains a simple PyTorch implementation of a Tunable Efficient Unitary Neural Network (EUNN) Cell.

The implementation is loosely based on the tunable EUNN presented in this paper: https://arxiv.org/abs/1612.05231.

Installation

    pip install torch_eunn

Usage

    from torch_eunn import EUNN # feed forward layer
    from torch_eunn import EURNN # recurrent unit

Note

The hidden_size and the capacity of the EUNN need to be even, as explained in the section "Difference with original implementation".

Examples

• 00: Simple Tests
• 01: Copying Task
• 02: MNIST Task

Requirements

• PyTorch >= 0.4.0: conda install pytorch -c pytorch

Difference with original implementation

This implementation of the EUNN has a major difference with the original implementation proposed in https://arxiv.org/abs/1612.05231, which is outlined below.

In the original implementation, the first output of the top directional coupler of a capacity-2 sublayer skips the second layer of directional couplers (indicated with dots in the ascii figure below) to connect to the next capacity-2 sublayer of the EUNN. The reverse happens at the bottom, where the first layer of the capacity-2 sublayer is skipped. This way, a (2*n+1)-dimensional unitary matrix representation is created, with n the number of mixing units in each capacity-1 sublayer.

  __  __......
    \/
  __/\____  __
          \/
  __  ____/\__
    \/
  __/\____  __
          \/
  ......__/\__

For each capacity-1 sublayer with N=2*n+1 inputs (N odd), we thus have N-1 parameters (each mixing unit has 2 parameters). Thus to have a unitary matrix representation that spans the full unitary space, one needs N capacity-1 layers and N extra phases appended to the back of the capacity-N sublayer to bring the total number of parameters in the unitary-matrix representation to N**2 (the total number of independent parameters in a unitary matrix).

In the implementation proposed here, the dots in each capacity-2 sublayer are connected onto themselves (periodic boundaries). This has the implication that for each capacity-1 sublayer with n directional couplers, there are N=2*n inputs and as many independent parameters. This means that we just need N capacity-1 sublayers and no extra phases to span the full unitary space with N**2 parameters.

This, however, has the implication that the hidden_size = N = 2*n of the unitary matrix should always be even. Also, because the forward pass is defined per capacity-2 sublayer (as opposed per capacity-1 sublayer in the original implementation) the capacity has to be even as well.

License

© Floris Laporte, 2018-2019.

Made available under the MIT license.