torch-snake 0.0.1

Sample PyTorch implementation of the snake activation function

Snake

Inspired by "Neural Networks Fail to Learn Periodic Functions and How to Fix It".

This is a PyTorch implementation of the snake activation function from the paper - or at least I think it is, no affiliation with the authors, use at your own risk, etc., etc.

A few variations of the function are discussed in the paper, this package implements:

Installation

From the main repo folder, run python setup.py install

Usage

Fairly easy really from snake.activations import Snake. The Snake constructor (code here) has an optional learnable parameter alpha which defaults to 1. The authors of the paper find values between 5 and 50 work quite well for "known-periodic" data, while for better results with non-periodic data, you should choose a small value like 0.2. The constructor also takes an alpha_learnable parameter which defaults to True, so that you can disable "learnability" if your experiments so require.

Sample code

There's a notebook, still quite rough - example.ipynb.

Acknowledgements

This code probably wouldn't have gotten written if it hadn't been for Alexandra Deis and her excellent article .