Skip to main content

Sample PyTorch implementation of the snake activation function

Project description

PyPI version


Based on "Neural Networks Fail to Learn Periodic Functions and How to Fix It" by Liu Ziyin, Tilman Hartwig, Masahito Ueda

This is a PyTorch implementation of the snake activation function from the paper - or at least I think it is, no affiliation with the authors, use at your own risk, etc., etc. Huge thanks to contributors klae01 and fedebotu who made big improvments to the code.

A few variations of the function are discussed in the paper, this package implements:

Snake is periodic, but also monotonic. To see how snake behaves for a range of x given various choices of a, watch this animation:


Two methods:

  • Using pip, pip install torch-snake
  • To install from source, first clone this repository. Then, from the main repo folder, run python install


Fairly easy really from snake.activations import Snake. The Snake constructor (code here) has an optional learnable parameter alpha which defaults to 1. The authors of the paper find values between 5 and 50 work quite well for "known-periodic" data, while for better results with non-periodic data, you should choose a small value like 0.2. The constructor also takes an alpha_learnable parameter which defaults to True, so that you can disable "learnability" if your experiments so require.

Sample code

There's a notebook, still quite rough - example.ipynb. Early indications are that good choices of hyperparameters are quite important for best results (though snake's own parameter trains quite readily).


This code probably wouldn't have gotten written if it hadn't been for Alexandra Deis and her excellent article .

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

torch-snake-1.0.1.tar.gz (4.8 kB view hashes)

Uploaded Source

Built Distribution

torch_snake-1.0.1-py3-none-any.whl (5.1 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page