Sample PyTorch implementation of the snake activation function
Project description
Snake
Inspired by "Neural Networks Fail to Learn Periodic Functions and How to Fix It".
This is a PyTorch implementation of the snake
activation function from the paper - or at least I think it is, no affiliation with the authors, use at your own risk, etc., etc.
A few variations of the function are discussed in the paper, this package implements:
Installation
From the main repo folder, run python setup.py install
Usage
Fairly easy really from snake.activations import Snake
. The Snake
constructor (code here) has an optional learnable parameter alpha which defaults to 1. The authors of the paper find values between 5 and 50 work quite well for "known-periodic" data, while for better results with non-periodic data, you should choose a small value like 0.2. The constructor also takes an alpha_learnable
parameter which defaults to True
, so that you can disable "learnability" if your experiments so require.
Sample code
There's a notebook, still quite rough - example.ipynb.
Acknowledgements
This code probably wouldn't have gotten written if it hadn't been for Alexandra Deis and her excellent article .
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for torch_snake-0.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 32b9f1efef2a139ce16e27cb77dc549f23eb8b6086b45071b0646738ec349ade |
|
MD5 | 0d38419bfde3de6fb10c276bf83d6083 |
|
BLAKE2b-256 | 9750b091873d4f7eb1e564383cc6e0af102dd95606c164cbeb000db519527049 |