tracetorch 0.1.0

An extension to PyTorch: SNN layers that function on traces.

traceTorch is a PyTorch-based library built on the principles of spiking neural networks, replacing the PyTorch default backpropagation through time with lightweight, per-layer input traces, enabling biologically inspired, constant time and memory consumption learning on arbitrarily long or even streaming sequences.

Documentation

It is highly recommended that you read the documentation first. It contains:

1. Introduction: An introduction to traceTorch, how and why it works, it's founding principles. It's thoroughly recommended that you read through the entire introduction and gain an intuitive understanding before proceeding.
2. Tutorials: Various tutorials to create your own traceTorch models. The resultant code can be found in tutorials/.
3. Documentation: The actual documentation to all the modules included in traceTorch. It includes detailed explanations, examples and math to gain a full understanding.

Roadmap

• Create the poisson click test example
• Implement the trace alternative to REINFORCE
• Make traceTorch into a PyPI library
• Finish writing the documentation
• Clean up the tutorial code
• Implement abstract graph based models, not just sequential

Installation

⚠️ WARNING, traceTorch is not yet a library. For now, you'll just have to clone this repository and use the tracetorch/ folder within.

git clone https://github.com/Yegor-men/tracetorch
cd tracetorch/
pip install -r requirements.txt

Then, within a python file where from where the repository root folder is visible, simply do:

from tracetorch import tracetorch

Usage examples

tutorials/ contains all the tutorial files, ready to run and playtest. The tutorials themselves can be found here.

To ensure that you have all the necessary packages for the tutorials installed, please execute the following command:

cd tutorials/
pip install -r requirements.txt

Authors

• @Yegor-men

Acknowledgements

I built traceTorch from the ground up, trying to reverse engineer biological neurons with a sprinkle of intelligent design, but I would also like to recognize the following projects and people who helped shape my thinking:

• snntorch for introducing me to SNN networks in the first place, and their principles of function. Ironically, its dependency on constructing the full autograd graph is what largely inspired me to make traceTorch.
• Artem Kirsanov for introducing me to computational neuroscience, presenting interesting concepts in an easy-to-understand manner. My earliest tests, when I naively wanted to implement 1:1 biological neurons, largely revolved around his work.
• e-prop (eligibility propagation) inspired the whole "trace" concept, the idea of keeping a decaying value. Earlier, before traceTorch, I wanted to use e-prop for online learning instead. Admittedly unsuccessful in my attempts, and a little put off by the relative difficulty, I instead wanted to make something simpler.

Contributing

Contributions are always welcome. Feel free to submit pull requests or report issues, I will occasionally check in on it.

You can also reach out to me via either email or Twitter:

• [e-mail]
• Twitter