reservoirpy 0.3.1.post1

A simple and flexible code for Reservoir Computing architectures like Echo State Networks.

ReservoirPy (v0.3.1) 🌀🧠

Simple and flexible code for Reservoir Computing architectures like Echo State Networks (ESN).

from reservoirpy.nodes import Reservoir, Ridge, Input

data = Input(input_dim=1)
reservoir = Reservoir(100, lr=0.3, sr=1.1)
readout   = Ridge(ridge=1e-6)

esn = data >> reservoir >> readout

forecast = esn.fit(X, y).run(timeseries)

ReservoirPy is a simple user-friendly library based on Python scientific modules. It provides a flexible interface to implement efficient Reservoir Computing (RC) architectures with a particular focus on Echo State Networks (ESN). Advanced features of ReservoirPy allow to improve computation time efficiency on a simple laptop compared to basic Python implementation, with datasets of any size.

Some of its features are: offline and online training, parallel implementation, sparse matrix computation, fast spectral initialization, advanced learning rules (e.g. Intrinsic Plasticity) etc. It also makes possible to easily create complex architectures with multiple reservoirs (e.g. deep reservoirs), readouts, and complex feedback loops. Moreover, graphical tools are included to easily explore hyperparameters with the help of the hyperopt library. Finally, it includes several tutorials exploring exotic architectures and examples of scientific papers reproduction.

This library works for Python 3.8 and higher.

Follow @reservoirpy updates and new releases on Twitter.

Offcial documentation 📖

See the official ReservoirPy's documentation to learn more about the main features of ReservoirPy, its API and the installation process. Or you can access directly the User Guide with tutorials.

Quick example of how to code a deep reservoir

Installation

To install it, use one of the following command:

pip install reservoirpy

or

pip install reservoirpy==0.3.0

If you want to use the previous version 0.2.4, you can install ReservoirPy using:

pip install reservoirpy==0.2.4

If you want to enable the hyper package and its hyperparameter optimization helpers using hyperopt, use:

pip install reservoirpy[hyper]

Quick try ⚡

An example on Chaotic timeseries prediction (MackeyGlass)

Step 1: Load the dataset

ReservoirPy comes with some handy data generator able to create synthetic timeseries for well-known tasks such as Mackey-Glass timeseries forecasting.

from reservoirpy.datasets import mackey_glass

X = mackey_glass(n_timesteps=2000)

Step 2: Create an Echo State Network...

...or any kind of model you wish to use to solve your task. In this simple use case, we will try out Echo State Networks (ESNs), one of the most minimal architecture of Reservoir Computing machines.

An ESN is made of a reservoir, a random recurrent network used to encode our inputs in a high-dimensional (non-linear) space, and a readout, a simple feed-forward layer of neurons in charge with reading-out the desired output from the activations of the reservoir.

from reservoirpy.nodes import Reservoir, Ridge

reservoir = Reservoir(units=100, lr=0.3, sr=1.25)
readout = Ridge(output_dim=1, ridge=1e-5)

We here obtain a reservoir with 100 neurons, a spectral radius of 1.25 and a leak rate of 0.3 (you can learn more about these hyperparameters going through the tutorial Understand and optimize hyperparameters). Here, our readout layer is just a single unit, that we will receive connections from (all units of) the reservoir. Note that only the readout layer connections are trained. This is one of the cornerstone of all Reservoir Computing techniques. In our case, we will train these connections using linear regression, with a regularization coefficient of 10^-5.

Now, let's connect everything using the >> operator.

esn = reservoir >> readout

That's it! Next step: fit the readout weights to perform the task we want. We will train the ESN to make one-step-ahead forecasts of our timeseries.

Step 3: Fit and run the ESN

We train our ESN on the first 500 timesteps of the timeseries.

esn.fit(X[:500], X[1:501])

Our ESN is now trained and ready to use. Let's run it on the remainder of the timeseries:

predictions = esn.run(X[501:-1])

As a shortcut, both operations can be performed in just one line!

predictions = esn.fit(X[:500], X[1:501]).run(X[501:-1])

Let's now evaluate its performances.

Step 4: Evaluate the ESN

from reservoirpy.observables import rmse, rsquare
print("RMSE:", rmse(X[502:], predictions),
      "R^2 score:", rsquare(X[502:], predictions))

Run and analyse this simple file (in the "tutorials/Simple Examples with Mackey-Glass" folder) to see a complete example of timeseries prediction with ESNs:

• simple_example_MackeyGlass.py (using the ESN class)

  python simple_example_MackeyGlass.py
  

If you have some issues testing some examples, have a look at the extended packages requirements in readthedocs.

More examples and tutorials 🎓

Go to the tutorial folder for tutorials in Jupyter Notebooks.

Go to the examples folder for examples and papers with codes, also in Jupyter Notebooks.

Paper with tutorials

Tutorial for ReservoirPy (v0.2) can be found in this Paper (Trouvain et al. 2020).

Explore Hyper-Parameters with Hyperopt

A quick tutorial on how to explore hyperparameters with ReservoirPy and Hyperopt can be found in this paper (Trouvain et al. 2020).

Take a look at our advices and our method to explore hyperparameters for reservoirs in our recent paper: (Hinaut et al 2021) HTML HAL

Turorial and Jupyter Notebook for hyper-parameter exploration

More info on hyperopt: Official website

Papers and projects using ReservoirPy

If you want your paper to appear here, please contact us (see contact link below).

• Trouvain & Hinaut (2021) Canary Song Decoder: Transduction and Implicit Segmentation with ESNs and LTSMs. ICANN 2021 HTML HAL PDF
• Pagliarini et al. (2021) Canary Vocal Sensorimotor Model with RNN Decoder and Low-dimensional GAN Generator. ICDL 2021. HTML
• Pagliarini et al. (2021) What does the Canary Say? Low-Dimensional GAN Applied to Birdsong. HAL preprint. HAL PDF
• Which Hype for My New Task? Hints and Random Search for Echo State Networks Hyperparameters. ICANN 2021 HTML HAL PDF

Contact

If you have a question regarding the library, please open an Issue. If you have more general question or feedback you can contact us on twitter or by email to nathan dot trouvain at inria dot fr and to xavier dot hinaut at inria dot fr.

Citing ReservoirPy

Trouvain, N., Pedrelli, L., Dinh, T. T., Hinaut, X. (2020) Reservoirpy: an efficient and user-friendly library to design echo state networks. In International Conference on Artificial Neural Networks (pp. 494-505). Springer, Cham. HTML HAL PDF

If you're using ReservoirPy in your work, please cite our package using the following bibtex entry:

@incollection{Trouvain2020,
  doi = {10.1007/978-3-030-61616-8_40},
  url = {https://doi.org/10.1007/978-3-030-61616-8_40},
  year = {2020},
  publisher = {Springer International Publishing},
  pages = {494--505},
  author = {Nathan Trouvain and Luca Pedrelli and Thanh Trung Dinh and Xavier Hinaut},
  title = {{ReservoirPy}: An Efficient and User-Friendly Library to Design Echo State Networks},
  booktitle = {Artificial Neural Networks and Machine Learning {\textendash} {ICANN} 2020}
}