PyRCN 0.0.13

A Python3 framework for Reservoir Computing with a scikit-learn-compatible API

PyRCN

A Python 3 framework for Reservoir Computing with a scikit-learn-compatible API.

PyRCN ("Python Reservoir Computing Networks") is a light-weight and transparent Python 3 framework for Reservoir Computing (currently only implementing Echo State Networks) and is based on widely used scientific Python packages, such as numpy or scipy. The API is fully scikit-learn-compatible, so that users of scikit-learn do not need to refactor their code in order to use the estimators implemented by this framework. Scikit-learn's built-in parameter optimization methods and example datasets can also be used in the usual way.

PyRCN is used by the Chair of Speech Technology and Cognitive Systems, Institute for Acoustics and Speech Communications, Technische Universität Dresden, Dresden, Germany and IDLab (Internet and Data Lab), Ghent University, Ghent, Belgium.

Currently, it implements Echo State Networks (ESNs) by Herbert Jaeger in different flavors, e.g. Classifier and Regressor. It is actively developed to be extended into several directions:

• Incorporate Feedback
• Better sequence handling with sktime
• A unified API to stack ESNs
• More towards future work: Related architectures, such as Liquid State Machines (LSMs) and Perturbative Neural Networks (PNNs)

PyRCN has successfully been used for several tasks:

• Music Information Retrieval (MIR)
  • Multipitch tracking
  • Onset detection
• Time Series Prediction
  • Mackey-Glass benchmark test
  • Stock price prediction
• Ongoing research tasks:
  • Beat tracking in music signals
  • Pattern recognition in sensor data
  • Phoneme recognition

Please see the References section for more information. For code examples, see Getting started.

Installation

Prerequisites

PyRCn is developed using Python 3.6 or newer. It depends on the following packages:

• numpy>=1.18.1
• scipy>=1.2.0
• scikit-learn>=0.23.1
• joblib>=0.13.2
• pandas>=1.0.0
• matplotlib

Installation from PyPI

The easiest and recommended way to install PyRCN is to use pip from PyPI :

pip install pyrcn   

Installation from source

If you plan to contribute to PyRCN, you can also install the package from source.

Clone the Git repository:

git clone https://github.com/TUD-STKS/PyRCN.git

Install the package using setup.py:

python setup.py install --user

Package structure

The package is structured in the following way:

• doc
  • This folder includes the package documentation.
• examples
  • This folder includes example code as Jupyter Notebooks and python scripts.
• images
  • This folder includes the logos used in ´README.md´.
• pyrcn
  • This folder includes the actual Python package.

Getting Started

PyRCN includes currently variants of Echo State Networks (ESNs) and Extreme Learning Machines (ELMs): Regressors and Classifiers.

Basic example for the ESNClassifier:

from pyrcn.echo_state_network import ESNClassifier


clf = ESNClassifier()
clf.fit(X=X_train, y=y_train)

y_pred_classes = clf.predict(X=X_test)  # output is the class for each input example
y_pred_proba = clf.predict_proba(X=X_test)  #  output are the class probabilities for each input example

Basic example for the ESNRegressor:

from pyrcn.echo_state_network import ESNRegressor


reg = ESNRegressor()
ref.fit(X=X_train, y=y_train)

y_pred = reg.predict(X=X_test)  # output is the prediction for each input example

An extensive introduction to getting started with PyRCN is included in the examples directory. The notebook digits or its corresponding Python script show how to set up an ESN for a small hand-written digit recognition experiment.

Launch the digits notebook on Binder:

Fore more advanced examples, please have a look at our Automatic Music Transcription Repository, in which we provide an entire feature extraction, training and test pipeline for multipitch tracking and for note onset detection using PyRCN.

Citation

If you use PyRCN, please cite the following publication:

@misc{steiner2021pyrcn,
      title={PyRCN: Exploration and Application of ESNs}, 
      author={Peter Steiner and Azarakhsh Jalalvand and Simon Stone and Peter Birkholz},
      year={2021},
      eprint={2103.04807},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

References

Glottal Closure Instant Detection using Echo State Networks

@InProceedings{src:Steiner-21a,
	title = {Glottal Closure Instant Detection using Echo State Networks},
	author = {Peter Steiner and Ian S. Howard and Peter Birkholz},
	year = {2021},
	pages = {161--168},
	keywords = {Oral},
	booktitle = {Studientexte zur Sprachkommunikation: Elektronische Sprachsignalverarbeitung 2021},
	editor = {Stefan Hillmann and Benjamin Weiss and Thilo Michael and Sebastian Möller},
	publisher = {TUDpress, Dresden},
	isbn = {978-3-95908-227-3}
} 

Cluster-based Input Weight Initialization for Echo State Networks

@misc{src:Steiner-21b,
    title={Cluster-based Input Weight Initialization for Echo State Networks},
    author={Peter Steiner and Azarakhsh Jalalvand and Peter Birkholz},
    year={2021},
    eprint={2103.04710},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

PyRCN: Exploration and Application of ESNs

@misc{steiner2021pyrcn,
      title={PyRCN: Exploration and Application of ESNs}, 
      author={Peter Steiner and Azarakhsh Jalalvand and Simon Stone and Peter Birkholz},
      year={2021},
      eprint={2103.04807},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

Note Onset Detection using Echo State Networks

@InProceedings{src:Steiner-20a,
	title = {Note Onset Detection using Echo State Networks},
	author = {Peter Steiner and Simon Stone and Peter Birkholz},
	year = {2020},
	pages = {157--164},
	keywords = {Poster},
	booktitle = {Studientexte zur Sprachkommunikation: Elektronische Sprachsignalverarbeitung 2020},
	editor = {Ronald Böck and Ingo Siegert and Andreas Wendemuth},
	publisher = {TUDpress, Dresden},
	isbn = {978-3-959081-93-1}
} 

Feature Engineering and Stacked ESNs for Musical Onset Detection

@INPROCEEDINGS{src:Steiner-20d,  
    author={Peter Steiner and Simon Stone and Azarakhsh Jalalvand and Peter Birkholz},  
    booktitle={2020 25th International Conference on Pattern Recognition (ICPR)},   
    title={Feature Engineering and Stacked ESNs for Musical Onset Detection},  
    year={2020},  
    volume={},  
    number={},  
    note={submitted},
}

Multipitch tracking in music signals using Echo State Networks

@INPROCEEDINGS{src:Steiner-20b,
    author={Peter Steiner and Simon Stone and Peter Birkholz and Azarakhsh Jalalvand},
    booktitle={28th European Signal Processing Conference (EUSIPCO), 2020},
    title={Multipitch tracking in music signals using Echo State Networks},
    year={2020},
    note={accepted},
}

Multiple-F0 Estimation using Echo State Networks

@inproceedings{src:Steiner-19,
  title={Multiple-F0 Estimation using Echo State Networks},
  booktitle={{MIREX}},
  author={Peter Steiner and Azarakhsh Jalalvand and Peter Birkholz},
  year={2019},
  url = {https://www.music-ir.org/mirex/abstracts/2019/SBJ1.pdf}
}

Acknowledgements

This research is funded by the European Social Fund (Application number: 100327771) and co-financed by tax funds based on the budget approved by the members of the Saxon State Parliament, and by Ghent University.