PyGOP: A Python library for Generalized Operational Perceptron (GOP) based algorithms

This package implements progressive learning algorithms using Generalized Operational Perceptron. PyGOP supports both single machine and cluster environment using CPU or GPU. This implementation includes the following algorithms:

• Progressive Operational Perceptron (POP)
• Heterogeneous Multilayer Generalized Operational Perceptron (HeMLGOP) and its variants
• Fast Progressive Operational Perceptron (POPfast)
• Progressive Operational Perceptron with Memory (POPmemO, POPmemH)

What is Generalized Operational Perceptron?

Generalized Operational Perceptron is an artificial neuron model that was proposed to replace the traditional McCulloch-Pitts neuron model. While standard perceptron model only performs a linear transformation followed by non-linear thresholding, GOP model encapsulates a diversity of both linear and non-linear operations (with traditional perceptron as a special case). Each GOP is characterized by learnable synaptic weights and an operator set comprising of 3 types of operations: nodal operation, pooling operation and activation operation. The 3 types of operations performed by a GOP loosely resemble the neuronal activities in a biological learning system of mammals in which each neuron conducts electrical signals over three distinct operations:

• Modification of input signal from the synapse connection in the Dendrites.
• Pooling operation of the modified input signals in the Soma.
• Sending pulses when the pooled potential exceeds a limit in the Axon hillock.

By defining a set of nodal operators, pooling operators and activation operators, each GOP can select the suitable operators based on the problem at hand. Thus learning a GOP-based network involves finding the suitable operators as well as updating the synaptic weights. The author of GOP proposed Progressive Operational Perceptron (POP) algorithm to progressively learn GOP-based networks. Later, Heterogeneous Multilayer Generalized Operational Perceptron (HeMLGOP) algorithm and its variants (HoMLGOP, HeMLRN, HoMLRN) were proposed to learn heterogeneous architecture of GOPs with efficient operator set search procedure. In addition, fast version of POP, i.e., POPfast was proposed together with memory extensions POPmemO, POPmemH that augment POPfast by incorporating memory path.

Installation

PyPi installation

Tensorflow version 1 is required before installing PyGOP. We suggest installing tensorflow 1.14.0 To install tensorflow CPU version through pip::

pip install tensorflow==1.14.0

Or the GPU version::

pip install tensorflow-gpu==1.14.0

To install PyGOP with required dependencies::

pip install pygop

At the moment, PyGOP only supports Linux with python 2 and python 3 (tested on Python 2.7 and Python 3.5, 3.6, 3.7 with tensorflow for cpu)

Installation from source

To install latest version from github, clone the source from the project repository and install with setup.py::

git clone https://github.com/viebboy/PyGOP
cd PyGOP
python setup.py install --user

Documentation

Full documentation can be found here

Reference

If you use one of the algorithms, please cite the corresponding articles:

• S. Kiranyaz, T. Ince, A. Iosifidis and M. Gabbouj, "Progressive Operational Perceptron", Neurocomputing, vol 224, pp. 142-154, 2017.
• D. T. Tran, S. Kiranyaz, M. Gabbouj and A. Iosifidis, "Heterogeneous Multilayer Generalized Operational Perceptron", IEEE Transactions on Neural Networks and Learning Systems, 2018.
• D. T. Tran, S. Kiranyaz, M. Gabbouj and A. Iosifidis, "Progressive Operational Perceptron with Memory", Neurocomputing, 2019.