sofenn 0.1.6

Keras model of a Self-Organizing Fuzzy Network

sofenn: Self-Organizing Fuzzy Neural Network

Welcome to sofenn (sounds like soften)!

This package is a Keras/TensorFlow implementation of a Self-Organizing Fuzzy Neural Network. The sofenn framework consists of two models:
(1) Fuzzy Network - Underlying fuzzy network whose parameters are optimized during training
(2) Self-Organizer - Metamodel that optimizes the architecture of the Fuzzy Network

FuzzyNetwork

Underlying neural network model that contains parameters to be optimized during training

FuzzySelfOrganizer

Metamodel to optimize the architecture of underlying fuzzy network

Installation

You can install the package using pip:

pip install sofenn

Usage

Demo notebooks available on Github.

Importing the model and self-organizer:

from sofenn import FuzzyNetwork, FuzzySelfOrganizer

# initialize model separate, and attach to self-organizer
model = FuzzyNetwork(input_shape, **_init_params)
sofnn = FuzzySelfOrganizer(model=model)
sofnn.self_organize(x, y)

# initialize sofnn directly
sofnn = FuzzySelfOrganizer(input_shape, **_init_params)
sofnn.self_organize(x, y)

Model Description

The model is implemented per the description in:

'An on-line algorithm for creating self-organizing fuzzy neural networks'
Leng, Prasad, McGinnity (2004)

Fuzzy Neural Network Architecture

Credit: Leng, Prasad, McGinnity (2004)

Layers

Inputs Layer (0)

Input layer of network

• input :
  • shape: (*, features)

Fuzzy Layer (1)

Radial (Ellipsoidal) Basis Function Layer

• Each neuron represents "if-part" or premise of a fuzzy rule

• Individual Membership Functions (MF) are applied to each feature for each neuron

• Output is product of Membership Functions

• Each MF is a Gaussian function:

  

  • for i features and j neurons:

  • = ith MF of jth neuron

  • = center of ith MF of jth neuron

  • = width of ith MF of jth neuron

• output for Fuzzy Layer is:

  

• input :

  • shape: (*, features)

• output :

  • shape: (*, neurons)

Information flow of r features within neuron j

Credit: Leng, Prasad, McGinnity (2004)

Normalize Layer (2)

Normalization Layer

• Output of each neuron is normalized by total output from the previous layer

• Number of outputs equal to the previous layer (# of neurons)

• Output for Normalize Layer is:

  

  = output of Fuzzy Layer neuron j

• input :

  • shape : (*, neurons)

• output :

  • shape : (*, neurons)

Weighted Layer (3)

Weighting of ith MF of each feature

• Yields the "consequence" of the jth fuzzy rule of the fuzzy model

• Each neuron has two inputs:

  • = output of previous related neuron
  • = weighted bias

• with:

  = number of original input features

  

  

  = output of jth neuron from normalize layer

• output for weighted layer is:

• inputs :

  • shape: [(*, 1+features), (*, neurons)]

• output :

  • shape: (*, neurons)

Output Layer (4)

Final Output

• Unweighted sum of each output of the previous layer ()

• Provide activation function to layer

• Function choice determines output shape (e.g., linear vs. softmax)

• Output for fuzzy layer is:

  

  for u neurons

• Provide activation function to layer (default: linear)

• Activation function determines output dimensions

Examples

Regression output:

• input :
  • shape: (*, neurons)
• output :
  • shape: (*,)

Softmax classification output:

• input :
  • shape: (*, )
• output :
  • shape: (*, classes)