sanfis 0.1.0

Implementation to the State-Adaptive Neurofuzzy Inference System (S-ANFIS) network

sanfis

This is a PyTorch-based implementation of my project S-ANFIS: State-ANFIS: A Generalized Regime-Switching Model for Financial Modeling (2022). S-ANFIS is an generalization of Jang's ANFIS: adaptive-network-based fuzzy inference system (1993). The implemenation can easliy be used to fit an ANFIS network.

1. What is S-ANFIS

S-ANFIS is a simple generalization of the ANFIS network, where the input to the premise and the consequence part of the model can be controlled separately. As general notation, I call the input the premise part "state" variables s and the input of the consequence part "input" or "explanatory" variables x.

For an in-depth explaination, check out our paper.

2. Installation

This package is intended to be installed on top of PyTorch, so you need to do that first.

Step 1: Install PyTorch

Make sure to consider the correct operating system: Windows, macOS (Intel / Apple Silicon) or Linux. Everything is explained on the developer's website.

To ensure that PyTorch was installed correctly, verify the installation by running sample PyTorch code:

import torch
x = torch.rand(5, 3)
print(x)

Step 2: Install sanfis

sanfis can be installed via pip:

pip install sanfis

3. Quick start

First let's generate some data! The given example is an AR(2)-process whoose AR-parameters depend on the regime of two independent state variables:

# Load modules
import numpy as np
import torch
from sanfis import SANFIS, plottingtools
from sanfis.datagenerators import sanfis_generator

# seed for reproducibility
np.random.seed(3)
torch.manual_seed(3)
## Generate Data ##
S, S_train, S_valid, X, X_train, X_valid, y, y_train, y_valid, = sanfis_generator.gen_data_ts(
    n_obs=1000, test_size=0.33, plot_dgp=True)

Set a list of membership functions for each of the state variables that enter the model:

# list of membership functions
membfuncs = [
    {'function': 'sigmoid',
     'n_memb': 2,
     'params': {'c': {'value': [0.0, 0.0],
                      'trainable': True},
                'gamma': {'value': [-2.5, 2.5],
                          'trainable': True}}},

    {'function': 'sigmoid',
     'n_memb': 2,
     'params': {'c': {'value': [0.0, 0.0],
                      'trainable': True},
                'gamma': {'value': [-2.5, 2.5],
                          'trainable': True}}}
]

The given example uses two sigmoid functions for each state variable.

Now create the model, fit and evaluate:

# make model / set loss function and optimizer
fis = SANFIS(membfuncs=membfuncs, n_input=2, scale='Std')
loss_function = torch.nn.MSELoss(reduction='mean')
optimizer = torch.optim.Adam(fis.parameters(), lr=0.005)

# fit model
history = fis.fit([S_train, X_train, y_train], [S_valid, X_valid, y_valid],
                  optimizer, loss_function, epochs=1000)
# eval model
y_pred = fis.predict([S, X])
plottingtools.plt_prediction(y, y_pred,
                             save_path='img/sanfis_prediction.pdf')
# plottingtools.plt_learningcurves(history)

4. Features

4.1 Membership functions

The implementation allows a very flexible usage of membership functions. For each input variable that enters the premise-part of the model, the type and number of membership functions can be flexibly chosen. As of today, three possible membership functions are implemented:

Gaussian

The Gaussian is described by 2 parameters, mu for the location and sigma for the wideness.

# Example
gaussian_membfunc = {'function': 'gaussian',
			 'n_memb': 3,	 # 3 membership functions
			 'params': {'mu': {'value': [-2.0, 0.0, 1.5], 
			                'trainable': True},
			           'sigma': {'value': [1.0, 0.5, 1.0],
			               'trainable': True}}
			}

In this example, three membership functions are considered.

General bell-shaped

The general bell-shaped function is described by three parameters, a (wideness), b (shape) and c (location).

bell_membfunc = {'function': 'bell',
			'n_memb': 2,
			'params': {'c': {'value': [-1.5, 1.5],
			                'trainable': True},
			            'a': {'value': [3.0, 1.0],
			                'trainable': False},
			            'b': {'value': [1.0, 3.0],
			                'trainable': False}}
					}

Sigmoid

The sigmoid is described by two parameters: c (location) and gamma (steepness).

sigmoid_membfunc = {'function': 'sigmoid',
			'n_memb': 2,
			'params': {'c': {'value': [0.0, 0.0],
			                'trainable': True},
			            'gamma': {'value': [-2.5, 2.5],
			                    'trainable': True}}
}

Remember to add a list of membership functions as membfunc argument when creating the SANFIS oject, e.g.:

MEMBFUNCS = [gaussian_membfunc, bell_membfunc, sigmoid_membfunc]
model = SANFIS(MEMBFUNCS, n_input=2)
model.plotmfs(bounds=[[-2.0, 2.0],  # plot bounds for first membfunc
                      [-4.0, 2.0],  # plot bounds for second membfunc
                      [-5.0, 5.0]],  # plot bounds fo third membfunc
              save_path='img/membfuncs.pdf')

4.2 Tensorboard

Tensorboard provides visualization needed for machine learning experimentation. Further information can be found here

Step 1: Install tensorboard

pip install tensorboard

Step 2: enable tensorboard usage during training

Tensorboard functionality can be added via arguments in the fit() function, e.g.

history = model.fit( ...
                    use_tensorboard=True,
                    logdir='logs/tb',
                    hparams_dict={}
                   )

Note that hparams_dict is an optional argument where you can store additional hyperparameters of you model, e.g. hparams_dict={'n_input':2}.

Step 3: Open tensorboard

tensorboard --logdir=logs/tb

5. Using the plain vanilla ANFIS network

To use the plain vanilla ANFIS network, simply remove the state variables s from the training (fit()). This automatically sets the same input for premise and consequence part of the model.

# Set 4 input variables with 3 gaussian membership functions each
MEMBFUNCS = [
    {'function': 'gaussian',
     'n_memb': 3,
     'params': {'mu': {'value': [-0.5, 0.0, 0.5],
                       'trainable': True},
                'sigma': {'value': [1.0, 1.0, 1.0],
                          'trainable': True}}},

    {'function': 'gaussian',
     'n_memb': 3,
     'params': {'mu': {'value': [-0.5, 0.0, 0.5],
                       'trainable': True},
                'sigma': {'value': [1.0, 1.0, 1.0],
                          'trainable': True}}},

    {'function': 'gaussian',
     'n_memb': 3,
     'params': {'mu': {'value': [-0.5, 0.0, 0.5],
                       'trainable': True},
                'sigma': {'value': [1.0, 1.0, 1.0],
                          'trainable': True}}},

    {'function': 'gaussian',
     'n_memb': 3,
     'params': {'mu': {'value': [-0.5, 0.0, 0.5],
                       'trainable': True},
                'sigma': {'value': [1.0, 1.0, 1.0],
                          'trainable': True}}},

]

# generate some data (mackey chaotic time series)
X, X_train, X_valid, y, y_train, y_valid = datagenerator.gen_data(data_id='mackey',
                                                                  n_obs=2080, n_input=4)

# create model
model = SANFIS(membfuncs=MEMBFUNCS,
               n_input=4,
               scale='Std')
optimizer = torch.optim.Adam(params=model.parameters())
loss_functions = torch.nn.MSELoss(reduction='mean')

# fit model
history = model.fit(train_data=[X_train, y_train],
                    valid_data=[X_valid, y_valid],
                    optimizer=optimizer,
                    loss_function=loss_functions,
                    epochs=200,
                    )

# predict data
y_pred = model.predict(X)

# plot learning curves
plottingtools.plt_learningcurves(history, save_path='img/learning_curves.pdf')

# plot prediction
plottingtools.plt_prediction(y, y_pred, save_path='img/mackey_prediction.pdf')

6. Related work

• AnfisTensorflow2.0 by me
• bare-bones implementation of ANFIS (manual derivatives) by twmeggs
• PyTorch implementation by James Power
• simple ANFIS based on Tensorflow 1.15.2 by Santiago Cuervo

Contact

I am very thankful for feedback. Also, if you have questions, please contact [e-mail]

References

If you use my work, please cite it appropriately:

G. Lenhard and D. Maringer, "State-ANFIS: A Generalized Regime-Switching Model for Financial Modeling," 2022 IEEE Symposium on Computational Intelligence for Financial Engineering and Economics (CIFEr), 2022, pp. 1-8, doi: 10.1109/CIFEr52523.2022.9776208.

BibTex:

@INPROCEEDINGS{lenhard2022sanfis,
  author={Lenhard, Gregor and Maringer, Dietmar},
  booktitle={2022 IEEE Symposium on Computational Intelligence for Financial Engineering and Economics ({CIFEr})}, 
  title={State-{ANFIS}: A Generalized Regime-Switching Model for Financial Modeling}, 
  year={2022},
  pages={1--8},
  doi={10.1109/CIFEr52523.2022.9776208},
  organization={IEEE}
  }