graforvfl 1.0.0

GrafoRVFL: A Python Library for Maximizing Performance of Random Vector Functional Link Network with Gradient-Free Optimization

GrafoRVFL (GRAdient Free Optimized Random Vector Functional Link)

---

GrafoRVFL is an open-source library in Python that employs gradient-free optimization ((GA, PSO, WOA, TLO, DE, ...) to optimize Random Vector Functional Link Networks. It is entirely implemented based on Numpy and fully compatible with the interfaces of the Scikit-Learn library. With GrafoRVFL, you can fine-tune the hyper-parameters of network or optimize weights in the network using gradient-free optimizers.

• Free software: GNU General Public License (GPL) V3 license
• Provided Estimator: RvflRegressor, RvflClassifier, MhaTuneRvfl
• Total Gradient Free based RVFL Regressor: > 200 Models
• Total Gradient Free based RVFL Classifier: > 200 Models
• Supported performance metrics: >= 67 (47 regressions and 20 classifications)
• Supported objective functions (as fitness functions or loss functions): >= 67 (47 regressions and 20 classifications)
• Documentation: https://graforvfl.readthedocs.io
• Python versions: >= 3.8.x
• Dependencies: numpy, scipy, scikit-learn, pandas, mealpy, permetrics

Citation Request

• Learn more about Random Vector Functional Link from this paper

• Learn more about on how to use Gradient Free Optimization to fine-tune the hyper-parameter of RVFL networks from this paper

• Learn more about on how to use Gradient Free Optimization to optimize the weights of RVFL netweorks from this paper

Please include these citations if you plan to use this library:

@software{nguyen_van_thieu_2023_10258280,
  author       = {Nguyen Van Thieu},
  title        = {GrafoRVFL: A Python Library for Maximizing Performance of Random Vector Functional Link Network with Gradient-Free Optimization},
  month        = dec,
  year         = 2023,
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.10258280},
  url          = {https://github.com/thieu1995/GrafoRVFL}
}

@article{van2023mealpy,
  title={MEALPY: An open-source library for latest meta-heuristic algorithms in Python},
  author={Van Thieu, Nguyen and Mirjalili, Seyedali},
  journal={Journal of Systems Architecture},
  year={2023},
  publisher={Elsevier},
  doi={10.1016/j.sysarc.2023.102871}
}

@inproceedings{nguyen2019building,
  title={Building resource auto-scaler with functional-link neural network and adaptive bacterial foraging optimization},
  author={Nguyen, Thieu and Nguyen, Binh Minh and Nguyen, Giang},
  booktitle={International Conference on Theory and Applications of Models of Computation},
  pages={501--517},
  year={2019},
  organization={Springer}
}

@inproceedings{nguyen2018resource,
  title={A resource usage prediction system using functional-link and genetic algorithm neural network for multivariate cloud metrics},
  author={Nguyen, Thieu and Tran, Nhuan and Nguyen, Binh Minh and Nguyen, Giang},
  booktitle={2018 IEEE 11th conference on service-oriented computing and applications (SOCA)},
  pages={49--56},
  year={2018},
  organization={IEEE},
  doi={10.1109/SOCA.2018.00014}
}

Installation

• Install the current PyPI release:

$ pip install graforvfl==1.0.0

• Install directly from source code

$ git clone https://github.com/thieu1995/GrafoRVFL.git
$ cd GrafoRVFL
$ python setup.py install

• In case, you want to install the development version from Github:

$ pip install git+https://github.com/thieu1995/GrafoRVFL 

After installation, you can import GrafoRVFL as any other Python module:

$ python
>>> import graforvfl
>>> graforvfl.__version__

Examples

Please check all use cases and examples in folder examples.

Current provided classes:

from graforvfl import DataTransformer, Data
from graforvfl import RvflRegressor, RvflClassifier, MhaTuneRvfl

DataTransformer class

We provide many scaler classes that you can select and make a combination of transforming your data via DataTransformer class. For example: scale data by Loge and then Sqrt and then MinMax:

from graforvfl import DataTransformer
import pandas as pd
from sklearn.model_selection import train_test_split

dataset = pd.read_csv('Position_Salaries.csv')
X = dataset.iloc[:, 1:5].values
y = dataset.iloc[:, 5].values
X_train, y_train, X_test, y_test = train_test_split(X, y, test_size=0.2)

dt = DataTransformer(scaling_methods=("loge", "sqrt", "minmax"))
X_train_scaled = dt.fit_transform(X_train)
X_test_scaled = dt.transform(X_test)

Data class

• You can load your dataset into Data class
• You can split dataset to train and test set
• You can scale dataset without using DataTransformer class
• You can scale labels using LabelEncoder

from graforvfl import Data
import pandas as pd

dataset = pd.read_csv('Position_Salaries.csv')
X = dataset.iloc[:, 1:5].values
y = dataset.iloc[:, 5].values

data = Data(X, y, name="position_salaries")

#### Split dataset into train and test set
data.split_train_test(test_size=0.2, shuffle=True, random_state=100, inplace=True)

#### Feature Scaling
data.X_train, scaler_X = data.scale(data.X_train, scaling_methods=("standard", "sqrt", "minmax"))
data.X_test = scaler_X.transform(data.X_test)

data.y_train, scaler_y = data.encode_label(data.y_train)  # This is for classification problem only
data.y_test = scaler_y.transform(data.y_test)

Neural Network class

from graforvfl import RvflRegressor, RvflClassifier, MhaTuneRvfl
from mealpy import IntegerVar, StringVar

## 1. Use standard RVFL model for regression problem
model = RvflRegressor(size_hidden=10, act_name='sigmoid', weight_initializer="random_uniform", trainer="OLS", alpha=0.5)

## 2. Use standard RVFL model for classification problem 
model = RvflClassifier(size_hidden=10, act_name='sigmoid', weight_initializer="random_normal", trainer="OLS", alpha=0.5)


## 3. Use Gradient Free Optimization to fine-tune the hyper-parameter of RVFL network for regression problem
# Design the boundary (parameters)
my_bounds = [
    IntegerVar(lb=2, ub=1000, name="size_hidden"),
    StringVar(valid_sets=("none", "relu", "leaky_relu", "celu", "prelu", "gelu",
                          "elu", "selu", "rrelu", "tanh", "sigmoid"), name="act_name"),
    StringVar(valid_sets=("orthogonal", "he_uniform", "he_normal", "glorot_uniform", "glorot_normal",
                          "lecun_uniform", "lecun_normal", "random_uniform", "random_normal"), name="weight_initializer")
]
opt_paras = {"name": "WOA", "epoch": 10, "pop_size": 20}
model = MhaTuneRvfl(problem_type="regression", bounds=my_bounds, cv=3, scoring="MSE",
                      optimizer="OriginalWOA", optimizer_paras=opt_paras, verbose=True)

Supported functions in model object

from graforvfl import RvflRegressor, Data 

data = Data()       # Assumption that you have provided this object like above

model = RvflRegressor(size_hidden=10, act_name='sigmoid', weight_initializer="random_uniform", trainer="OLS", alpha=0.5)

## Train the model
model.fit(data.X_train, data.y_train)

## Predicting a new result
y_pred = model.predict(data.X_test)

## Calculate metrics using score or scores functions.
print(model.score(data.X_test, data.y_test, method="MAE"))
print(model.scores(data.X_test, data.y_test, list_methods=["MAPE", "NNSE", "KGE", "MASE", "R2", "R", "R2S"]))

## Calculate metrics using evaluate function
print(model.evaluate(data.y_test, y_pred, list_metrics=("MSE", "RMSE", "MAPE", "NSE")))

## Save performance metrics to csv file
model.save_metrics(data.y_test, y_pred, list_metrics=("RMSE", "MAE"), save_path="history", filename="metrics.csv")

## Save training loss to csv file
model.save_loss_train(save_path="history", filename="loss.csv")

## Save predicted label
model.save_y_predicted(X=data.X_test, y_true=data.y_test, save_path="history", filename="y_predicted.csv")

## Save model
model.save_model(save_path="history", filename="traditional_mlp.pkl")

## Load model 
trained_model = RvflRegressor.load_model(load_path="history", filename="traditional_mlp.pkl")

Support (questions, problems)

Official Links

• Official source code repo: https://github.com/thieu1995/GrafoRVFL

• Official document: https://graforvfl.readthedocs.io/

• Download releases: https://pypi.org/project/graforvfl/

• Issue tracker: https://github.com/thieu1995/GrafoRVFL/issues

• Notable changes log: https://github.com/thieu1995/GrafoRVFL/blob/main/ChangeLog.md

• Official chat group: https://t.me/+fRVCJGuGJg1mNDg1

• This project also related to our another projects which are "optimization" and "machine learning", check it here:

  • https://github.com/thieu1995/mealpy
  • https://github.com/thieu1995/metaheuristics
  • https://github.com/thieu1995/opfunu
  • https://github.com/thieu1995/enoppy
  • https://github.com/thieu1995/permetrics
  • https://github.com/thieu1995/MetaCluster
  • https://github.com/thieu1995/pfevaluator
  • https://github.com/thieu1995/IntelELM
  • https://github.com/thieu1995/reflame
  • https://github.com/thieu1995/MetaPerceptron
  • https://github.com/aiir-team