evolution-opt 0.0.5

Evolution inspired optimisation algorithms

Description

This package is a Python aims at providing a range of nature-inspired optimisation algorithms. The purpose of an optimisation algorithm is to find the maximum or minimum of a function.
Genetic algorithms are particularly useful when it comes to high-dimensional, non-linear and non-convex problems (e.g. finding a needle in a 10-dimensional hay). They have a wide range of application from supply chain optimisation to hyperparameter tuning. This first version includes an implementation of genetic algorithm with "regularized evolution".

Genetic algorithms are very useful in machine learning, especially in hyperparameter tuning. The example folder contains two examples of genetic algorithms used to:

1. Optimise the architecture and hyperparameters of a Neural Network (link)
   
2. Tune the hyperparameters of a Support Vector Machine and XGBoost model (link)

The full documentation can be found here.

Installation

This package can be installed with "pip" or by cloning this repository

$ pip install evolution_opt

Dependencies

To install and run evolution_opt make sure that you have installed the following packages

$ pip install numpy pandas scipy matplotlib

Importing evolution_opt

import numpy as np
import pandas as pd
from evolution_opt.genetic import *

Example Usage

1) Define a function to be optimised

This function has to take a dictionary of parameter as argument:

def difficult_problem(param_dict):
    result = param_dict['x']**2 + (param_dict['y']+1)**2
    if param_dict['luck'] == 'lucky':
        pass
    else:
        result += 10
    return result

This function could be any process that takes parameters as input and outputs a scalar value.

It could evaluate a model's cross-validation score based on given hyperparameter values, a profit/cost function, the efficiency of a resourcing plan... The possibilities are limitless.

2) Define a search space

search_space = [
    Integer(-100,100, 'x'),
    Real(-100,100, 'y'),
    Categorical(['lucky', 'unlucky'], 'luck')
]

The search space can be composed of Integer, Real and Categorical variables. Numeric parameters are initialised with a lower bound, upper bound and a parameter name. Categorical parameters require a list of possible values and a parameter name.

3) Run the evolutionary algorithm

best_params = optimise(difficult_problem,search_space,minimize=True, 
                           population_size=20,n_rounds=500)   

# Prints:
# Number of Iterations: 500
# Best score: 0.00410559779230605
# Best parameters: {'x': -0.0, 'y': -1.0640749388786759, 'luck': 'lucky'}

Credits

• Icon featured in the logo: Icon made by Freepik from www.flaticon.com
• Regularized Evolution Algorithm inspiration: Saltori, Cristiano, et al. "Regularized Evolutionary Algorithm for Dynamic Neural Topology Search." International Conference on Image Analysis and Processing. Springer, Cham, 2019.