pywinEA 0.0.2

Package with basic implementations of mono and multi-objective genetic algorithms for feature selection.

PyWinEA

Python package with lightweight implementations of genetic algorithms for classification/regression tasks.

Description

The pywinEA module is a native python implementation of some of the most widely used genetic algorithms. This package has been developed on the top of scikit-learn which allows to use any model already implemented. This module aims to provide a good alternative to other feature selection techniques with full scikit-learn compatibility.

Why evolutionary algorithms?

One of the first stages in the development of any machine learning model is to filter out redundant and/or irrelevant attributes. However, the complexity of finding the best combination of attributes is most often an NP problem.

Among the most frequent feature selection strategies are embedded methods. These methods combine a heuristic search strategy with a classification/regression model. This is where genetic algorithms come into play. This type of strategy represents one of the best alternatives to address the immense search space generally reaching good solutions.

Install

Dependencies

PyWinEA requires:

• Python (>= 3.6)
• NumPy (>= 1.13.3)
• SciPy (>= 0.19.1)
• Scikit-learn (>= 0.20.0)
• tqdm (>= 4.42.1)
• matplotlib (>= 3.1.3)
• pandas (>= 1.0.1)

pip install pywinEA

It is possible that older versions of the packages listed above may work. However, full compatibility is not guaranteed.

Usage

Examples of the basic use of the package can be found in the notebooks directory. A diagram of the module structure is also shown below. For more advanced use it is recommended to look at the documentation.

Additionally by using the classes defined in the interface subpackage it is possible to implement new operators, algorithms, etc. Feel free to add things.

The following is an example of the most basic implementation of a genetic algorithm.

# Basic GA
from pywinEAt.algorithm import GA
from sklearn.naive_bayes import GaussianNB  # Fitness function

# Data loading and processing...

POPULATION_SIZE = 50
GENERATIONS = 200
FITNESS = GaussianNB()
CV = 5
ANNHILATION = 0.1
ELITISM = 0.2
MUTATION = 0.1

ga_basic = GA(
    population_size=POPULATION_SIZE, generations=GENERATIONS, cv=CV,
    fitness=FITNESS, annihilation=ANNHILATION, elitism=ELITISM, 
    mutation_rate=MUTATION, positive_class=1, id="BasicGA"
)

ga_basic.set_features(feature_names)   # Selection of the feature names (optional)

# Fit algorithm 
ga_basic.fit(x_data, y_data) 

# Get the names of the most relevant features
ga_basic.best_features 

This type of algorithm usually works well, however we may be interested in maximizing two objectives, for example the performance of the classifier (maximization) and the number of characteristics (minimization). In this case the multi-target algorithms (NSGA2 and SPEA2) are the best alternative.

# Multi-objective
from pywinEA.algorithm import NSGA2
from sklearn.naive_bayes import GaussianNB  # Fitness function

# Data loading and processing...

POPULATION_SIZE = 50
GENERATIONS = 200
FITNESS = GaussianNB()
CV = 5
MUTATION = 0.1

nsga = NSGA2(
    population_size=POPULATION_SIZE, generations=GENERATIONS, 
    fitness=FITNESS, cv=CV, mutation_rate=MUTATION, 
    optimize_features=True, positive_class=1, id="NSGA2"
)

nsga.set_features(feature_names)   # Selection of the feature names (optional)

# Fit algorithm 
nsga.fit(x_data, y_data) 

# Get the names of the most relevant features
nsga.best_features 

The result of the multi-objective algorithms is a non-dominant front of solutions. For example:

For more examples it is recommended to take a look at the notebooks.

Module structure

Notes

The package is still in testing, it is possible to find some unexpected errors. Any problem 👉 issues

License

MIT ©