evolearn 1.0.8

Machine learning related GA tools.

Evolutionary Algorithm For Machine Learning

Installation

To use evolearn, first install it using pip:

pip install evolearn

Genetic Optimization CV

To perform hyperparameter tuning using genetic algoritm, you need to first import other modules from

1. evolearn.hyperparameter_tuning.initialization
2. evolearn.hyperparameter_tuning.evaluation
3. evolearn.hyperparameter_tuning.selection
4. evolearn.hyperparameter_tuning.mating
5. evolearn.hyperparameter_tuning.reproduction
6. evolearn.hyperparameter_tuning.mutation
7. evolearn.hyperparameter_tuning.environment (optional)
8. evolearn.hyperparameter_tuning.genetic_hyperparameter_tuning

Although the modules from environment are optional for you to determine to use them in your search or not, the searching might end up stopping early or not finding the ideal results. These modules can help to prevent pre-mature convergence and also control other hyperparameters for GA.

For example:

from evolearn.hyperparameter_tuning.initialization import Genes
from evolearn.hyperparameter_tuning.evaluation import FitnessFunction
from evolearn.hyperparameter_tuning.selection import (RankSelection,
                                           				RouletteWheelSelection,
                                            				SteadyStateSelection,
                                            				TournamentSelection,
                                            				StochasticUniversalSampling,
                                            				BoltzmannSelection
                                            				)
from evolearn.hyperparameter_tuning.mating import MatingFunction
from evolearn.hyperparameter_tuning.reproduction import (KPointCrossover,
                                               			      LinearCombinationCrossover,
                                               			      FitnessProportionateAverage
                                             				      )
from evolearn.hyperparameter_tuning.mutation import (Boundary,
                                           				Shrink
                                          				 )
from evolearn.hyperparameter_tuning.environment import (AdaptiveReproduction,
                                              			     AdaptiveMutation,
                                              			     Elitism
                                              			     )
from evolearn.hyperparameter_tuning.genetic_hyperparameter_tuning import GenesSearchCV
from sklearn.ensemble import RandomForestRegressor
search_space_rf = {
          'max_depth':(1, 16, 'uniform'),
          'n_estimators':(100, 1000, 'uniform'),
          'criterion':('squared_error', 'absolute_error', 'poisson')
      }  
opt = GenesSearchCV(
      n_gen=10,
      initialization_fn=Genes(search_space=search_space_rf, pop_size=30),
      fitness_fn=FitnessFunction(
          estimator=RandomForestRegressor(n_jobs=-1),
          cv=3,
          scoring='neg_mean_absolute_error',
      ),
      selection_fn=StochasticUniversalSampling(.7),
      mating_fn=MatingFunction(increst_prevention=False),
      reproduction_fn=KPointCrossover(1),
      mutation_fn=Shrink(),
      adaptive_population=AdaptiveReproduction(10),
      elitism=Elitism(),
      adaptive_mutation=AdaptiveMutation()
  )   
opt.fit(X_train, y_train)

Max Fitness: -2023.200579609583
{'max_depth': 5, 'n_estimators': 561, 'criterion': 'absolute_error'}

The choices of selection_fn, reproduction_fn, mutation_fn are actually up to your personal preference. One can pick what they believe are most benefit to their searching preocess.

Genetic Feature Selection

To perform feature selection using genetic algoritm, you need to first import other modules from

1. evolearn.feature_selection.initialization
2. evolearn.feature_selection.evaluation
3. evolearn.feature_selection.selection
4. evolearn.feature_selection.mating
5. evolearn.feature_selection.reproduction
6. evolearn.feature_selection.mutation
7. evolearn.feature_selection.environment (optional)
8. evolearn.feature_selection.genetic_feature_selection

The modules looks similar to those modules from the GenesSearchCV section, but in fact their internal mechanisim work slightly differently. You need to be ware of importing the wrong modules when using genetic feature selection.

For example:

from evolearn.feature_selection.initialization import Genes
from evolearn.feature_selection.evaluation import FitnessFunction
from evolearn.feature_selection.selection import (RankSelection,
                                                   RouletteWheelSelection,
                                                   SteadyStateSelection,
                                                   TournamentSelection,
                                                   StochasticUniversalSampling,
                                                   BoltzmannSelection
                                                   )
from evolearn.feature_selection.mating import MatingFunction
from evolearn.feature_selection.reproduction import KPointCrossover
from evolearn.feature_selection.mutation import (BitStringMutation,
                                                ExchangeMutation,
                                                ShiftMutation
                                                )

from evolearn.feature_selection.environment import (AdaptiveReproduction,
                                                AdaptiveMutation,
                                                Elitism
                                                )

from evolearn.feature_selection.genetic_feature_selection import GeneticFeatureSelectionCV
from sklearn.ensemble import RandomForestRegressor
opt = GeneticFeatureSelectionCV(
   n_gen=10,
   initialization_fn=Genes(pop_size=50),
   fitness_fn=FitnessFunction(
       estimator=RandomForestRegressor(n_jobs=-1),
       cv=3,
       scoring='neg_mean_absolute_error'
   ),
   selection_fn=RouletteWheelSelection(.7),
   mating_fn=MatingFunction(),
   reproduction_fn=KPointCrossover(k=4),
   mutation_fn=BitStringMutation(),
   adaptive_population=None,
   elitism=None,
   adaptive_mutation=None
   )

opt.fit(X_train, y_train)
print(opt.best_fitness_)
print(opt.best_params_)

-2797.7245589631652
{'age': True, 'sex': False, 'bmi': True, 'children': True, 'smoker': True, 'region': False}