sklearn-genetic-opt 0.13.2

Scikit-learn models hyperparameter tuning and feature selection using evolutionary algorithms

sklearn-genetic-opt

Hyperparameter tuning and feature selection for scikit-learn models using genetic algorithms.

sklearn-genetic-opt is a scikit-learn-compatible optimization toolkit for users who want a smarter alternative to GridSearchCV and RandomizedSearchCV. Its genetic algorithm evaluates complete parameter configurations — finding regions where learning_rate × n_estimators or C × gamma are jointly optimal, something one-parameter-at-a-time approaches miss. It also provides GAFeatureSelectionCV, a wrapper-based selector that searches the full space of feature subsets simultaneously instead of eliminating features one at a time.

Why use sklearn-genetic-opt?

• Drop-in scikit-learn API — GASearchCV has the same fit / predict / best_params_ interface as GridSearchCV; replace it in one line.

• Handles interacting parameters — genetic algorithms evaluate complete configurations, naturally finding cross-parameter sweet spots that random or grid search miss.

• Joint feature selection and tuning — run GAFeatureSelectionCV and GASearchCV in a two-stage workflow; no separate feature-selection library needed.

• Mixed search spaces — Integer, Continuous (uniform or log-uniform), and Categorical types in the same search.

• Smart initialization — Latin hypercube seeding, estimator defaults, warm-start configs, and duplicate avoidance give the first generation a head start over random initialization.

• Early stopping callbacks — ConsecutiveStopping, DeltaThreshold, and TimerStopping end the search automatically when it converges or runs out of time.

• Adaptive schedules — crossover and mutation rates anneal over generations, shifting from exploration to exploitation.

• Optimization history and plots — per-generation fitness, diversity, and telemetry stored in history; built-in plots visualize the full search.

• MLflow integration — every evaluated candidate is automatically logged as a child run for experiment comparison.

• Parallel execution — n_jobs=-1 parallelizes candidate or fold evaluation.

When should you use it?

• Your model is expensive to train and you can only afford 50–200 total evaluations.

• Your search space has 5+ hyperparameters that interact (gradient boosting, SVM, regularized regression).

• You want feature selection and hyperparameter tuning in a single reproducible workflow.

• You want optimization history, convergence plots, callbacks, or MLflow tracking built in.

• You have known-good configurations to warm-start from (prior runs, published defaults).

• GridSearchCV is too slow and RandomizedSearchCV keeps returning similar bad results.

When should you NOT use it?

• You need a fast baseline — start with a fixed configuration or RandomizedSearchCV(n_iter=20); it’s faster and good enough to validate your pipeline.

• Your grid is tiny (fewer than 50 combinations) — GridSearchCV covers it exhaustively and is simpler to reason about.

• Your model and dataset are fast (< 1 s per fit) — the overhead of managing a population adds up relative to just running all combinations.

• You need distributed optimization across a cluster — use Optuna with its distributed backends.

• You need strict Bayesian guarantees on the exploration-exploitation trade-off — use Optuna (TPE) or scikit-optimize.

How it compares

Tool	Best for	Key limitation	Where sklearn-genetic-opt helps
GridSearchCV	Small, fully discrete grids; guaranteed complete coverage	Combinatorial explosion on 4+ params; no continuous params natively	Large, mixed, or continuous spaces with interacting parameters
RandomizedSearchCV	Larger budgets; simple independent parameter spaces	No learning from past evaluations; treats each parameter independently	Exploits cross-parameter interactions; adaptive schedules; early stopping
Optuna	Sequential Bayesian (TPE) search; distributed optimization; neural architecture search	No native sklearn cross-validation; no built-in wrapper feature selection	Drop-in sklearn API; built-in GAFeatureSelectionCV
RFE	Greedy feature elimination for models with coef_ or feature_importances_	Greedy and sequential; can miss non-greedy optimal subsets	Evaluates all subsets simultaneously; works with any estimator
SelectFromModel	Fast embedded selection via a threshold on feature importance	Tied to model-specific importances; no cross-estimator comparison	Estimator-agnostic wrapper; combinable with hyperparameter tuning in one workflow
sklearn-genetic-opt	Large or mixed spaces; joint feature + parameter search; history, plots, callbacks	Slower than Bayesian methods on small smooth spaces; population size needs tuning	—

Quick Start

Install

pip install sklearn-genetic-opt
# With optional plotting, MLflow, and TensorBoard extras:
# pip install sklearn-genetic-opt[all]

Hyperparameter search

from sklearn.datasets import load_breast_cancer
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import StratifiedKFold, train_test_split

from sklearn_genetic import GASearchCV, EvolutionConfig, RuntimeConfig
from sklearn_genetic.space import Categorical, Continuous, Integer

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=42
)

search = GASearchCV(
    estimator=RandomForestClassifier(random_state=42),
    param_grid={
        "n_estimators": Integer(50, 300),
        "max_depth":    Integer(3, 20),
        "max_features": Continuous(0.1, 1.0),
        "class_weight": Categorical([None, "balanced"]),
    },
    cv=StratifiedKFold(n_splits=5, shuffle=True, random_state=42),
    scoring="roc_auc",
    evolution_config=EvolutionConfig(population_size=15, generations=12),
    runtime_config=RuntimeConfig(n_jobs=-1, verbose=True),
    random_state=42,
)
search.fit(X_train, y_train)

print(search.best_params_)          # best hyperparameter configuration
print(search.best_score_)           # best cross-validated ROC-AUC
print(search.score(X_test, y_test)) # test-set score

Feature selection

import numpy as np
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import StratifiedKFold, train_test_split

from sklearn_genetic import GAFeatureSelectionCV, EvolutionConfig, RuntimeConfig

X, y = load_iris(return_X_y=True)
# Add 8 noise features — the selector should drop them
X = np.hstack([X, np.random.default_rng(42).uniform(0, 1, (X.shape[0], 8))])
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=42
)

selector = GAFeatureSelectionCV(
    estimator=RandomForestClassifier(n_estimators=100, random_state=42),
    cv=StratifiedKFold(n_splits=5, shuffle=True, random_state=42),
    scoring="accuracy",
    evolution_config=EvolutionConfig(population_size=20, generations=15),
    runtime_config=RuntimeConfig(n_jobs=-1, verbose=True),
    random_state=42,
)
selector.fit(X_train, y_train)

print(selector.support_)                    # boolean mask of selected features
print(selector.score(X_test, y_test))       # accuracy on selected features

Read the full Getting Started guide →

What you can do

Track optimization progress generation by generation. The fitness curve shows when the search converges so you know whether to add more generations or stop early.

See where the search explored and which parameter combinations scored highest. The scatter plot reveals the productive region of the learning_rate × n_estimators interaction — a band a one-at-a-time sweep cannot find.

Inspect the full search in one view. The search overview panel combines scores, parameter distributions, diversity, and candidate decisions.

See the full Plotting Gallery →

Common use cases

Hyperparameter tuning

• Tune RandomForestClassifier — 7-parameter joint search, classification and regression

• Tune XGBoost — 9 interacting params, n_jobs=1 oversubscription fix, interaction visualization

• Tune LightGBM — num_leaves / max_depth interaction, parameter scatter plots

• Tune CatBoost — bagging_temperature, border_count, GPU tip

• Tune Gradient Boosting (sklearn) — HistGBM vs classic GBM, speed comparison

• Tune Logistic Regression — solver / penalty compatibility, multi-penalty with SAGA

• Tune SVM (C, kernel, gamma) — C–gamma interaction, mandatory Pipeline + StandardScaler

• Tune a scikit-learn Pipeline — step prefix patterns, ColumnTransformer

• Tune for imbalanced datasets — class_weight as a search param, balanced accuracy

Feature selection

• Feature selection with genetic algorithms — 3-stage: select, retune, validate

• Combine feature selection + hyperparameter tuning — two-stage pipeline recipe

Experiment tracking and tooling

• MLflow experiment tracking — log every candidate as a child run

• Callbacks and early stopping — ConsecutiveStopping, TimerStopping, DeltaThreshold

• Checkpointing and resume — save and continue long searches

• Visualize optimization progress — full gallery of available plots

Browse all Recipes → Browse all Tutorials →

Learning paths

New user

Install → Getting Started with GASearchCV → How Hyperparameter Optimization Works → Pick a Recipe

ML practitioner

When to Use Genetic Algorithm Search → Choosing the Right Search Space → Model-specific Tutorials → Optuna vs sklearn-genetic-opt

Contributor

Contributing guide → Open issues → Benchmarks documentation

Benchmarks

The repository includes benchmark scripts that compare GASearchCV against RandomizedSearchCV, GridSearchCV, and Optuna (TPE) using the Bayesmark experimental design — same datasets, same search spaces, equal evaluation budget:

pip install sklearn-genetic-opt[benchmark]
python benchmarks/benchmark_bayesmark.py --quick

See the Benchmarks documentation for methodology and full results.

Installation

# Core package
pip install sklearn-genetic-opt

# With plotting, MLflow, and TensorBoard:
pip install sklearn-genetic-opt[all]

# conda
conda install -c conda-forge sklearn-genetic-opt

Requires Python ≥ 3.12 and scikit-learn ≥ 1.5.0. See Installation for the full requirements table and optional extras.

Contributing

Contributions of all sizes are welcome — from fixing a typo to adding a new tutorial or benchmark.

Good ways to start:

• Add a Recipe for an estimator or workflow not yet covered — see existing Recipes.

• Write or improve a tutorial (new models, edge cases, regression examples).

• Test with a new estimator from another framework and report the results.

• Add a benchmark comparing search methods on a real dataset.

• Fix typing, CI, or formatting — black . keeps the style consistent.

• Answer questions in open issues.

• Share your work — add a blog post, article, or video to the external references file.

git clone https://github.com/rodrigo-arenas/Sklearn-genetic-opt.git
cd Sklearn-genetic-opt
pip install -r dev-requirements.txt
pytest sklearn_genetic

Read the contribution guide before opening a pull request. If you are not sure where to start, open an issue and ask — small contributions are very welcome.

If sklearn-genetic-opt helps your work, consider starring the repository so others can find it.

Links

• Documentation

• API reference

• Tutorials

• Recipes

• Grid vs Random vs Bayesian vs Genetic comparison

• Common hyperparameter tuning mistakes

• Troubleshooting

• Release notes

• PyPI

• Issues