pyensemblefs: a multi-threading Python library for ensemble feature selection.

This repository hosts pyensemblefs, a Python library for ensemble feature selection. Supports heterogeneous ensembles, bootstrapped evaluation, and stability analysis across feature selectors.
It assists researchers in feature selection tasks without requiring significant programming effort.

---

Installation and setup

pip install pyensemblefs

To download the source code, you can clone it from the GitHub repository:

git clone git@github.com:cdchushig/pyensemblefs.git

Requirements: Python ≥ 3.9, scikit-learn ≥ 1.2

---

Library Highlights

pyensemblefs automatically extracts relevant features in datasets using bootstrapping and ensemble aggregation.

• Intuitive, reproducible workflows: compatible with scikit-learn pipelines.
• Comprehensive documentation: each feature selection and aggregation method is fully described.
• Extensible architecture: easily add custom selection or aggregation strategies.

---

Main Features

• Bootstrap-based ensemble selection: assess variability across resamples.

• Heterogeneous ensembles: combine different feature selectors (e.g., ANOVA, MI, Chi²).

• Unified aggregator API: aggregate results from scores, ranks, or binary supports.

• Visualization tools: plot selection frequency, consensus ranks, and stability matrices.

• Stability metrics: compute indices such as Kuncheva, Jaccard, or Spearman correlation.

• Extensible design: register custom selectors and aggregators via a single factory call.

---

Get started

Example using a built-in dataset and a simple configuration:

import pyensemblefs
from pyensemblefs.datasets import load_pima_dataset

# Load dataset
df = load_pima_dataset()

# Retrieve a pre-defined configuration (e.g., Relief filter)
cfg = pyensemblefs.get_config('relief', n_bootstrap=100, agg='voting')

# Compute feature scores
df_feature_scores = pyensemblefs.compute_scores(cfg, df)

# Extract the most relevant features
top_features, df_filtered = pyensemblefs.extract_features(10, df_feature_scores)

---

Heterogeneous ensemble feature selection

from sklearn.datasets import load_breast_cancer
from sklearn.feature_selection import SelectKBest, f_classif, mutual_info_classif, chi2
from pyensemblefs.ensemble.bootstrapper import Bootstrapper

X, y = load_breast_cancer(return_X_y=True)

fs_methods = [
    SelectKBest(score_func=f_classif, k=10),
    SelectKBest(score_func=mutual_info_classif, k=10),
    SelectKBest(score_func=chi2, k=10),
]

# Assign higher weight to ANOVA
weights = {"SelectKBest": 2.0}

boot = MetaBootstrapper(
    fs_methods,
    n_bootstraps=20,
    n_jobs=4,
    random_state=42,
    strategy="random",
    normalize_scores=True,
    method_weights=weights,
    verbose=True
)

boot.fit(X, y)

print("First bootstrap method:", boot.methods_used_[0])
print("First bootstrap binary support:", boot.results_[0])
if boot.score_mat_ is not None:
    print("Normalized + weighted scores:", boot.score_mat_[0][:10])

---

Visualization of frequency, top-k, and stability

from sklearn.datasets import load_breast_cancer
from sklearn.feature_selection import SelectKBest, mutual_info_classif
from pyensemblefs.ensemble.bootstrapper import Bootstrapper
from pyensemblefs.aggregators.rank import MeanRankAggregator
from pyensemblefs.aggregators.score import MeanAggregator
from pyensemblefs.viz.visualizer import Visualizer

X, y = load_breast_cancer(return_X_y=True)
fs = SelectKBest(score_func=mutual_info_classif, k=10)
boot = Bootstrapper(fsmethod=fs, n_bootstraps=25, n_jobs=2)
boot.fit(X, y)

# Aggregators
rank_agg = MeanRankAggregator(top_k=10).fit(boot.results_)
mean_agg = MeanAggregator(top_k=10).fit(boot.results_)

# Visualizations
Visualizer.stability_heatmap(boot.results_, n_features=X.shape[1])
Visualizer.compare_aggregators_heatmap({
    "RankAggregator": rank_agg.final_ranking_,
    "MeanAggregator": mean_agg.final_ranking_,
}, top_k=10)

Both heatmaps return (fig, ax) and accept the same presentation/export options:

font_sizes = {
    "title": 18, "x_label": 20, "y_label": 20,
    "x_ticks": 16, "y_ticks": 16, "colorbar_ticks": 16,
}
options = dict(
    title=None, font_sizes=font_sizes,
    max_tick_labels=32, tick_interval=15,
    output_dir="figures", save_png=True, save_pdf=True, dpi=300,
    show=False,
)
fig, ax = Visualizer.stability_heatmap(
    boot.results_, n_features=X.shape[1], filename="stability_heatmap", **options,
)
comparison_fig, comparison_ax = Visualizer.compare_aggregators_heatmap(
    {"RankAggregator": rank_agg.final_ranking_,
     "MeanAggregator": mean_agg.final_ranking_},
    top_k=10, filename="aggregators_comparison_heatmap", **options,
)

This creates both PNG and PDF files in figures/, creating the directory if needed. Figures stay open for further customization; show=True (the default) displays them. Explicit title=None omits the title. If omitted, titles and font sizes retain their previous defaults. Feature labels now default to f1 through fn; pass feature_names in column order to use real names. Bootstrap labels start at 1. Axes with at most 32 cells show all labels; larger axes show every 15th label plus the last, without dropping or moving matrix cells.

Legacy filename="path/plot.png" calls still save one file at 150 DPI by default. Without filename or export flags, figures are not saved automatically. Existing positional arguments and the interpretation of score/index/rank inputs are unchanged.

---

Stability Metrics

Stability analysis quantifies how consistent the selected features remain across bootstrap samples.

from pyensemblefs.stability.evaluator import StabilityEvaluator

stab = StabilityEvaluator(metrics="jaccard")
stability_result = stab.compute(boot.results_)
print("Stability (Jaccard):", stability_result.values["jaccard"])

import numpy as np
from pyensemblefs.stability.helpers import build_similarity
from pyensemblefs.stability.evaluator import StabilityEvaluator

sim_abs = build_similarity(X_tr_musk, mode="abs-corr")

eval12 = StabilityEvaluator(
    metrics="all12",
    mode="subset",
    sim_matrix=sim_abs,
    penalty=1.0,
    threshold=0.7,  # explicitly chosen similarity cutoff
    correction_for_chance={"yu": "estimate"},
    N=1000, seed=42,
)

res_homo = eval12.compute(R_bin)
res_hetero = eval12.compute(R_bin_hetero)

print("\nPer-metric (homogeneous):", res_homo.values)
print("Per-metric (heterogeneous):", res_hetero.values)

Available metrics include Jaccard, Dice, Ochiai, Hamming, Novovicova, Davis, Lustgarten, Phi, Kappa, Nogueira, Yu, and Zucknick. Compare each metric between ensembles; do not average different metrics into one scientific score. Yu and Zucknick use explicitly identified stabm 1.2.2 adaptations. See definitions, validation and API migration.

---

Usage examples (scikit-learn compatible)

from sklearn.datasets import load_breast_cancer
from sklearn.feature_selection import SelectKBest, f_classif
from pyensemblefs.ensemble.bootstrapper import Bootstrapper
from pyensemblefs.aggregators.score import MeanAggregator
from pyensemblefs.aggregators.rank import MeanRankAggregator

X, y = load_breast_cancer(return_X_y=True)

fs = SelectKBest(score_func=f_classif, k=10)
boot = Bootstrapper(fs, n_bootstraps=30, n_jobs=4, random_state=42)
boot.fit(X, y)

# Aggregate scores and ranks
mean_agg = MeanAggregator().fit(boot.results_)
rank_agg = MeanRankAggregator().fit(boot.results_)

print("Consensus scores:", mean_agg.scores_[:10])
print("Consensus ranks:", rank_agg.rank_[:10])

from sklearn.feature_selection import SelectKBest, chi2
from pyensemblefs.ensemble.bootstrapper import Bootstrapper
from pyensemblefs.aggregators.score import MeanAggregator

fs = SelectKBest(score_func=chi2, k=5)
boot = Bootstrapper(fs, n_bootstraps=15, n_jobs=2)
boot.fit(X, y)

mean_agg = MeanAggregator().fit(boot.results_)
print("Consensus Scores (Chi2):", mean_agg.scores_)

---

How It Fits Together

Data → Bootstrapper / MetaBootstrapper
         ↓
Aggregators (Score / Rank / Subset)
         ↓
Visualizer / StabilityEvaluator → Reports

---

Citation

If you use pyensemblefs in academic work, please cite:

@software{pyensemblefs2025,
author = {Peralta-Arboleda B.A, Chushig-Muzo, C.D, Milara E, Conceição Granja, Soguero-Ruiz C.},
title = {pyensemblefs: Ensemble Feature Selection Library},
year = {2025},
url = {https://github.com/cdchushig/pyensemblefs} }

---measures

License

This project is licensed under the MIT License – see the LICENSE file for details.