pyensemblefs 0.3.14

Ensemble feature selection with bootstrapping, heterogeneous selectors, and stability analysis.

pyensemblefs: a multi-threading Python library for ensemble feature selection.

This repository hosts pyensemblefs, a Python library for ensemble feature selection. Supports heterogeneous ensembles, bootstrapped evaluation, and stability analysis across feature selectors.
It assists researchers in feature selection tasks without requiring significant programming effort.

---

Installation and setup

pip install pyensemblefs

To download the source code, you can clone it from the GitHub repository:

git clone [e-mail]:cdchushig/pyensemblefs.git

Requirements: Python ≥ 3.9, scikit-learn ≥ 1.2

---

Library Highlights

pyensemblefs automatically extracts relevant features in datasets using bootstrapping and ensemble aggregation.

• Intuitive, reproducible workflows: compatible with scikit-learn pipelines.
• Comprehensive documentation: each feature selection and aggregation method is fully described.
• Extensible architecture: easily add custom selection or aggregation strategies.

---

Main Features

• Bootstrap-based ensemble selection: assess variability across resamples.

• Heterogeneous ensembles: combine different feature selectors (e.g., ANOVA, MI, Chi²).

• Unified aggregator API: aggregate results from scores, ranks, or binary supports.

• Visualization tools: plot selection frequency, consensus ranks, and stability matrices.

• Stability metrics: compute indices such as Kuncheva, Jaccard, or Spearman correlation.

• Extensible design: register custom selectors and aggregators via a single factory call.

---

Get started

Example using a built-in dataset and a simple configuration:

import pyensemblefs
from pyensemblefs.datasets import load_pima_dataset

# Load dataset
df = load_pima_dataset()

# Retrieve a pre-defined configuration (e.g., Relief filter)
cfg = pyensemblefs.get_config('relief', n_bootstrap=100, agg='voting')

# Compute feature scores
df_feature_scores = pyensemblefs.compute_scores(cfg, df)

# Extract the most relevant features
top_features, df_filtered = pyensemblefs.extract_features(10, df_feature_scores)

---

Heterogeneous ensemble feature selection

from sklearn.datasets import load_breast_cancer
from sklearn.feature_selection import SelectKBest, f_classif, mutual_info_classif, chi2
from pyensemblefs.ensemble.bootstrapper import Bootstrapper

X, y = load_breast_cancer(return_X_y=True)

fs_methods = [
    SelectKBest(score_func=f_classif, k=10),
    SelectKBest(score_func=mutual_info_classif, k=10),
    SelectKBest(score_func=chi2, k=10),
]

# Assign higher weight to ANOVA
weights = {"SelectKBest": 2.0}

boot = MetaBootstrapper(
    fs_methods,
    n_bootstraps=20,
    n_jobs=4,
    random_state=42,
    strategy="random",
    normalize_scores=True,
    method_weights=weights,
    verbose=True
)

boot.fit(X, y)

print("First bootstrap method:", boot.methods_used_[0])
print("First bootstrap binary support:", boot.results_[0])
if boot.score_mat_ is not None:
    print("Normalized + weighted scores:", boot.score_mat_[0][:10])

---

Visualization of frequency, top-k, and stability

from sklearn.datasets import load_breast_cancer
from sklearn.feature_selection import SelectKBest, mutual_info_classif
from pyensemblefs.ensemble.bootstrapper import Bootstrapper
from pyensemblefs.aggregators.rank import MeanRankAggregator
from pyensemblefs.aggregators.score import MeanAggregator
from pyensemblefs.viz.visualizer import Visualizer

X, y = load_breast_cancer(return_X_y=True)
fs = SelectKBest(score_func=mutual_info_classif, k=10)
boot = Bootstrapper(fsmethod=fs, n_bootstraps=25, n_jobs=2)
boot.fit(X, y)

# Aggregators
rank_agg = MeanRankAggregator(top_k=10).fit(boot.results_)
mean_agg = MeanAggregator(top_k=10).fit(boot.results_)

# Visualizations
Visualizer.stability_heatmap(boot.results_, n_features=X.shape[1])
Visualizer.compare_aggregators_heatmap({
    "RankAggregator": rank_agg.final_ranking_,
    "MeanAggregator": mean_agg.final_ranking_,
}, top_k=10)

All figures are automatically saved under ./images/.

---

Stability Metrics

Stability analysis quantifies how consistent the selected features remain across bootstrap samples.

from pyensemblefs.stability.evaluator import StabilityEvaluator

stab = StabilityEvaluator(metrics="jaccard")
stability_result = stab.compute(boot.results_)
print("Stability (Jaccard):", stability_result.summary)

import numpy as np
from pyensemblefs.stability.helpers import build_similarity
from pyensemblefs.stability.evaluator import StabilityEvaluator

sim_abs = build_similarity(X_tr_musk, mode="abs-corr")

eval12 = StabilityEvaluator(
    metrics="all12",
    mode="subset",
    sim_matrix=sim_abs,
    penalty=1.0
)

res_homo = eval12.compute(R_bin)
res_hetero = eval12.compute(R_bin_hetero)

print("Summary (homogeneous):", res_homo.summary)
print("Summary (heterogeneous):", res_hetero.summary)
print("\nPer-metric (homogeneous):", res_homo.values)
print("Per-metric (heterogeneous):", res_hetero.values)

Available metrics include Jaccard, Dice, Ochiai, Hamming, Novovicova, Davis, Lustgarten, Phi, Kappa, Nogueira, Yu, and Zucknick. They can be directly compared between homogeneous and heterogeneous ensembles.

---

Usage examples (scikit-learn compatible)

from sklearn.datasets import load_breast_cancer
from sklearn.feature_selection import SelectKBest, f_classif
from pyensemblefs.ensemble.bootstrapper import Bootstrapper
from pyensemblefs.aggregators.score import MeanAggregator
from pyensemblefs.aggregators.rank import MeanRankAggregator

X, y = load_breast_cancer(return_X_y=True)

fs = SelectKBest(score_func=f_classif, k=10)
boot = Bootstrapper(fs, n_bootstraps=30, n_jobs=4, random_state=42)
boot.fit(X, y)

# Aggregate scores and ranks
mean_agg = MeanAggregator().fit(boot.results_)
rank_agg = MeanRankAggregator().fit(boot.results_)

print("Consensus scores:", mean_agg.scores_[:10])
print("Consensus ranks:", rank_agg.rank_[:10])

from sklearn.feature_selection import SelectKBest, chi2
from pyensemblefs.ensemble.bootstrapper import Bootstrapper
from pyensemblefs.aggregators.score import MeanAggregator

fs = SelectKBest(score_func=chi2, k=5)
boot = Bootstrapper(fs, n_bootstraps=15, n_jobs=2)
boot.fit(X, y)

mean_agg = MeanAggregator().fit(boot.results_)
print("Consensus Scores (Chi2):", mean_agg.scores_)

---

How It Fits Together

Data → Bootstrapper / MetaBootstrapper
         ↓
Aggregators (Score / Rank / Subset)
         ↓
Visualizer / StabilityEvaluator → Reports

---

Citation

If you use pyensemblefs in academic work, please cite:

@software{pyensemblefs2025,
author = {Peralta-Arboleda B.A, Chushig-Muzo, C.D. and collaborators},
title = {pyensemblefs: Ensemble Feature Selection Library},
year = {2025},
url = {https://github.com/cdchushig/pyensemblefs} }

---

License

This project is licensed under the MIT License – see the LICENSE file for details.