featureranker

Ensemble feature ranking for any numeric feature matrix: tabular datasets, transformer embeddings, pooled hidden states, engineered features. Five methods rank every feature, a typed result object holds the evidence, and weighted rank voting produces one consensus ordering, for classification and regression. Unnamed matrices work directly: pass a numpy array and features get stable generated IDs.

Featured in:

• Machine learning classifiers predict key genomic and evolutionary traits across the kingdoms of life (Nature Scientific Reports, 2023)
• cdsBERT - Extending Protein Language Models with Codon Awareness (bioRxiv, 2023)

Installation

pip install featureranker

Requires Python >= 3.11.

Quick start

from sklearn.datasets import load_breast_cancer
from featureranker import feature_ranking, voting, plot_rankings, plot_after_vote

cancer = load_breast_cancer(as_frame=True)
result = feature_ranking(cancer.data, cancer.target, task="classification")

vote_table = voting(result)                # ["feature", "score"], best first
plot_rankings(result, top_n=15)            # per-method ranks as a dot plot
plot_after_vote(vote_table, top_n=15)      # consensus scores

feature_ranking returns a RankingResult with per-method ranking tables, rank and score matrices, diagnostics, and save/load. Results are deterministic for a given random_state at any n_jobs.

Hugging Face datasets

Pass a Hub path, name the label column, list what to exclude; the rest become features. Categorical columns one-hot expand into named sub-features (sex-Female, workclass-Private) by default.

from featureranker import feature_ranking, get_hf_data

X, y = get_hf_data(
    "scikit-learn/adult-census-income",
    target="income",
    columns_to_drop=["fnlwgt"],
)
result = feature_ranking(X, y, task="classification")

One score from the best features

fit_convex finds the optimal convex combination of the top consensus features: weights >= 0 that sum to one, so each weight is that feature's share of a single interpretable scoring function.

fit = result.fit_convex(X, y, top_n=10)
fit.table()               # ["feature", "weight"], largest first
scores = fit.predict(X)   # rank rows by the combined score

Ranking methods

Examples and the selection-vs-reduction study

20 datasets (transformer embeddings, classical NLP feature spaces, pixels, molecules, sensors) each run the same ablation: every ranking method and the ensemble vote as selectors against PCA, ICA, random projection, kernel PCA, Isomap, UMAP, and t-SNE at matched budgets, scored by linear, kNN, and SVM probes. Findings: the research report. Per-dataset pages live in docs/examples/, regenerated by the scripts in examples/; the ModernBERT sentiment deep dive walks the methodology on unnamed transformer dimensions.

Documentation

The example notebook walks through classification and regression end to end.

Development

git clone https://github.com/lhallee/featureranker.git
cd featureranker
pip install -e ".[dev]"
pytest

Citation

@article{Hallee2023,
  title = {Machine learning classifiers predict key genomic and evolutionary traits across the kingdoms of life},
  volume = {13},
  ISSN = {2045-2322},
  url = {http://dx.doi.org/10.1038/s41598-023-28965-7},
  DOI = {10.1038/s41598-023-28965-7},
  number = {1},
  journal = {Scientific Reports},
  publisher = {Springer Science and Business Media LLC},
  author = {Hallee, Logan and Khomtchouk, Bohdan B.},
  year = {2023},
  month = feb
}

@article{Hallee2023cds,
  title = {cdsBERT - Extending Protein Language Models with Codon Awareness},
  url = {http://dx.doi.org/10.1101/2023.09.15.558027},
  DOI = {10.1101/2023.09.15.558027},
  publisher = {Cold Spring Harbor Laboratory},
  author = {Hallee, Logan and Rafailidis, Nikolaos and Gleghorn, Jason P.},
  year = {2023},
  month = sep
}

License

MIT