🔥 Bankai Boruta

BankaiBoruta is a fast, scalable feature selection library built for large datasets.

It combines the Boruta all-relevant selection algorithm with the native Rust backend of Bankai Random Forest. Classic Boruta is the default; greedy=True enables the fixed-round Greedy Boruta variant. BankaiRF supplies exact or binned Extra Trees, gain importance and parallel tree training through a scikit-learn-compatible selector. In 0.1.0a6, the Rust selector forwards supported forest parameters from the caller.

⚠️ Bankai Boruta is in Alpha. Its API and fitted objects may change.

Install

Python 3.11 or newer is required. The release installation command is:

python -m pip install bankai-boruta

Quick start

This example generates a binary classification dataset with 1,000 rows and 20 features. It fits selection on the training partition and applies the same feature mask to the held-out partition. A pandas DataFrame supplies feature names.

import pandas as pd
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split

from bankai_boruta import BankaiBoruta

values, y = make_classification(
    n_samples=1_000,
    n_features=20,
    n_informative=5,
    n_redundant=3,
    random_state=2077,
)
X = pd.DataFrame(values, columns=[f"feature_{i}" for i in range(values.shape[1])])
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=2077,
)

selector = BankaiBoruta(
    n_estimators="auto",
    perc=100,
    alpha=0.05,
    max_iter=100,
    random_state=2077,
)
selector.fit(X_train, y_train)

X_train_selected = selector.transform(X_train)
X_test_selected = selector.transform(X_test)
print(f"Selected features: {selector.n_features_}")
print(selector.get_feature_names_out())
print(f"Train shape: {X_train_selected.shape}; test shape: {X_test_selected.shape}")

selection = pd.DataFrame({
    "feature": selector.feature_names_in_,
    "confirmed": selector.support_,
    "tentative": selector.support_weak_,
    "rank": selector.ranking_,
})
print(selection.sort_values("rank"))

transform returns confirmed features only. fit_transform(X, y) combines the fit and transform steps. NumPy arrays and pandas DataFrames are supported; pandas inputs are converted to NumPy for training.

Selector reference

BankaiBoruta follows scikit-learn's estimator conventions and uses parameter names familiar from BorutaPy. Its constructor defaults are:

from bankai_boruta import BankaiBoruta

selector = BankaiBoruta(
    estimator=None,
    n_estimators="auto",
    perc=100,
    alpha=0.05,
    two_step=True,
    max_iter=100,
    random_state=None,
    verbose=0,
    early_stopping=False,
    n_iter_no_change=20,
    greedy=False,
    bonferroni=True,
    importance_getter="auto",
    store_history=False,
)

The default classifier is configured as follows; remaining parameters follow BankaiRF a15 defaults:

from bankai_random_forest import BankaiRandomForestClassifier

forest = BankaiRandomForestClassifier(
    extra_trees=True,
    extra_trees_backend="binned",
    max_bins=63,
    importance_type="gain",
    max_depth=6,
    n_jobs=-1,
)

Pass a classifier for full forest customization, then use nested parameters such as estimator__max_depth with set_params or parameter search. Weights and original training labels, including text labels, are passed to the Python adapter without re-encoding the labels supplied to the classifier.

forest = BankaiRandomForestClassifier(
    n_estimators=100,
    max_depth=5,
    class_weight="balanced",
    n_jobs=-1,
)
selector = BankaiBoruta(forest, n_estimators=None, random_state=42)
selector.set_params(estimator__max_depth=7)
selector.fit(X_train, y_train)

Native forest controls

The native selector forwards the supplied BankaiRandomForestClassifier settings for the supported native subset. Both exact and binned Extra Trees are available. Exact native selection currently requires float32 input; exact float64 input uses the adapter so BankaiRF can preserve its float64 semantics. This example runs the exact backend with custom bootstrap sampling, criterion, and tree constraints:

forest = BankaiRandomForestClassifier(
    n_estimators=150, max_depth=8, max_bins=None,
    extra_trees=True, extra_trees_backend="exact", importance_type="gain",
    criterion="entropy", min_samples_split=4, min_samples_leaf=2,
    max_leaf_nodes=128, max_samples=0.8, bootstrap=True,
    n_jobs=4, max_features=0.15,
)
selector = BankaiBoruta(forest, n_estimators=150, random_state=42)
selector.fit(X_train.astype("float32"), y_train)
assert selector.diagnostics_["path"] == "rust"

Worker and feature controls use the pinned a15 forest's validation and resolution rules: n_jobs=None means one worker, positive values specify the worker count, and negative values resolve relative to the CPU count. max_features accepts "sqrt", "log2", a positive integer, a fraction in (0, 1], or None. Feature counts resolve against each round's current real-plus-shadow width, including after Classic rejects columns, and fractional counts are floored with a minimum of one. Integer counts must fit that round's width. The resolved worker count controls native preparation and forest training; tree parallelism is capped by the number of trees.

The Rust path also forwards criterion, min_samples_split, min_samples_leaf, min_weight_fraction_leaf, min_impurity_decrease, max_leaf_nodes, bootstrap, max_samples, and the binned backend's binning_strategy, bin_sample_size, and binning_seed. Parameters that are not represented by this native boundary (including sample/class weights, warm_start, OOB scoring, and monotonic constraints) use the Python adapter. To benchmark the materialized Python adapter with the same forest, set importance_getter="feature_importances_" and assert selector.diagnostics_["path"] == "python". Native virtual shadows avoid the augmented dense matrix. Since a5, native float32 and float64 arrays retain their dtype; other inputs and all adapter inputs use float64. Each value is promoted to f64 only when read by Rust. Permutations and their reusable row index buffer use uint32 for up to 2**32 rows, otherwise int64 within platform addressing limits. Bins remain materialized and training is unchanged.

fit(X, y, sample_weight=None) accepts non-negative sample weights with positive total weight. Classic selection counts hits and uses binomial tests; its final median comparison marks surviving tentative features as weak. The fitted selector exposes:

The selector supports scikit-learn pipelines and cloning. A fresh clone is used for each round when a supplied classifier enables warm_start, preventing trees from retaining previous shadows. estimator_ contains cloned configuration on the Rust path and the last round's fitted classifier on the adapter path.

BankaiRF backend and performance foundations

The backend combines several sources of computational efficiency:

The scientific papers and author article are linked under References. The LightGBM reference describes binning and histogram principles; the a15 implementation determines the actual forest storage and split behavior. The default binned Extra Trees path uses its own column-bin statistics rather than the global histogram caches of BankaiRF's ordinary Random Forest path.

Rust execution is selected automatically for supported exact or binned Extra Trees gain configurations. Supported forest settings are forwarded into the native fit; unsupported settings, weights, monotonic constraints and custom getters use the Python adapter, preserving estimator parameters and a15 validation. Shadow monotonic constraints are always zero. diagnostics_ records the path and its reason.

Both Python and Rust dependencies are aligned to BankaiRF 0.1.0a15; the Rust revision is caa46c22ef93434cd87319490513e877d4378564. Training storage is rebuilt each round as required by a15. Immutable inputs and permutation buffers are reused within a fit; training caches are not shared across independent fits. diagnostics_ also reports input_dtype (received dtype), native_input_dtype (validated dtype on the native path, otherwise None), input_conversion_bytes (size of the converted input buffer, zero for a shared NumPy input), permutation_dtype, and permutation_bytes (largest per-round size of permutations plus the reusable row-index buffer during the fit). These are explicit buffer sizes, not RSS or total allocations. The existing copied_bytes remains cumulative augmented float-matrix writes across rounds; it excludes input conversions and permutations. For a 100,000 × 500 float32 array, a5 removes 400,000,000 input-conversion bytes and 200,400,000 permutation bytes versus a4: approximately 572.6 MiB. RSS and timings require measurement. Only local results are covered by the a5 report; the four-profile Linux campaign remains pending.

Copy metrics cover augmented float-matrix writes, while a15's internal bin and cache allocations remain part of preparation/training time and measured RSS.

See the reproducible validation report for Rust/adapter importance parity, reference-algorithm comparisons, timing, peak memory, recall and false positives. The measurements describe specific datasets and seeds; actual speed and selection quality depend on the workload and configuration.

Migration from a2

estimator is now the first argument. Configure forest parameters on that classifier instead of the removed selector arguments forest_backend, pruning, importance, importance_scale, input_mode, max_features, max_depth, n_jobs, and max_bins. greedy=True enables hit-once selection; the former aggressive pruning rule is a different algorithm. The bankai-rf, polars, and arrow-test extras were removed. This release supports NumPy and pandas; Polars and Arrow integration is outside its scope.

Greedy mode

selector = BankaiBoruta(
    greedy=True,
    bonferroni=True,
    alpha=0.05,
    random_state=2077,
)
X_selected = selector.fit_transform(X_train, y_train)
print(f"Rounds: {selector.n_iter_}; effective budget: {selector.max_iter_}")

Greedy retains every original feature throughout the campaign. With p original features, it runs K = ceil(log2(p / alpha)) rounds when bonferroni=True, or K = ceil(log2(1 / alpha)) otherwise. The exact floating-point ratio is used to compute the ceiling safely at power-of-two boundaries and very small alpha.

A feature is confirmed after at least one importance above the shadow percentile. Features with no hits are rejected at completion. support_weak_ is always false and ranking_ uses 1 for confirmed and 2 for rejected features. max_iter, two_step, early_stopping, and n_iter_no_change do not control this mode. Greedy relaxes confirmation and can select more noise; the validation report measures that tradeoff without assuming a universal recall advantage.

Development

Local builds need Git, Python 3.11+, stable Rust, and uv. Install Xcode command-line tools on macOS or the Rust MSVC build tools on Windows.

git clone https://github.com/breno-jesus-fernandes/bankai-boruta.git
cd bankai-boruta
git switch feature/greedy-bankai-a15
uv python install 3.11
uv sync --locked --no-install-project --extra test
uv run --no-sync maturin develop --release --locked

Run the Python and Rust checks, then reproduce the benchmark campaign as needed:

uv run --no-sync pytest -q
uv run --no-sync cargo test --workspace --locked
uv run --no-sync python benchmarks/validate_a3.py
uv run --no-sync python benchmarks/summarize_a3.py

Both lockfiles are committed. CI tests Python 3.11–3.14 and builds ABI3 wheels for Linux x86_64/ARM64, macOS x86_64/ARM64 and Windows x86_64, with installed-wheel smoke tests. The scheduled BankaiRF release check reports upstream versions; upgrades must align both dependency pins and pass compatibility/performance checks.

Read the development guide, contribution guide, and Code of Conduct.

References

1. Nicolas Vana (2025), The Greedy Boruta Algorithm: Faster Feature Selection Without Sacrificing Recall, author article in Towards Data Science. The published GreedyBoruta 1.0.0 implementation supplies the fixed-round, hit-once reference used here.
2. Pierre Geurts, Damien Ernst and Louis Wehenkel (2006), Extremely randomized trees, Machine Learning 63, 3–42. Author preprint and publication record.
3. Guolin Ke et al. (2017), LightGBM: A Highly Efficient Gradient Boosting Decision Tree, NeurIPS 30, particularly Section 2.1 on histogram-based learning. Paper PDF and official histogram documentation.
4. Miron B. Kursa and Witold R. Rudnicki (2010), Feature Selection with the Boruta Package, Journal of Statistical Software 36(11), 1–13. BorutaPy implementation.

License

Bankai Boruta is distributed under GPL-3.0-or-later. Its BankaiRF Rust backend includes the maintained GPL-3.0-or-later XRF fork.