PrismBoost

PrismBoost is a gradient-boosting classifier/regressor that uses SEFR oblique splits at internal nodes (hyperplane splits instead of axis-aligned thresholds), with an optional fast C++ backend.

from prismboost import PrismBoostClassifier
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=0, stratify=y
)

clf = PrismBoostClassifier(random_state=0)   # capacity parameters adapt to the data
clf.fit(X_train, y_train)
print(clf.score(X_test, y_test))
print(clf.auto_config_)                      # what was chosen for this training set

Adaptive defaults

Every capacity parameter defaults to "auto" and is resolved from the training-set shape at fit time, the way CatBoost adapts its learning rate to dataset size. The rules were calibrated on the per-dataset Optuna optima of the 121 PMLB benchmark datasets, so an untuned model starts near a sensible configuration instead of one fixed point that under-fits large data and over-fits small data.

Parameter	"auto" rule
n_estimators	200
learning_rate	0.05 below 500 rows, else 0.1 (keeps learning_rate * n_estimators near the tuned optimum)
max_depth	4 below 500 rows, else 6
min_samples_leaf	sqrt(n_samples) / 2, clipped to [5, 30]
min_samples_split	2 * min_samples_leaf
subsample	0.8, or 1.0 below 100 rows
split_mode	hybrid up to 50 features, else hybrid_sampled

Only the shape of X is used, never y, so folds of equal size resolve identically and no label information leaks. Passing an explicit value disables adaptation for that parameter alone:

clf = PrismBoostClassifier(max_depth=3, random_state=0)  # depth fixed, the rest still auto
clf.fit(X_train, y_train)
clf.max_depth_, clf.n_estimators_   # (3, 200)

Class imbalance is deliberately left alone (class_weight=None), matching XGBoost and CatBoost defaults. To reproduce pre-0.2 behaviour, pass the old values explicitly: n_estimators=100, learning_rate=0.1, max_depth=3, min_samples_leaf=10, min_samples_split=2, subsample=1.0, split_mode="hybrid_sampled".

Leaf regularization (reg_lambda)

reg_lambda is the L2 penalty on leaf weights, the same knob as XGBoost's reg_lambda. It enters the Newton step as sum(w r) / (sum(w h) + reg_lambda) and the split gain as G^2 / (H + reg_lambda).

It defaults to 0.0, so results from earlier versions are unchanged unless you set it. Raising it matters most on imbalanced classification: a nearly pure leaf has h = p (1 - p) close to zero, so the unregularized Newton step is large and the accumulated scores can saturate the softmax. On a 3-class problem at a 2/10/88 class split, 200 rounds:

reg_lambda	test log loss
0.0	0.399
1.0	0.233
5.0	0.207
20.0	0.188

(Predicting the class prior scores 0.437 on the same split.) Accuracy-style metrics are much less sensitive to this than log loss is, which is why the PMLB study above, scored on macro-F1 and ROC-AUC, did not surface it.

Why oblique boosting?

Axis-aligned GBDTs approximate curved boundaries with staircases. PrismBoost fits linear (oblique) splits, so decision surfaces on non-linear problems are typically smoother.

Predicted-probability surfaces on moons: PrismBoost (left) vs XGBoost (right).

Decision boundaries on six synthetic 2D datasets (rows) across classifiers (columns). Lower surface roughness S is smoother.

Benchmark highlights (PMLB)

Evaluated on 121 Penn Machine Learning Benchmark classification datasets against strong baselines (CatBoost, LightGBM, LightGBM-linear, XGBoost, SPORF, Random Forest, Logistic Regression). Hyperparameters are tuned with Optuna; scores are repeated stratified CV.

Median scores (higher is better for F1 / ROC-AUC; lower is better for inference latency):

Model	Median macro-F1	Median ROC-AUC	Median inference (ms/row)
PrismBoost	0.903	0.976	0.056
CatBoost	0.892	0.976	0.184
XGBoost	0.875	0.970	0.615
LightGBM-linear	0.875	0.970	0.567
LightGBM	0.870	0.972	0.550
Random Forest	0.862	0.964	5.140
SPORF	0.856	0.970	10.058
Logistic Regression	0.822	0.942	0.053

Average ranks (1 = best; Friedman tests significant for macro-F1 and ROC-AUC):

Model	Macro-F1 rank	ROC-AUC rank
CatBoost	3.33	3.48
PrismBoost	3.88	4.26
LightGBM-linear	3.99	4.26
LightGBM	4.10	4.10
XGBoost	4.31	4.24
Logistic Regression	5.31	5.66
Random Forest	5.48	5.31
SPORF	5.61	4.70

Nemenyi critical-difference diagrams (α = 0.05). Models connected by a bar are not significantly different.

On these data, PrismBoost is competitive with modern GBDTs on accuracy while remaining among the fastest at inference (second only to logistic regression; fastest non-linear model by median latency).

Install

pip install prismboost

Requires Python 3.10–3.13. A C++17 compiler and CMake are used when building the optional native extension (included for common platforms via wheels / sdist build).

From source (editable / development):

pip install -e ".[dev]"

If the C++ extension fails to build, the package still works via the pure-Python backend.

Extras:

pip install "prismboost[examples]"   # Optuna for the tuning example
pip install "prismboost[dev]"        # pytest, ruff

Public API

Name	Description
PrismBoostClassifier	Classifier (sklearn-compatible)
PrismBoostRegressor	Regressor
SEFR	Linear weak learner used inside oblique splits
auto_boosting_config(n_samples, n_features)	The "auto" default rules, callable for inspection

Legacy names

The project was previously called SEFRBoost. Those names are aliases of the classes above — the same objects, so isinstance checks and old pickles keep working — and are kept for backwards compatibility:

Legacy name	Now
SEFRBoostClassifier / SEFRBoostRegressor	PrismBoostClassifier / PrismBoostRegressor
SEFRGradientBoostingClassifier / SEFRGradientBoostingRegressor	PrismBoostClassifier / PrismBoostRegressor
prismboost.sefr_gbdt, prismboost.sefr_boost	prismboost.prism_boost

SEFR itself is not legacy: it is the linear model that produces each oblique split, and it keeps its name.

Features

• Oblique tree splits from SEFR (closed-form linear separator)
• Binary and multiclass classification; regression
• Newton (second-order) split gain and leaf values, as in XGBoost and CatBoost; second_order=False selects the first-order variant, where the per-sample Hessian is replaced by 1 so the gain becomes variance reduction and leaves hold the mean residual
• Optional C++ backend for faster fit/predict
• sklearn estimator API (fit, predict, predict_proba, pipelines, pickling)
• Works with Optuna / GridSearchCV / RandomizedSearchCV

Examples

pip install "prismboost[examples]"
python examples/quickstart.py
python examples/optuna_tuning.py --n-trials 20

• examples/quickstart.py — minimal fit / score
• examples/optuna_tuning.py — Optuna CV search over trees, depth, learning rate, split_mode, and scaler

Docs mirror: docs/optuna_tuning.rst.

Tests

pip install -e ".[dev]"
pytest -q

License

This project is licensed under the MIT License.

Third-party note: prismboost._utils includes code derived from wnb under the BSD 3-Clause License.

Citation

If you use PrismBoost in academic work, please cite the accompanying paper (to be updated on publication).

Authors

• Hamidreza Keshavarz
• Reza Rawassizadeh