SmallGBM

Gradient boosting for small tabular data.
_{Outperforms XGBoost · Beats LightGBM · Lowest variance}

What is SmallGBM?

SmallGBM is a gradient boosting library designed for small datasets (n < 1000). It combines robust leaf weight estimation with stochastic split selection to outperform XGBoost and LightGBM — with lower variance and no hyperparameter tuning.

Benchmark

27 datasets (15 synthetic + 12 real-world) · 5-fold cross-validation · mean ROC-AUC

SmallGBM outperforms XGBoost by +0.85%, RandomForest by +1.0%, LightGBM by +1.9%, and has the lowest variance among all models.

Why Robust Leaf Weights?

Standard gradient boosting uses the mean of residuals per leaf. On small data, one outlier can destroy the estimate.

SmallGBM uses:

• Median for leaves with n ≤ 30
• Inverse-distance weighted mean for larger leaves
• Signal-adaptive shrinkage toward the parent node

This makes predictions robust to outliers and label noise — the main enemies of small-sample learning.

Why Stochastic Split Selection?

Full enumeration of all possible split thresholds overfits on small data. SmallGBM uses 5 random thresholds per feature instead — less overfitting, faster training, and better generalization.

Installation

pip install smallgbm

Quickstart

from smallgbm import SmallGBMClassifier

model = SmallGBMClassifier()
model.fit(X_train, y_train)
proba = model.predict_proba(X_test)

Parameters

Features

• Robust leaf weights — median + adaptive shrinkage
• Stochastic split selection — 5 random thresholds, less overfitting
• Column subsampling — fights overfitting in high-dimensional small data
• Uncertainty estimates — predict_with_uncertainty()
• scikit-learn compatible — fit, predict, predict_proba
• Pure Python + NumPy — no compilation, easy install

Citation

@software{emelyanov2026smallgbm,
  author = {Emelyanov, Ilya},
  title = {SmallGBM: Gradient Boosting with Robust Leaf Regularization for Small-Sample Tabular Data},
  year = {2026},
  doi = {10.5281/zenodo.21934674},
  url = {https://github.com/nsdmlk/SmallGBM}
}

License

MIT © Emelyanov Ilya, 2026

_{Built for researchers and engineers working with limited data.}