SmallGBM

Gradient boosting for small tabular data.
_{Statistical parity with XGBoost · Outperforms LightGBM · Lowest variance}

What is SmallGBM?

SmallGBM is a gradient boosting library designed for small datasets (n < 1000). It combines robust leaf weight estimation with controlled column subsampling to deliver accuracy on par with XGBoost — with lower variance and no hyperparameter tuning.

Benchmark

27 datasets (15 synthetic + 12 real-world) · 5-fold cross-validation · mean ROC-AUC

SmallGBM achieves statistical parity with XGBoost (p > 0.05), outperforms RandomForest by +0.17%, LightGBM by +1.1%, and has the lowest variance among all models.

Why Robust Leaf Weights?

Standard gradient boosting uses the mean of residuals per leaf. On small data, one outlier can destroy the estimate.

SmallGBM uses:

• Median for leaves with n ≤ 30
• Inverse-distance weighted mean for larger leaves
• Signal-adaptive shrinkage toward the parent node

This makes predictions robust to outliers and label noise — the main enemies of small-sample learning.

Installation

pip install smallgbm

Quickstart

from smallgbm import SmallGBMClassifier

model = SmallGBMClassifier()
model.fit(X_train, y_train)
proba = model.predict_proba(X_test)

Parameters

Features

• Robust leaf weights — median + adaptive shrinkage
• Column subsampling — fights overfitting in high-dimensional small data
• Uncertainty estimates — predict_with_uncertainty()
• scikit-learn compatible — fit, predict, predict_proba
• Pure Python + NumPy — no compilation, easy install

Citation

@software{emelyanov2026smallgbm,
  author = {Emelyanov, Ilya},
  title = {SmallGBM: Gradient Boosting with Robust Leaf Regularization for Small-Sample Tabular Data},
  year = {2026},
  doi = {10.5281/zenodo.21934674},
  url = {https://github.com/nsdmlk/SmallGBM}
}

License

MIT © Emelyanov Ilya, 2026

_{Built for researchers and engineers working with limited data.}