SmallGBM

Gradient boosting that actually works on small data.

Why SmallGBM?

XGBoost and LightGBM are built for scale. They shine on thousands of rows. But when you only have 50, 100, or 500 samples, their default hyperparameters fail — overfitting, instability, unpredictable results.

SmallGBM is designed from the ground up for datasets with fewer than 1000 samples.

Noise Stability

SmallGBM degrades gracefully when labels are noisy — unlike XGBoost and LightGBM which drop sharply.

At 20% label noise, SmallGBM is the best performer. Bayesian regularization keeps it stable when others collapse.

Learning Curve

Clear, predictable improvement as data grows. Reliable performance starts at n ≈ 40.

No sudden jumps, no catastrophic failures. A safe choice when data is limited.

Installation

pip install smallgbm

Quickstart

from smallgbm import SmallGBMClassifier

model = SmallGBMClassifier()
model.fit(X_train, y_train)
proba = model.predict_proba(X_test)

API

SmallGBMClassifier

SmallGBMRegressor

Same parameters, for regression tasks.

from smallgbm import SmallGBMRegressor

model = SmallGBMRegressor()
model.fit(X_train, y_train)
preds = model.predict(X_test)

Research

Full characterisation notebook with 5 experiments: benchmark_final.ipynb

• Noise stability analysis
• Prior sensitivity
• Sample size curve
• Regression performance
• Class imbalance tolerance

License

MIT © Emelyanov Ilya 2026

_{Built with ❤️ for the small data community}