chimeraboost 0.9.2

CatBoost-inspired gradient boosting in pure Python with a numba backend

chimeraboost

What if CatBoost, but ~5× faster with bagging built in, and all in Python?

⚠️ Project is in active development: breaking changes should be expected.

• Installation

pip install chimeraboost

• Sample code:

from chimeraboost import ChimeraBoostClassifier, ChimeraBoostRegressor

# classification
clf = ChimeraBoostClassifier(early_stopping=True)
clf.fit(X, y, cat_features=[0, 1], sample_weight=w, n_ensembles=2)
proba = clf.predict_proba(X_test)

# regression (RMSE, MAE, or Quantile)
reg = ChimeraBoostRegressor(loss="Quantile", alpha=0.9, early_stopping=True, n_ensembles=10)
reg.fit(X, y)

• Reproduce the benchmark

python benchmarks/run_benchmarks.py --grinsztajn --save

• What?

  • Exceedingly opinionated GBDT library that only depends on common Python libraries
    • Categorical features (catboost-like processing) and sample weights
    • Bagging as a first-class feature (n_ensembles)
    • Automatic early stopping, with optional grouped splitting for the validation set
  • Supports regression, quantile regression, binary and multiclass classification
  • Matches CatBoost within ~0.5% F1, ~0.5% Brier, and ~2% RMSE (% of best) on the 59-dataset Grinsztajn (2022) tabular benchmark, at ~5× the speed

• Tuning tips

  • Interaction-heavy regression (many features with strong cross-terms — e.g. the pol dataset): raise depth to 8–10. The depth=6 default is deliberately conservative to keep small datasets from overfitting; on large, interaction-heavy tasks a deeper oblivious tree is decisively better. On pol (n≈15k), depth=10 cuts RMSE ~11% below depth=6 and beats CatBoost, LightGBM and sklearn HGB by ~12%. Keep depth=6 for small (≲4k-row) data, where deeper trees overfit.

• Why?

  • I want to be able to modify my GBDT library at will
  • I know Python and I don't know C