chimeraboost
pronounced kai-MEER-uh-boost
Lightning-fast Gradient Boosting, near-Catboost quality, all in Python
📖 Documentation: bbstats.github.io/chimeraboost
#2 On TabArena GBDT Elo (Defaults)
Installation
pip install chimeraboost
Quickstart
from chimeraboost import ChimeraBoostClassifier, ChimeraBoostRegressor
# quality picks the speed/accuracy trade-off: 1 is fastest - 5 is strongest,
# default is 3.
clf = ChimeraBoostClassifier(quality=4)
clf.fit(X, y, cat_features=[0, 1], sample_weight=w)
proba = clf.predict_proba(X_test)
reg = ChimeraBoostRegressor()
reg.fit(X, y)
What?
- Regression, quantile regression, binary and multiclass classification
- Fast training and inference, all in python
- Extremely high quality predictions
How?
- Bagging as a first-class feature (using
n_ensembles) - Automatic early stopping
- Automatic linear-leaf auditioning
- Replay mechanism for faster refitting after early stopping
- Gradient-matrix free multi-quantile split search
- numba is very fast
Why?
- I want to be able to modify my GBDT library at will
- I know Python and I don't know C
Documentation
Benchmarking
Want to help? Run either one, then open an issue or PR with the JSON.
pip install -e ".[bench,competitors]"
python benchmarks/run_benchmarks.py --synth --seeds 3 --save # quick, synthetic
python benchmarks/run_benchmarks.py --decide --seeds 3 --save # slower, 103 real datasets
Each writes benchmarks/results/<timestamp>.json:
{
"provenance": {"chimeraboost": "0.30.0", "platform": "Linux-6.1", "cpu_count": 12,
"libraries": {"catboost": "1.2.10", "lightgbm": "4.6.0"}},
"records": [
{"dataset": "diabetes", "model": "ChimeraBoost", "seed": 0,
"metrics": {"primary": -59.82, "rmse": 59.82}, "fit_time": 0.23}
]
}
Inspirations / Citations
Most ideas in ChimeraBoost were someone else's.
- CatBoost, Prokhorenkova et al., NeurIPS 2018. Oblivious trees (the tree type itself from Kohavi & Li, IJCAI 1995), ordered target statistics, feature combinations, ordered boosting, and the size-dependent automatic learning rate (
adaptive_learning_rate). - XGBoost, Chen & Guestrin, KDD 2016. Regularized objective, second-order split gain, Newton leaf estimation,
min_child_weight. - LightGBM, Ke et al., NeurIPS 2017. Histogram-based split finding, which that paper itself treats as prior art (McRank, Li et al. 2007; pGBRT, Tyree et al. 2011).
- Minimum Variance Sampling, Ibragimov & Gusev, NeurIPS 2019. Gradient-weighted row sampling (
subsample). - Quantized GBDT training, Shi, Ke et al., NeurIPS 2022. Integer gradient histograms (
quantize_gradients). - SketchBoost, Iosipoi & Vakhrushev, NeurIPS 2022, and GBDT-MO, Zhang & Jung. Vector leaves and the projected gradient used for multiclass and multi-quantile splits.
- Linear-leaf trees, Shi, Li & Li, IJCAI 2019 (arXiv:1802.05640). Piece-wise-linear regression trees (
linear_leaves); model trees back to Quinlan's M5, 1992. - TreeSHAP, Lundberg et al., Nature Machine Intelligence 2020 (orig. SHAP, NeurIPS 2017). Exact additive feature attributions (
shap_values). - OpenFE, Zhang et al., ICML 2023 (arXiv:2211.12507). Automated pairwise feature generation (
cross_features).
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
chimeraboost-0.32.0.tar.gz
(279.9 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
chimeraboost-0.32.0-py3-none-any.whl
(163.8 kB
view details)
File details
Details for the file chimeraboost-0.32.0.tar.gz.
File metadata
- Download URL: chimeraboost-0.32.0.tar.gz
- Upload date:
- Size: 279.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/6.2.0 CPython/3.13.13
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
d4d666b19fea5e42c480b5c7fbeef129a1665c85db72a914a02138527bb5b1ab
|
|
| MD5 |
526e939bbd9d187fd50a6eec951cc638
|
|
| BLAKE2b-256 |
6bb58b47930a9449be2d98e4d5001a67dc06c5fe3c4945037f1cf45b233fca3d
|
File details
Details for the file chimeraboost-0.32.0-py3-none-any.whl.
File metadata
- Download URL: chimeraboost-0.32.0-py3-none-any.whl
- Upload date:
- Size: 163.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/6.2.0 CPython/3.13.13
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
6198e78fdff11e13f6d1514b46e4505c6e98b743c2f0c101b2ec5ab9ad220f82
|
|
| MD5 |
ae49062de40907d5912f177b9244bbcf
|
|
| BLAKE2b-256 |
0c10c5f459bf67ad0ae4f1886e9ef633c908e6e5afe3309ea329521551043b5b
|
