bonsai-gbt 1.5.4

Histogram gradient-boosted trees with a C++23 core

A histogram gradient-boosted tree library and CLI in modern C++23.

Documentation  ·  Install  ·  Guide  ·  Decisions  ·  Releases

---

What is bonsai?

bonsai is a from-scratch, histogram-based gradient boosted trees (GBT) library and command-line tool written in C++23. It pairs a small, concept-checked component API (objectives, growers, split finders, samplers) with compile-time dispatch in the training hot path, and ships the benchmark harness that pits it against XGBoost, LightGBM, and CatBoost on real data. The aim is a readable, thoroughly documented GBT: a reference-grade implementation that competes with the production libraries instead of merely tolerating comparison with them.

• Compile-time dispatch, concept-checked components. The runtime TOML config resolves once to a monomorphized Booster<Objective, Grower, Splitter, Sampler>; no virtual calls in the hot path, and contract violations fail at compile time. Adding a component is a short recipe.
• Five growers, one engine. depthwise (XGBoost-style), leafwise (LightGBM-style), oblivious (CatBoost-style), and their CUDA twins cuda_depthwise / cuda_oblivious; with 7 objectives and 3 samplers the dispatch space is 105 statically-typed combinations, selectable per run from config.
• Deterministic parallelism. Models are bit-identical across runs, thread counts, and even CPU architectures (arm64 == x86-64), a property no reference library offers, enforced per-commit in CI (the contract).
• A guide, not just docs. The guide explains gradient boosting chapter by chapter: concept, math, then the ~50 real lines that implement it here, then an experiment against the reference libraries.

Install

pip install bonsai-gbt

Wheels cover Linux x86_64/aarch64 and macOS arm64, Python 3.9 to 3.13, no toolchain needed. The linux x86_64 wheel is CUDA-enabled at 2.3MB total: GPU training works out of the box on any NVIDIA driver R525+, it behaves exactly like a CPU wheel on machines without a GPU, and every release's CUDA wheel passes a live GPU validation before it ships (decision 70). The full story, docker image included, is Install; everything past a wheel (the CLI binary, development setups, CUDA source builds) is Building from source.

Quick start

import bonsai

model = bonsai.BonsaiRegressor(
    n_iters=200, learning_rate=0.05, grower="leafwise",
    early_stopping_rounds=20,
    params={"tree.lambda_l1": 0.5},   # any dotted config key the CLI accepts
)
model.fit(X_train, y_train, eval_set=(X_valid, y_valid))
pred = model.predict(X_test)
model.save("model.msgpack")           # loadable by `bonsai predict` and vice versa

The CLI (a source-build artifact) drives the same engine with the same keys and the same models:

bonsai fit      -c CONFIG --model OUT.msgpack
bonsai predict  -c CONFIG --model IN.msgpack [--data CSV] --out PREDS.csv
bonsai eval     -c CONFIG --model IN.msgpack [--data CSV]
bonsai info                        # list (objective, grower, sampler) combos
bonsai params                      # dump the default config as TOML

Any key overrides inline (bonsai fit -c config.toml --set tree.max_depth=8 --set dispatch.grower_name=oblivious ...), and make fit-benchmark trains and times bonsai against xgboost/lightgbm/catboost on California Housing in one command. The rest of the API is one read: the API tour.

Results

Two divisions, per the benchmark charter: perf (latency and memory, accuracy as a sanity guard) and quality (accuracy, timing never citable). The evidence is the results ledger, one generated page per study.

Perf

bonsai's CUDA growers hold the fastest slot at every measured row scale and, in the 2026-07-30 studies, at every width and aspect ratio from 100 to 65,536 features. Peak host memory at 16M rows is 7.0GB against XGBoost's 22.2GB and CatBoost's 19.4GB, and measured device memory sizes to the problem (3.4GB at 16M x 128) where CatBoost reserves 90.2GB of the card at every shape. Same-pod re-baseline, test r² in parentheses:

rows	bonsai cuda dw	bonsai cuda obl	xgb cuda	catboost gpu	lgbm cpu	bonsai cpu obl
250k	0.5s (.871)	1.0s (.875)	0.8s (.872)	1.6s (.875)	2.5s (.872)	5.2s (.875)
1M	1.1s (.876)	1.4s (.876)	1.7s (.876)	2.3s (.876)	5.0s (.877)	7.3s (.876)
4M	4.5s (.878)	4.4s (.875)	5.3s (.878)	5.0s (.877)	19.9s (.879)	20.2s (.875)
16M	20.5s (.879)	18.4s (.876)	19.9s (.880)	18.5s (.876)	111.3s (.879)	73.3s (.876)

Identical-model GPUs across the rental fleet measure up to ~25% apart, so only same-pod columns compare; the width, shape, and accuracy-time frontier tables live in the ledger.

Quality

On the Grinsztajn et al. tabular benchmark (55 OpenML tasks selected by third parties, three seeds, matched knobs, best variant per library), bonsai takes the best mean rank with 36 outright wins:

library	mean rank	outright wins
bonsai	1.44	36
lightgbm	2.51	5
xgboost	2.84	6
catboost	3.22	8

The one knob that translates ambiguously between libraries is bracketed in both directions on the standings page; reproduce with pip install bonsai-gbt[bench], then python -m bonsai.bench.grinsztajn out.jsonl --report.

Every headline claim links a reproducible run and the decision that records it: claims and proofs.

Documentation

The home is daniel-m-campos.github.io/bonsai, four doors:

• Learn: gradient boosting from intuition to the shipping code, one concept per chapter, each with an experiment against the reference libraries.
• Use: install, the API in one read, parameters, building from source.
• Results: the measurement discipline; its rules in the benchmark charter, its evidence in the results ledger.
• Design: the system map, the concept-to-type tour, determinism as a contract, and the archive (decisions log, architecture notes).

Repo-only records, unpublished by design: the project retrospective, the original proposal, and the context/roadmap.

Project layout

include/bonsai/   public headers (Booster, Tree, Grower, Sampler, …)
src/              implementation + CLI (src/cli/)
python/           the bonsai package (bindings, encoding, bonsai.bench)
tests/unit/       Catch2 unit + parity tests (ctest)
benchmarks/       evidence docs + committed results data
scripts/          uv-managed Python: compare.py, probes, render_results.py
configs/          example TOML configs
docs/             the documentation site source + design records

License

MIT © 2026 Daniel M Campos. See LICENSE.