HNBM

Heterogeneous Newton Boosting Machine (HNBM) — a scikit-learn-compatible gradient boosting framework that stochastically mixes heterogeneous base learners at each iteration.

Unlike standard gradient boosting libraries that use a single learner type (typically decision trees), HNBM lets you define a pool of base learners with selection probabilities. At each boosting round, a learner is drawn from that pool and fit to the Newton step (gradient divided by Hessian, weighted by the Hessian).

This is the core framework behind SnapBoost, inspired by SnapBoost: A Heterogeneous Boosting Machine (Parnell et al., NeurIPS 2020).

Installation

From source (recommended until PyPI release):

git clone https://github.com/qiancapital-dev/hnbm.git
cd hnbm
pip install .

Requirements: Python ≥ 3.8, NumPy, scikit-learn, tqdm.

Quick Start

Subclass HNBM and configure your base learner pool before training:

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeRegressor
from hnbm import HNBM

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)


class TreeBoost(HNBM):
    def __init__(self, max_depth=5, **kwargs):
        super().__init__(**kwargs)
        self.base_learners_ = [DecisionTreeRegressor(max_depth=max_depth)]
        self.probabilities_ = [1.0]


model = TreeBoost(
    num_iterations=100,
    learning_rate=0.1,
    mode="classification",
    random_state=42,
)
model.fit(X_train, y_train)

print("Accuracy:", model.score(X_test, y_test))
model.evaluate(X_test, y_test)

API Reference

HNBM

Methods

Subclass contract: set base_learners_ (list of unfitted sklearn regressors) and probabilities_ (list summing to 1) before calling fit.

Loss functions

hnbm.losses provides Logistic (classification) and MeanSquaredError (regression), each with a compute_derivatives(y, f) method returning gradient and Hessian vectors.

Docker

docker build -t hnbm .
docker run --rm hnbm

Development

git clone https://github.com/qiancapital-dev/hnbm.git
cd hnbm
pip install -r requirements.txt
pip install -e .

Related projects

• snapboost — a concrete HNBM using decision trees and kernel ridge regressors

License

MIT — See LICENSE for full text.