ctboost 0.1.0

A GPU-accelerated gradient boosting library using Conditional Inference Trees.

CTBoost

CTBoost is a gradient boosting library built around Conditional Inference Trees, with a native C++17 core, Python bindings via pybind11, optional CUDA support for source builds, and a scikit-learn style API.

The current codebase supports end-to-end training and prediction for regression, binary classification, and multiclass classification, plus native pandas DataFrame ingestion with automatic categorical detection and early stopping via eval_set.

Current Status

• Language mix: Python + C++17, with optional CUDA
• Python support: 3.8 through 3.12
• Packaging: scikit-build-core
• CI/CD: GitHub Actions for CMake validation and cibuildwheel release builds
• Status: actively evolving native + Python package

What Works Today

• Native gradient boosting backend exposed as ctboost._core
• Pool abstraction for dense tabular data and categorical feature indices
• Native pandas DataFrame and Series support
• Automatic categorical detection for pandas category and object columns
• Regression training with ctboost.train(...)
• scikit-learn compatible CTBoostClassifier and CTBoostRegressor
• Binary and multiclass classification
• Early stopping with eval_set and early_stopping_rounds
• Feature importance reporting
• Build metadata reporting through ctboost.build_info()
• CPU builds on standard CI runners
• Optional CUDA compilation when building from source with a suitable toolkit

Current Limitations

• The Python package currently targets dense NumPy-compatible input
• Dedicated GPU wheel automation currently targets Linux CPython 3.10
• CUDA wheel builds in CI depend on container-side toolkit provisioning

Installation

For local development or source builds:

pip install .

Install development dependencies:

pip install -e .[dev]

CPU-Only Source Build

To force a CPU-only native build:

CMAKE_ARGS="-DCTBOOST_ENABLE_CUDA=OFF" pip install .

On PowerShell:

$env:CMAKE_ARGS="-DCTBOOST_ENABLE_CUDA=OFF"
pip install .

CUDA Source Build

CTBoost can compile a CUDA backend when the CUDA toolkit and compiler are available. CUDA is enabled by default in CMake, but the build automatically falls back to CPU-only when no toolkit is detected.

pip install .

You can inspect the compiled package after installation:

import ctboost
print(ctboost.build_info())

Quick Start

scikit-learn Style Classification

import pandas as pd
from sklearn.datasets import make_classification

from ctboost import CTBoostClassifier

X, y = make_classification(
    n_samples=256,
    n_features=8,
    n_informative=5,
    n_redundant=0,
    random_state=13,
).astype("float32")
X = pd.DataFrame(X, columns=[f"f{i}" for i in range(X.shape[1])])
X["segment"] = pd.Categorical(["a" if i % 2 == 0 else "b" for i in range(len(X))])
y = y.astype("float32")

model = CTBoostClassifier(
    iterations=256,
    learning_rate=0.1,
    max_depth=3,
    alpha=1.0,
    lambda_l2=1.0,
    task_type="CPU",
)

model.fit(
    X.iloc[:200],
    y[:200],
    eval_set=[(X.iloc[200:], y[200:])],
    early_stopping_rounds=20,
)
proba = model.predict_proba(X)
pred = model.predict(X)
importance = model.feature_importances_
best_iteration = model.best_iteration_

Low-Level Training API

import numpy as np

import ctboost

X = np.array([[0.0, 1.0], [1.0, 0.0], [0.5, 0.5]], dtype=np.float32)
y = np.array([0.0, 1.0, 0.5], dtype=np.float32)

pool = ctboost.Pool(X, y)
booster = ctboost.train(
    pool,
    {
        "objective": "RMSE",
        "learning_rate": 0.2,
        "max_depth": 2,
        "alpha": 1.0,
        "lambda_l2": 1.0,
        "max_bins": 64,
        "task_type": "CPU",
    },
    num_boost_round=10,
)

predictions = booster.predict(pool)
loss_history = booster.loss_history

Working With Categorical Features

Categorical columns can still be marked manually through the Pool API:

import numpy as np
import ctboost

X = np.array([[0.0], [1.0], [2.0], [3.0]], dtype=np.float32)
y = np.array([1.0, 0.0, 1.0, 0.0], dtype=np.float32)

pool = ctboost.Pool(X, y, cat_features=[0])

For pandas inputs, categorical/object columns are detected automatically:

import pandas as pd
import ctboost

frame = pd.DataFrame(
    {
        "value": [1.0, 2.0, 3.0, 4.0],
        "city": pd.Categorical(["berlin", "paris", "berlin", "rome"]),
        "segment": ["retail", "enterprise", "retail", "enterprise"],
    }
)
label = pd.Series([0.0, 1.0, 0.0, 1.0], dtype="float32")

pool = ctboost.Pool(frame, label)
assert pool.cat_features == [1, 2]

Public Python API

The main entry points are:

• ctboost.Pool
• ctboost.train
• ctboost.Booster
• ctboost.CTBoostClassifier
• ctboost.CTBoostRegressor
• ctboost.CBoostClassifier
• ctboost.CBoostRegressor
• ctboost.build_info

Build and Test

Run the test suite:

pytest tests

Build an sdist:

python -m build --sdist

Configure and build the native extension directly with CMake:

python -m pip install pybind11 numpy pandas scikit-learn pytest
cmake -S . -B build -DCTBOOST_ENABLE_CUDA=OFF -Dpybind11_DIR="$(python -m pybind11 --cmakedir)"
cmake --build build --config Release --parallel

Wheel builds are configured through cibuildwheel for:

• Windows
• Linux
• macOS
• CPython 3.8, 3.9, 3.10, 3.11, and 3.12

GitHub Actions workflows:

• .github/workflows/cmake.yml: configures, builds, installs, and tests CPU builds on Ubuntu, Windows, and macOS for pushes and pull requests
• .github/workflows/publish.yml: builds release wheels and the sdist, publishes tagged releases, and uploads a dedicated Linux GPU wheel artifact

The standard release workflow builds CPU-only wheels by setting:

CMAKE_ARGS='-DCTBOOST_ENABLE_CUDA=OFF'

Project Layout

ctboost/      Python API layer
include/      public C++ headers
src/core/     core boosting, objectives, trees, statistics
src/bindings/ pybind11 extension bindings
cuda/         optional CUDA backend
tests/        Python test suite

License

Apache 2.0. See LICENSE.