hoptimal 0.1.0

High-performance hyperparameter optimisation — C++ core, Python bindings

hoptimal

A from-scratch C++ Gaussian-Process Bayesian optimizer with Python bindings — competitive with Optuna and scikit-optimize on sample efficiency.

hoptimal implements Bayesian optimization (a Gaussian-Process surrogate + acquisition functions) in modern C++20, exposed to Python through pybind11. The GP, the Cholesky-based inference, the kernel-hyperparameter fitting, and the acquisition optimization are all hand-written — no GPyTorch, no BoTorch, no scikit-learn under the hood.

Benchmark

Median regret (best-found − known optimum, lower is better) vs trial number on 8 standard optimization test functions, 8 seeds each — hoptimal against Optuna and scikit-optimize:

hoptimal's GP methods are best-in-class: across the 8 functions they win 4 outright and place 2nd on the other 4, beating Optuna's TPE and CMA-ES and trading wins with scikit-optimize's GP — especially strong on the harder higher-dimensional problems (Hartmann-6, Ackley, Rosenbrock, Rastrigin).

Function	hoptimal (best)	Optuna (best)	scikit-optimize
branin	0.0018	0.70	0.0011
hartmann3	0.013	0.15	0.0002
hartmann6	0.14	1.06	0.23
ackley5	2.48	3.30	2.78
rosenbrock4	5.29	10.4	9.28
levy5	2.17	2.07	3.98
rastrigin5	18.3	27.7	41.1
styblinski5	48.8	55.5	27.7

Reproduce: python benchmarks/python/competitors/bench_competitors.py && python benchmarks/python/visualize.py

Quickstart

import hoptimal

study = hoptimal.create_study("minimize")   # GP + Expected Improvement by default

def objective(trial):
    x = trial.suggest_float("x", -5.0, 5.0)
    y = trial.suggest_float("y", -5.0, 5.0)
    return (x - 2.0) ** 2 + (y + 1.0) ** 2

study.optimize(objective, n_trials=50)
print(study.best_value, study.best_params)

The Trial API mirrors Optuna's, so migration is mostly mechanical. For a runnable tour (toy problem → Branin convergence plot → real sklearn model → head-to-head vs Optuna) see examples/demo.ipynb.

Features

Capability	Status
GP Bayesian optimization — EI, UCB, PI acquisitions	✅
RBF and Matérn-5/2 kernels, MLE hyperparameter fitting (L-BFGS)	✅
Float / int / log-scale / categorical parameters	✅
Multi-objective optimization (Pareto front)	✅
Median and Hyperband pruners (early stopping)	✅
Parallel trials (n_jobs) with constant-liar	✅
ML integrations: scikit-learn, XGBoost, LightGBM, CatBoost, PyTorch, JAX/Flax, TensorFlow/Keras	✅

All integrations are validated on clean Linux in CI (see the Dockerfile).

scikit-learn example

from sklearn.svm import SVC
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from hoptimal.integrations.sklearn import HoptimalSearchCV

pipe = Pipeline([("scaler", StandardScaler()), ("svc", SVC())])
search = HoptimalSearchCV(
    pipe,
    {"svc__C":     ("float", 1e-3, 1e3, {"log": True}),
     "svc__gamma": ("float", 1e-4, 1e0, {"log": True}),
     "svc__kernel":("categorical", ["rbf", "sigmoid"])},
    n_trials=40, cv=5, scoring="accuracy",
)
search.fit(X, y)
print(search.best_score_, search.best_params_)

Build from source

With Docker (recommended — builds + tests + validates integrations on clean Linux):

docker build --target core -t hoptimal .          # C++ core + tests + bindings
docker build --target integrations -t hoptimal .  # + sklearn/xgboost/lightgbm/catboost
docker build --target dl -t hoptimal .            # + pytorch/jax/tensorflow (heavy)

Locally (needs a C++20 compiler, CMake, Eigen3; GoogleTest/pybind11 are fetched automatically if absent):

# C++ library + tests
cmake -B build -DHOPTIMAL_BUILD_TESTS=ON
cmake --build build --parallel
ctest --test-dir build --output-on-failure

# Python package
pip install .

How it works

Each trial, hoptimal fits a Gaussian Process to the observations so far, then picks the next point by optimizing an acquisition function over the GP's posterior. The implementation is hand-written C++:

• GP regression with RBF / Matérn-5/2 kernels; inference via a Cholesky factorization of K + σ²I (with jitter for stability) rather than an explicit inverse.
• Kernel hyperparameters (amplitude, length-scale) fitted by maximizing the log marginal likelihood with L-BFGS and random restarts.
• Acquisition (EI / UCB / PI) optimized over the normalized [0,1]ᵈ space via a quasi-random candidate set refined with L-BFGS.

The cost center — repeated Cholesky factorizations and acquisition optimization — is exactly why the core is in C++.

License

MIT — see LICENSE.