Skip to main content

DisCoMBO: Steering Expert-in-the-Loop Black-Box Optimization via Distributional Conformance

This repository contains the code of DisCoMBO: Steering Expert-in-the-Loop Black-Box Optimization via Distributional Conformance (NeurIPS 2026).

Abstract

Sequential Model-Based Optimization (SMBO) traditionally relies on Bayesian or ensembling surrogates for uncertainty quantification. While historically treated as fully data-driven, SMBO increasingly integrates external domain expertise to accelerate discovery. To overcome the opaque guidance and diminished integration fidelity of standard acquisition re-weighting, Probabilistic Circuits (PCs) have emerged as a generative surrogate alternative, enabling direct knowledge injection via conditional sampling. However, these generative routines lack the formal exploration-exploitation semantics required for rigorous optimization. We introduce the Distributional Conformance Score (DisCo), a novel metric that unifies the flexibility and efficiency of PCs with a formal uncertainty framework. DisCo provides a bounded, [0, 1]-normalized measure of model "surprise" that (1) recovers properties comparable to kernel-based uncertainty known from, e.g., Gaussian Processes, while maintaining linear-time inference, and (2) enables accurate assessment of conformance of external knowledge w.r.t. model evidence. We then present DisCoMBO, a framework leveraging these properties for robust, knowledge-aware optimization. We prove that DisCoMBO is a zero-regret algorithm and demonstrate its effectiveness across diverse benchmarks from AutoML, material optimization, and wind park optimization.

Branches

Branch Content
main DisCoMBO only, with minimal dependencies (Python >= 3.10, SPFlow 1.1). This is the version released on PyPI.
paper The code and environment that produced the results of the paper: DisCoMBO, all baselines and benchmarks (legacy SPFlow, Python 3.9, Docker).
new_spflow_benchmarking The benchmarking environment of the paper migrated to SPFlow 1.1.0 (Python 3.10, Docker).

Installation

DisCoMBO requires Python >= 3.10 and is built on SPFlow 1.1, PyTorch, NumPy, SciPy and scikit-learn.

From PyPI:

pip install discombo

From the repository:

pip install git+https://github.com/bewit/DisCoMBO.git

or, for development, as an editable install of a clone (including the test dependencies):

git clone https://github.com/bewit/DisCoMBO.git
cd DisCoMBO
pip install -e ".[test]"

For Optuna: DisCoMBO is available as an Optuna sampler on OptunaHub. The sampler is loaded at runtime with optunahub (see Usage with Optuna below); its dependencies, including DisCoMBO from PyPI, are installed with

pip install optuna optunahub
pip install -r https://hub.optuna.org/samplers/discombo/requirements.txt

CPU-only PyTorch: DisCoMBO runs on the CPU. On Linux, pip installs the CUDA build of PyTorch by default, which takes several GB. To get the much smaller CPU build, install PyTorch from the PyTorch CPU index before installing DisCoMBO with any of the options above:

pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install discombo

On macOS and Windows, the default PyTorch build from PyPI is already CPU-only.

Usage

DisCoMBO maximizes an objective over a search space. The search space is defined by a dictionary mapping each hyperparameter to its type and domain; the objective takes a configuration (a dictionary) and returns a BenchQueryResult, whose validation performance is maximized.

from discombo import BenchQueryResult, MCOptimizer, SimpleSearchSpace

space = SimpleSearchSpace({
    'lr': {'dtype': 'float', 'min': 1e-4, 'max': 1e-1, 'is_log': True},
    'layers': {'dtype': 'int', 'allowed': [1, 2, 3, 4]},
    'optimizer': {'dtype': 'str', 'allowed': ['sgd', 'adam']},
})

def objective(cfg):
    score = ...  # evaluate cfg
    return BenchQueryResult(score, score, score)  # train, validation and test performance

optimizer = MCOptimizer(space, objective, iterations=50, num_samples=10, initial_samples=20, seed=0)
best_cfg, best_result = optimizer.optimize()

Continuous hyperparameters use 'dtype': 'float' with 'min', 'max' and optionally 'is_log'; discrete hyperparameters use 'dtype' 'int', 'str', 'cat' or 'bool' with a list of 'allowed' values.

Runs are reproducible: all randomness of DisCoMBO is drawn from a random state seeded with seed, and the global random states of NumPy and PyTorch are left untouched. Custom search spaces (subclasses of SearchSpace) should draw from the rng passed to their sample method.

DisCoMBO reports its progress (the best observed value per iteration and the applied interventions) via Python's logging under the logger name discombo. To show it, configure logging in your application, e.g.

import logging
logging.basicConfig(level=logging.INFO)

Expert knowledge

Expert beliefs are injected as interventions in given iterations, either as fixed values (point interventions) or as distributions over values. Discrete hyperparameters are referred to by the index of the value in 'allowed'.

optimizer.intervene(
    [{'lr': {'dist': 'gauss', 'parameters': [-4.6, 0.5]}},    # distributions: 'gauss', 'uniform', 'int_uniform', 'cat'
     {'optimizer': 1}],                                         # point intervention: 'adam'
    [10, 25],                                                   # iterations in which the interventions are applied
)

Continuous hyperparameters with 'is_log': True are modeled in log space, so their intervention values are given in log space as well. After optimization, optimizer.get_intervention_pval_log() returns the distributional conformance (p-values) of each intervention w.r.t. the model.

Usage with Optuna

import optuna
import optunahub

module = optunahub.load_module("samplers/discombo")
study = optuna.create_study(direction="maximize", sampler=module.DisCoMBOSampler(seed=0))
study.optimize(objective, n_trials=100)  # objective(trial) as usual in Optuna

Structure

  • discombo/optimizers/mc_optimizer.py: the DisCoMBO optimizer.
  • discombo/circuits/: the probabilistic circuits on top of SPFlow: LearnSPN structure learning, conditioning, conditional sampling and the bottom-up p-value inference (weighted arithmetic mean at sum nodes, Fisher's method at product nodes).
  • discombo/search_spaces/: the search space interface and SimpleSearchSpace.

Tests

pip install -e ".[test]"
pytest tests/

Citation

If you use DisCoMBO, please cite our paper (CITATION.bib):

@inproceedings{seng2026discombo,
  title         = {{DisCoMBO}: Steering Expert-in-the-Loop Black-Box Optimization via Distributional Conformance},
  author        = {Seng, Jonas and Wittelsbach, Bennet and Kersting, Kristian},
  booktitle     = {Advances in Neural Information Processing Systems},
  volume        = {40},
  year          = {2026},
  eprint        = {2609.36472},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  url           = {https://arxiv.org/abs/2609.36472}
}

Metadata

Release files for discombo 1.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for discombo 1.0.0
File Size Uploaded Provenance Actions
discombo-1.0.0.tar.gz 38.7 kB Without provenance Details for discombo-1.0.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for discombo 1.0.0
File Interpreter ABI Platform Actions
discombo-1.0.0-py3-none-any.whl Python 3 none any Details for discombo-1.0.0-py3-none-any.whl

Total release size: 76.2 kB

Release files / discombo-1.0.0.tar.gz

Download URL discombo-1.0.0.tar.gz
Size 38.7 kB
Tags Source
SHA-256 checksum
How to use checksums
82f3e6ec1bb4c23011b6a182b93677c6d05427ff52367433a60d22583a7ab689
BLAKE2b-256 checksum
How to use checksums
5b513ac63e16459cec27409570456c330de3c6f2c53db2da6fb6418477b696bd
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.13

Release files / discombo-1.0.0-py3-none-any.whl

Download URL discombo-1.0.0-py3-none-any.whl
Size 37.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
bd8e046a3f8fe5b67ae247d35a6896a7b8e0b40055b90895d959d0d8ebb5e32b
BLAKE2b-256 checksum
How to use checksums
b714d8e9a917050051f869734d516ae9be163df3d68b9a5834e74d744d6754a2
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.13

Security for discombo 1.0.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page