DisCoMBO: Steering Expert-in-the-Loop Black-Box Optimization via Distributional Conformance
This repository contains the code of DisCoMBO: Steering Expert-in-the-Loop Black-Box Optimization via Distributional Conformance (NeurIPS 2026).
Abstract
Sequential Model-Based Optimization (SMBO) traditionally relies on Bayesian or ensembling surrogates for uncertainty quantification. While historically treated as fully data-driven, SMBO increasingly integrates external domain expertise to accelerate discovery. To overcome the opaque guidance and diminished integration fidelity of standard acquisition re-weighting, Probabilistic Circuits (PCs) have emerged as a generative surrogate alternative, enabling direct knowledge injection via conditional sampling. However, these generative routines lack the formal exploration-exploitation semantics required for rigorous optimization. We introduce the Distributional Conformance Score (DisCo), a novel metric that unifies the flexibility and efficiency of PCs with a formal uncertainty framework. DisCo provides a bounded, [0, 1]-normalized measure of model "surprise" that (1) recovers properties comparable to kernel-based uncertainty known from, e.g., Gaussian Processes, while maintaining linear-time inference, and (2) enables accurate assessment of conformance of external knowledge w.r.t. model evidence. We then present DisCoMBO, a framework leveraging these properties for robust, knowledge-aware optimization. We prove that DisCoMBO is a zero-regret algorithm and demonstrate its effectiveness across diverse benchmarks from AutoML, material optimization, and wind park optimization.
Branches
| Branch | Content |
|---|---|
main |
DisCoMBO only, with minimal dependencies (Python >= 3.10, SPFlow 1.1). This is the version released on PyPI. |
paper |
The code and environment that produced the results of the paper: DisCoMBO, all baselines and benchmarks (legacy SPFlow, Python 3.9, Docker). |
new_spflow_benchmarking |
The benchmarking environment of the paper migrated to SPFlow 1.1.0 (Python 3.10, Docker). |
Installation
DisCoMBO requires Python >= 3.10 and is built on SPFlow 1.1, PyTorch, NumPy, SciPy and scikit-learn.
From PyPI:
pip install discombo
From the repository:
pip install git+https://github.com/bewit/DisCoMBO.git
or, for development, as an editable install of a clone (including the test dependencies):
git clone https://github.com/bewit/DisCoMBO.git
cd DisCoMBO
pip install -e ".[test]"
For Optuna: DisCoMBO is available as an Optuna sampler on OptunaHub. The sampler is loaded at runtime with optunahub (see Usage with Optuna below); its dependencies, including DisCoMBO from PyPI, are installed with
pip install optuna optunahub
pip install -r https://hub.optuna.org/samplers/discombo/requirements.txt
CPU-only PyTorch: DisCoMBO runs on the CPU. On Linux, pip installs the CUDA build of PyTorch by default, which takes several GB.
To get the much smaller CPU build, install PyTorch from the PyTorch CPU index before installing DisCoMBO with any of the options above:
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install discombo
On macOS and Windows, the default PyTorch build from PyPI is already CPU-only.
Usage
DisCoMBO maximizes an objective over a search space. The search space is defined by a dictionary mapping each hyperparameter to its type and domain;
the objective takes a configuration (a dictionary) and returns a BenchQueryResult, whose validation performance is maximized.
from discombo import BenchQueryResult, MCOptimizer, SimpleSearchSpace
space = SimpleSearchSpace({
'lr': {'dtype': 'float', 'min': 1e-4, 'max': 1e-1, 'is_log': True},
'layers': {'dtype': 'int', 'allowed': [1, 2, 3, 4]},
'optimizer': {'dtype': 'str', 'allowed': ['sgd', 'adam']},
})
def objective(cfg):
score = ... # evaluate cfg
return BenchQueryResult(score, score, score) # train, validation and test performance
optimizer = MCOptimizer(space, objective, iterations=50, num_samples=10, initial_samples=20, seed=0)
best_cfg, best_result = optimizer.optimize()
Continuous hyperparameters use 'dtype': 'float' with 'min', 'max' and optionally 'is_log'; discrete hyperparameters use 'dtype' 'int', 'str', 'cat' or 'bool' with a list of 'allowed' values.
Runs are reproducible: all randomness of DisCoMBO is drawn from a random state seeded with seed, and the global random states of NumPy and PyTorch are left untouched.
Custom search spaces (subclasses of SearchSpace) should draw from the rng passed to their sample method.
DisCoMBO reports its progress (the best observed value per iteration and the applied interventions) via Python's logging under the logger name discombo.
To show it, configure logging in your application, e.g.
import logging
logging.basicConfig(level=logging.INFO)
Expert knowledge
Expert beliefs are injected as interventions in given iterations, either as fixed values (point interventions) or as distributions over values.
Discrete hyperparameters are referred to by the index of the value in 'allowed'.
optimizer.intervene(
[{'lr': {'dist': 'gauss', 'parameters': [-4.6, 0.5]}}, # distributions: 'gauss', 'uniform', 'int_uniform', 'cat'
{'optimizer': 1}], # point intervention: 'adam'
[10, 25], # iterations in which the interventions are applied
)
Continuous hyperparameters with 'is_log': True are modeled in log space, so their intervention values are given in log space as well.
After optimization, optimizer.get_intervention_pval_log() returns the distributional conformance (p-values) of each intervention w.r.t. the model.
Usage with Optuna
import optuna
import optunahub
module = optunahub.load_module("samplers/discombo")
study = optuna.create_study(direction="maximize", sampler=module.DisCoMBOSampler(seed=0))
study.optimize(objective, n_trials=100) # objective(trial) as usual in Optuna
Structure
discombo/optimizers/mc_optimizer.py: the DisCoMBO optimizer.discombo/circuits/: the probabilistic circuits on top of SPFlow: LearnSPN structure learning, conditioning, conditional sampling and the bottom-up p-value inference (weighted arithmetic mean at sum nodes, Fisher's method at product nodes).discombo/search_spaces/: the search space interface andSimpleSearchSpace.
Tests
pip install -e ".[test]"
pytest tests/
Citation
If you use DisCoMBO, please cite our paper (CITATION.bib):
@inproceedings{seng2026discombo,
title = {{DisCoMBO}: Steering Expert-in-the-Loop Black-Box Optimization via Distributional Conformance},
author = {Seng, Jonas and Wittelsbach, Bennet and Kersting, Kristian},
booktitle = {Advances in Neural Information Processing Systems},
volume = {40},
year = {2026},
eprint = {2609.36472},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2609.36472}
}
Metadata
Release files for discombo 1.0.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| discombo-1.0.0.tar.gz | 38.7 kB | Without provenance | Details for discombo-1.0.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| discombo-1.0.0-py3-none-any.whl | Python 3 | none | any | Details for discombo-1.0.0-py3-none-any.whl |
Total release size: 76.2 kB
Release files / discombo-1.0.0.tar.gz
| Download URL | discombo-1.0.0.tar.gz |
|---|---|
| Size | 38.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
82f3e6ec1bb4c23011b6a182b93677c6d05427ff52367433a60d22583a7ab689
|
|
BLAKE2b-256 checksum How to use checksums |
5b513ac63e16459cec27409570456c330de3c6f2c53db2da6fb6418477b696bd
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.13
|
Release files / discombo-1.0.0-py3-none-any.whl
| Download URL | discombo-1.0.0-py3-none-any.whl |
|---|---|
| Size | 37.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
bd8e046a3f8fe5b67ae247d35a6896a7b8e0b40055b90895d959d0d8ebb5e32b
|
|
BLAKE2b-256 checksum How to use checksums |
b714d8e9a917050051f869734d516ae9be163df3d68b9a5834e74d744d6754a2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.13
|
Security for discombo 1.0.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.