BBOBax

Black-Box Optimization Benchmarking in JAX.

BBOBax is a JAX implementation of COCO's BBOB benchmark: the 24 standard noiseless functions, the noisy suite f101–f130, and Many-Affine BBOB. Every function is jit-compilable and vmap-friendly, so populations, instances, and whole experiments run as single compiled programs on CPU, GPU, or TPU — and every value is verified numerically against the official implementation.

Why BBOBax?

• Faithful 🎯 — verified against the official 2009 BBOB implementation on identical instances, to ≤ 1e-9 relative in float64, re-proven on every test run. Where the papers and the official code disagree, the code wins, with a comment at the site.
• Fast ⚡ — one vmap evaluates a population, another batches instances, and jit compiles the whole optimization loop. Nothing is stateful, so nothing blocks vectorization.
• Simple 🌱 — one contract: a problem is one function at one dimension, sample(key) draws an instance, evaluate(key, x, params) scores a solution. A suite is a plain dict.
• Composable 🧩 — noise models are held by problems, any function pairs with any descriptor for Quality-Diversity, and Many-Affine blends all 24 into a continuous space of problems.

Suites

A suite is a dict[str, BBOBProblem] — composing your own is a dict comprehension.

Getting Started

import jax
from bbobax import Rastrigin

problem = Rastrigin(num_dims=10)  # one function at one dimension

key_instance, key_x, key_eval = jax.random.split(jax.random.key(0), 3)
params = problem.sample(key_instance)  # draw an instance: optimum, rotations, noise

# Evaluate a whole population in one compiled call
population = jax.vmap(problem.sample_x)(jax.random.split(key_x, 1024))
keys = jax.random.split(key_eval, 1024)
fitness = jax.vmap(problem.evaluate, in_axes=(0, 0, None))(
    keys, population, params
).fitness  # (1024,)

Quality-Diversity is composition — any function pairs with any of six descriptor families (RandomProjection, IrregularProjection, QuantizedProjection, FourierProjection, SubsetProjection, AlignedProjection), each isolating one phenomenon of real behavior maps. Descriptor space is [-1, 1]^k by construction, so problem.descriptor_range is exact ground truth — whether your algorithm is told is your experiment's choice:

from bbobax import QDProblem, RandomProjection, Rastrigin

problem = QDProblem(Rastrigin(num_dims=10), RandomProjection(descriptor_size=2))
evaluation = problem.evaluate(key, x, params)  # .fitness and .descriptor

For Sphere under a linear descriptor, the best fitness achievable at any descriptor value has a closed form — sphere_descriptor_optimum — giving QD archives an exact per-cell reference no other benchmark provides.

To cover many functions or dimensions, hold many problems and loop — the loop unrolls under jit, so every problem keeps its own compiled code and none pays for dispatch:

import bbobax

for num_dims in bbobax.DIMENSIONS:  # (2, 3, 5, 10, 20, 40), COCO's own set
    for name, problem in bbobax.bbob_suite(num_dims=num_dims).items():
        ...

Tutorials

Notebook	Colab
Getting Started — the contract on one function
The Suites — bbob, bbob-noisy, and Many-Affine
Black-Box Optimization — one problem, one evolution strategy
Benchmarking — six algorithms across the suite, the COCO way
Meta-Black-Box Optimization — meta-learning a component over the suite
Quality-Diversity — five algorithms on one skeleton

Fidelity

The official 2009 implementation (bbobbenchmarks.py) is vendored as test ground truth, and tests/test_alignment.py re-proves the numerical agreement on every run. Guarantees you can rely on:

• params.x_opt is always the function's true argmin: f(x_opt) = f_opt for all 24. The six functions that constrain where their optimum may sit draw it accordingly.
• Noise applies to the raw function value only; the boundary penalty and f_opt are added outside it, and the three official noise models stabilize themselves — below the 1e-8 target precision the undisturbed value comes back, so noise can never hide a solved problem.
• The bbob-noisy suite is not "the 24 plus noise": boundary handling is replaced with a uniform factor of 100, each problem is pinned to one of the paper's two severities, and two of the bases are reparameterized — exactly as in the reference.

Deliberate, documented deviations: instances are sampled from a key rather than enumerated from COCO's seed table, f_opt defaults to 0, and rotations are Haar on SO(n). Each deviation is documented where it lives, with the official behavior quoted.

Precision

BBOBax follows JAX's float32 default, which is right for comparing algorithms and for meta-learning. Measuring proximity to the optimum at BBOB's 1e-8 target precision is a float64 question — float32 carries about 7 decimal digits, so on a landscape of order 1 the target sits below what the format can represent:

import jax

jax.config.update("jax_enable_x64", True)  # before any array is created

The test suite runs in float64; a separate sweep proves all 24 functions stay finite in float32 across all six standard dimensions.

Installation

BBOBax requires Python 3.11 or later, and a working JAX installation.

pip install bbobax

or with uv:

uv add bbobax

Citation

If you use BBOBax in your research, please cite:

@software{faldor2025bbobax,
  author = {Faldor, Maxence},
  title = {{BBOBax}: Black-Box Optimization Benchmarking in {JAX}},
  url = {https://github.com/maxencefaldor/bbobax},
  year = {2025},
}

References

• Hansen, N., Finck, S., Ros, R., & Auger, A. (2009). Real-parameter black-box optimization benchmarking 2009: Noiseless functions definitions. PDF
• Hansen, N., Finck, S., Ros, R., & Auger, A. (2009). Real-parameter black-box optimization benchmarking 2009: Noisy functions definitions. PDF
• Vermetten, D., Ye, F., Bäck, T., & Doerr, C. (2023). MA-BBOB: A problem generator for black-box optimization using affine combinations and shifts. arXiv

Contributing

Contributions are welcome. Fidelity is the contract: any new function, model, or suite comes with a numerical alignment test against its official reference.