Benchmark Functions for JAX

| GitHub | PyPI | Documentation | Zenodo

JAX implementations of the BBOB benchmark functions (Finck et al., 2009) ¹, the CEC 2005 benchmark functions (Suganthan et al., 2005) ² and the CEC 2017 benchmark functions (Awad et al., 2016) ³ for black-box optimization.

First publication: October 17, 2025

Summary

bbob-jax is a pure-JAX reimplementation of three standard black-box optimization benchmark suites: the 24 noise-free BBOB functions (Finck et al., 2009), the 25 CEC 2005 real-parameter functions (Suganthan et al., 2005) and the 29 CEC 2017 bound-constrained functions (Awad et al., 2016; F2 was officially withdrawn). Every function is differentiable, JIT-compilable, and vectorizable via vmap, and is exposed through a simple registry that returns a ready-to-call objective together with its global minimum, as well as a problem() accessor that additionally bundles the optimum location, search-space bounds and metadata tags in one lookup. Both randomized (shifted and rotated) and deterministic factory variants are provided.

Statement of need

The BBOB and CEC benchmark suites are cornerstones of black-box optimization research. This repository provides JAX reimplementations of all three: the 24 BBOB noise-free functions originally written in C, the 25 CEC 2005 real-parameter functions, and the 29 CEC 2017 bound-constrained functions (simple, hybrid and composition). Translating these suites to JAX enables automatic differentiation, just-in-time (JIT) compilation, and XLA-accelerated performance — making them ideal for research in optimization, machine learning, and evolutionary algorithms.

3D surface plots of the 24 BBOB benchmark functions.


2D contour plots of the 24 BBOB benchmark functions.


3D surface plots of the 25 CEC 2005 benchmark functions.


2D contour plots of the 25 CEC 2005 benchmark functions.


3D surface plots of the 29 CEC 2017 benchmark functions.


2D contour plots of the 29 CEC 2017 benchmark functions. Panels marked "(2D slice of nD)" show a 2D slice through the optimum plane of functions only defined from n dimensions up.

Authorship & Citation

Authors:

• Martin van der Schelling (m.p.vanderschelling@tudelft.nl)

Authors affiliation:

• Delft University of Technology (Bessa Research Group)

Maintainer:

• Martin van der Schelling (m.p.vanderschelling@tudelft.nl)

Maintainer affiliation:

• Delft University of Technology (Bessa Research Group)

If you use bbob-jax in your research or in a scientific publication, it is appreciated that you cite the paper below:

Zenodo (link):

@software{vanderSchelling2025,
  title        = {Black-box optimization benchmarking (bbob) problem
                   set for JAX},
  author       = {van der Schelling, M. P. and Bessa, M A.},
  month        = {nov},
  year         = {2025},
  publisher    = {Zenodo},
  version      = {v1.0.0},
  doi          = {10.5281/zenodo.17426894},
  url          = {https://doi.org/10.5281/zenodo.17426894},
}

Gradient-friendly implementations

Many BBOB functions use non-smooth operations (abs, sign, sqrt, max, min) that produce zero, undefined, or infinite gradients at certain points. This library uses softjax straight-through estimators to provide well-defined gradients everywhere while keeping the forward pass exactly equal to the original function definitions.

For example, jnp.abs(x) has a zero gradient at x = 0 and jnp.sqrt(x) has an infinite gradient at x = 0. The softjax replacements (sj.abs_st, sj.sqrt) return the exact same values but route gradients through smooth approximations during the backward pass. This means jax.grad produces useful, finite gradients without any loss of benchmark fidelity.

Functions that are intentionally non-smooth (F7 step_ellipsoid, F23 katsuura) are left unchanged — smoothing them would defeat their benchmarking purpose.

Getting started

To install the package, use pip:

pip install bbob-jax

Related Work

This project builds on and complements established benchmarking efforts and tooling in black-box optimization. The resources below are closely related and provide broader context and utilities.

• COCO platform (COmparing Continuous Optimisers): benchmarking framework and tools for black-box optimization. ⁴
• EvoSax: JAX-based evolution strategies library that includes BBOB function support and benchmarking utilities. ⁵

Community Support

If you find any issues, bugs or problems with this package, please use the GitHub issue tracker to report them.

License

Copyright (c) 2025, Martin van der Schelling

All rights reserved.

This project is licensed under the BSD 3-Clause License. See LICENSE for the full license text.

Related repositories

bbob-jax provides the benchmark functions used across the L2CO ecosystem developed in the Bessa Research Group. The repositories below work together:

• l2co — Learning to Choose Optimizers: a meta-learner that selects an optimizer from problem features before any evaluations, then reassesses that choice from the observed optimization trajectory.
• rl2co — Reinforcement Learning to Choose Optimizers: a JAX-based RL agent that dynamically switches between optimizers during a run.
• l2co-tasks — Optimization task definitions (BBOB, CEC 2005, PDE, spiral, …) compatible with the L2CO library.
• l2co_experiments — Hydra + f3dasm experiment pipelines (dataset creation, training, rollouts, figures) for the L2CO studies.
• agentic-l2co — An LLM-agent drop-in replacement for l2co.L2COModel, driving two-stage optimizer selection with an Ollama-hosted LLM.
• bbob-jax — JAX implementations of the BBOB, CEC 2005 and CEC 2017 black-box optimization benchmark functions.
• f3dasm — Framework for Data-Driven Design and Analysis of Structures and Materials; provides ExperimentData, pipelines, and SLURM orchestration.