bbob-jax 1.6.5

BBOB Benchmark function implemented in JAX

Benchmark Functions for JAX

| GitHub | PyPI | Documentation | Zenodo

JAX implementations of the BBOB benchmark functions (Finck et al., 2009) ¹ and the CEC 2005 benchmark functions (Suganthan et al., 2005) ² for black-box optimization.

First publication: October 17, 2025

---

Statement of need

The BBOB and CEC 2005 benchmark suites are cornerstones of black-box optimization research. This repository provides JAX reimplementations of both: the 24 BBOB noise-free functions originally written in C, and the 25 CEC 2005 real-parameter functions. Translating these suites to JAX enables automatic differentiation, just-in-time (JIT) compilation, and XLA-accelerated performance — making them ideal for research in optimization, machine learning, and evolutionary algorithms.

3D surface plots of the 24 BBOB benchmark functions.


2D contour plots of the 24 BBOB benchmark functions.


3D surface plots of the 25 CEC 2005 benchmark functions.


2D contour plots of the 25 CEC 2005 benchmark functions.

Authorship & Citation

Authors:

• Martin van der Schelling ([e-mail])

Authors affiliation:

• Delft University of Technology (Bessa Research Group)

Maintainer:

• Martin van der Schelling ([e-mail])

Maintainer affiliation:

• Delft University of Technology (Bessa Research Group)

If you use bbob-jax in your research or in a scientific publication, it is appreciated that you cite the paper below:

Zenodo (link):

@software{vanderSchelling2025,
  title        = {Black-box optimization benchmarking (bbob) problem
                   set for JAX},
  author       = {van der Schelling, M. P. and Bessa, M A.},
  month        = {nov},
  year         = {2025},
  publisher    = {Zenodo},
  version      = {v1.0.0},
  doi          = {10.5281/zenodo.17426894},
  url          = {https://doi.org/10.5281/zenodo.17426894},
}

Gradient-friendly implementations

Many BBOB functions use non-smooth operations (abs, sign, sqrt, max, min) that produce zero, undefined, or infinite gradients at certain points. This library uses softjax straight-through estimators to provide well-defined gradients everywhere while keeping the forward pass exactly equal to the original function definitions.

For example, jnp.abs(x) has a zero gradient at x = 0 and jnp.sqrt(x) has an infinite gradient at x = 0. The softjax replacements (sj.abs_st, sj.sqrt) return the exact same values but route gradients through smooth approximations during the backward pass. This means jax.grad produces useful, finite gradients without any loss of benchmark fidelity.

Functions that are intentionally non-smooth (F7 step_ellipsoid, F23 katsuura) are left unchanged — smoothing them would defeat their benchmarking purpose.

Getting started

To install the package, use pip:

pip install bbob-jax

Related Work

This project builds on and complements established benchmarking efforts and tooling in black-box optimization. The resources below are closely related and provide broader context and utilities.

• COCO platform (COmparing Continuous Optimisers): benchmarking framework and tools for black-box optimization. ³
• EvoSax: JAX-based evolution strategies library that includes BBOB function support and benchmarking utilities. ⁴

Community Support

If you find any issues, bugs or problems with this package, please use the GitHub issue tracker to report them.

License

Copyright (c) 2025, Martin van der Schelling

All rights reserved.

This project is licensed under the BSD 3-Clause License. See LICENSE for the full license text.

1. Finck, S., Hansen, N., Ros, R., and Auger, A. (2009), Real-parameter black-box optimization benchmarking 2009: Noiseless functions definitions, INRIA. ↩

2. Suganthan, P. N., Hansen, N., Liang, J. J., and Deb, K. (2005), Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization. ↩

3. Hansen, N., Auger, A., Ros, R., Mersmann, O., Tušar, T., and Brockhoff, D. (2021), COCO: A Platform for Comparing Continuous Optimizers in a Black-Box Setting. Optimization Methods and Software, 36(1), 114–144. https://doi.org/10.1080/10556788.2020.1808977 ↩

4. Lange, R. T. (2022), evosax: JAX-based Evolution Strategies. arXiv preprint arXiv:2212.04180. ↩