Python Benchmark Functions for Optimization

Quick Start • Benchmark Functions

py-benchamrk-functions is a simple library that provides benchmark functions for global optimization. It exposes implementations in major computing frameworks such as NumPy, TensorFlow and PyTorch. All implementations support batch-evaluation of coordinates, allowing for performatic evaluation of candidate solutions in the search space. The main goal of this library is to provide up-to-date implementations of multiple common benchmark functions in the scientific literature.

Quick Start

Start by installing the library using your preferred package manager:

python -m pip install --upgrade pip uv
python -m uv pip install py_benchmark_functions

TensorFlow and Torch backend are optional and can be enable with:

python -m uv pip install py_benchmark_functions[tensorflow]
python -m uv pip install py_benchmark_functions[torch]

You can check if the library was correctly installed by running the following:

import py_benchmark_functions as bf

print(bf.available_backends())
# Output: {'numpy', 'tensorflow', 'torch'}

print(bf.available_functions())
# Output: ['Ackley', ..., 'Zakharov']

Instantiating and using Functions

The library is designed with the following entities:

• core.Function: class that represents a benchmark function. An instance of this class represents an instance of the becnmark function for a given domain (core.Domain) and number of dimensions/coordinates.
• core.Transformation: class that represents a transformed (i.e., shifted, scaled, etc) function. It allows for programatically building new functions from existing ones.
• core.Metadata: class that represent metadata about a given function (i.e., known global optima, default search space, default parameters, etc). A transformation inherits such metadata from the base function.

The benchmark functions can be instantiated in 3 ways:

1. Directly importing from py_benchmark_functions.imp.{numpy,tensorflow,torch} (e.g., from py_benchmark_functions.imp.numpy import AckleyNumpy);

from py_benchmark_functions.imp.numpy import AckleyNumpy

fn = AckleyNumpy(dims=2)
print(fn.name, fn.domain)
# Output: Ackley Domain(min=[-35.0, -35.0], max=[35.0, 35.0])

print(fn.metadata)
# Output: Metadata(default_search_space=(-35.0, 35.0), references=['https://arxiv.org/abs/1308.4008', 'https://www.sfu.ca/~ssurjano/optimization.html'], comments='', default_parameters={'a': 20.0, 'b': 0.2, 'c': 6.283185307179586}, global_optimum=0.0, global_optimum_coordinates=<...>)

2. Using the global get_fn, get_np_function or get_tf_function from py_benchmark_functions;

import py_benchmark_functions as bf

fn = bf.get_fn("Zakharov", 2)
print(fn, type(fn))
# Output: Zakharov(domain=Domain(min=[-5.0, -5.0], max=[10.0, 10.0])) <class 'py_benchmark_functions.imp.numpy.ZakharovNumpy'>

fn1 = bf.get_np_function("Zakharov", 2)
print(fn1, type(fn1))
# Output: Zakharov(domain=Domain(min=[-5.0, -5.0], max=[10.0, 10.0])) <class 'py_benchmark_functions.imp.numpy.ZakharovNumpy'>

fn2 = bf.get_tf_function("Zakharov", 2)
print(fn2, type(fn2))
# Output: Zakharov(domain=Domain(min=[-5.0, -5.0], max=[10.0, 10.0])) <class 'py_benchmark_functions.imp.tensorflow.ZakharovTensorflow'>

fn3 = bf.get_torch_function("Zakharov", 2)
print(fn3, type(fn3))
# Output: Zakharov(domain=Domain(min=[-5.0, -5.0], max=[10.0, 10.0])) <class 'py_benchmark_functions.imp.torch.ZakharovTorch'>

3. Using the Builder class;

from py_benchmark_functions import Builder

fn = Builder().function("Alpine2").dims(4).transform(vshift=1.0).tensorflow().build()
print(fn, type(fn))
# Output: Transformed(Alpine2) <class 'py_benchmark_functions.imp.tensorflow.TensorflowTransformation'>

Regardless of how you get an instance of a function, all of them define the __call__ method, which allows them to be called directly. Every __call__ receives an x as argument (for NumPy, x should be an np.ndarray, for Tensorflow a tf.Tensor, and for PyTorch a torch.Tensor). The shape of x can either be (batch_size, dims) or (dims,), while the output is (batch_size,) or () (a scalar). Those properties are illustrated below:

import py_benchmark_functions as bf
import numpy as np

fn = bf.get_fn("Ackley", 2)
x = np.array([0.0, 0.0], dtype=np.float32)

print(fn(x))
# Output: -9.536743e-07

x = np.expand_dims(x, axis=0)
print(x, fn(x))
# Output: [[0. 0.]] [-9.536743e-07]

x = np.repeat(x, 3, axis=0)
print(x, fn(x))
# Output:
# [[0. 0.]
# [0. 0.]
# [0. 0.]] [-9.536743e-07 -9.536743e-07 -9.536743e-07]

[!NOTE]
Additionally, for the torch and tensorflow backends, it is possible to use their autograd to differentiate any of the functions. Specifically, they expose the methods .grads(x) -> Tensor and .grads_at(x) -> Tuple[Tensor, Tensor] which returns the gradients for the input x and, for grads_at, the value of the function at x (in this order).

[!WARNING] Beware that some functions are not continuously differentiable, which might return NaN's values! For the specifics of how those backends handle such cases one should refer to the respective official documentation (see A Gentle Introduction to torch.autograd and Introduction to gradients and automatic differentiation).

Benchmark Functions

The following table lists the functions officially supported by the library. If you have any suggestion for new functions to add, we encourage you to open an issue or pull request.

References:

[1]: Jamil, M., & Yang, X.-S. (2013). A Literature Survey of Benchmark Functions For Global Optimization Problems. arXiv. https://doi.org/10.48550/ARXIV.1308.4008

[2]: https://www.sfu.ca/~ssurjano/optimization.html

[3]: Special Session & Competition on Single Objective Bound Constrained Optimization (CEC2021)

[4]: https://al-roomi.org/benchmarks/unconstrained/n-dimensions/231-deb-s-function-no-01

[5]: https://infinity77.net/global_optimization/test_functions_nd_M.html

All the images can be generated using the Drawer utility.