Skip to main content

Covariance Matrix Adaptation Evolution Strategy (CMA-ES) implemented with TensorFlow v2

Project description

Covariance Matrix Adaptation Evolution Strategy (CMA-ES)

A Tensorflow v2 implementation.

What is CMA-ES?

Quoting The CMA Evolution Strategy homepage:

The CMA-ES (Covariance Matrix Adaptation Evolution Strategy) is an evolutionary algorithm for difficult non-linear non-convex black-box optimisation problems in continuous domain. It is considered as state-of-the-art in evolutionary computation and has been adopted as one of the standard tools for continuous optimisation in many (probably hundreds of) research labs and industrial environments around the world.

Installation

The package is available on PyPI and can be installed with pip:

pip install cma-es

Example Usage

1. Define the fitness function

The CMA class expects a fitness function with the following signature:

Args:
  x: tf.Tensor of shape (M, N)

Returns:
  Fitness evaluations: tf.Tensor of shape (M,)

Where M is the number of solutions to evaluate and N is the dimension of a single solution.

def fitness_fn(x):
    """
    Six-Hump Camel Function
    https://www.sfu.ca/~ssurjano/camel6.html
    """
    return (
        (4 - 2.1 * x[:,0]**2 + x[:,0]**4 / 3) * x[:,0]**2 +
        x[:,0] * x[:,1] +
        (-4 + 4 * x[:,1]**2) * x[:,1]**2
    )

Figure1: Six-Hump Camel Function

2. Configure CMA-ES

from cma import CMA

cma = CMA(
    initial_solution=[1.5, -0.4],
    initial_step_size=1.0,
    fitness_function=fitness_fn,
)

The initial solution and initial step size (i.e. initial standard deviation of the search distribution) are problem specific.

The population size is automatically set by default, but it can be overidden by specifying the parameter population_size.

For bounded constraint optimization problems, the parameter enforce_bounds can be set, e.g. enforce_bounds=[[-2, 2], [-1, 1]] for a 2D function.

3. Run the optimizer

The search method runs until the maximum number of generation is reached or until one of the early termination criteria is met. By default, the maximum number of generations is 500.

best_solution, best_fitness = cma.search()

The notebook Example 1 - Six Hump Camel Function goes into more details, including ways to plot the optimization path such as in the figure below.

Figure 2: Optimization path

Logging

A user-defined callback function can be specified to inspect variables during the search.

It is mainly intended for logging purpose, e.g:

max_epochs = 500

def logging_function(cma, logger):
    if cma.generation % 10 == 0:
        fitness = cma.best_fitness()
        logger.info(f'Generation {cma.generation} - fitness {fitness}')

    if cma.termination_criterion_met or cma.generation == max_epochs:
        sol = cma.best_solution()
        fitness = cma.best_fitness()
        logger.info(f'Final solution at gen {cma.generation}: {sol} (fitness: {fitness})')

cma = CMA(
    initial_solution=[1.5, -0.4],
    initial_step_size=1.0,
    fitness_function=fitness_fn,
    callback_function=logging_function,
)
cma.search(max_epochs)

Check out an example logging progress to TensorBoard: tensorboard_example.py

Run on a GPU

By virtue of being written using TensorFlow, it is trivial to run CMA on a GPU:

with tf.device('/GPU:0'):
    cma.search()

More examples

Resources

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for cma-es, version 1.3.0
Filename, size File type Python version Upload date Hashes
Filename, size cma-es-1.3.0.tar.gz (8.6 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page