ParetoBench

ParetoBench is a Python library that provides a collection of tools for the benchmarking of multi-objective optimization algorithms. It includes the following.

• Multi-objective benchmark problems including analytical Pareto fronts when available
• Container objects for storing and manipulating data from optimization algorithms
• A standardized file format for saving the results of optimizations on benchmark problems
• Tools for calculating convergence metrics on results and running statistical analyses on them to compare algorithms
• Plotting utilities for objectives/decision variables and for both populations and series of populations (history objects)

Installation

ParetoBench is available from pip and conda.

pip install paretobench

or

conda install paretobench

Containers and File Format

Objects and a file format for storing data from multi-objective optimization algorithms are included in the package.

• Population - The atomic class of the library. Represents a single generation in a genetic algorithm complete with variables (x), objectives (f), and constraints (g).
• History - A collection of populations representing the history of one run of a genetic algorithm.
• Experiment - A benchmarking experiment with multiple histories representing multiple evaluations of a genetic algorithm, potentially on multiple problems as is used in benchmarking.

The Experiment objects may be saved to a standardized HDF5-backed format for long-term storage and interchange between codes.

Learn more about the containers in the following example notebook.

container_objects.ipynb

Plotting

Tools for plotting the data from multi-objective optimization algorithms are also included.

• Pairwise decision variables plots for Population and History objects
  • Variable boundaries
  • Color coding or animation for showing multiple populations
  • Markers and alpha to distinguish non-dominated / infeasible solutions
• Objective scatter plots for Population and History objects
  • Analytical Pareto fronts for library problems
  • Attainment surfaces in 2D and 3D
  • Color coding or animation for showing multiple populations
  • Markers and alpha to distinguish non-dominated / infeasible solutions

See more information in the following notebooks.

• plotting_populations.ipynb
• plotting_histories.ipynb

Benchmark Problems

Analytical Pareto Fronts

When possible, the benchmark problems include analytical Pareto fronts.

Parameter Naming Conventions

To help standardize the code in this package, the following naming convention is used throughout for parameters.

Some names are reserved for specific purposes. These are the following.

• n: The dimension of the input vector to the problem, ie the number of decision variables.
• m: The number of objectives.

All parameters should follow the PEP 8 naming scheme for variables. Whenever this leads to a parameter being named something different than what it was called in the problem's defining paper, this change must be documented in the class.

For Developers

Installation

1. Install the development conda environment.

conda create env -f environment.yml

2. Install the package in editing mode (with dependencies required for running tests)

pip install -e .[test]

Testing

Tests are written with the pytest framework. They can be run by calling pytest from the base of this repo with the package installed.