craceplot: Visualisation of Data from crace

[ Homepage ] [ Source Code ] [ User Guide ] [ Report A Bug ] [ Discussions ]

Authors: Yunshuang Xiao .

Contact: crace developers race.autoconfig@gmail.com

Relevant literature:

• Yunshuang Xiao, Leslie Pérez Cáceres, Manuel López-Ibáñez, and Thomas Stützle. Algorithm Configuration via Continuously Racing: Preliminary Results. In Proceedings of the Companion Conference on Genetic and Evolutionary Computation (GECCO '23 Companion ). Association for Computing Machinery, New York, NY, USA, 1744–1752. doi: 10.1145/3583133.3596408

Introduction

craceplot provides a set of functions to create plots to visualize the data generated by the crace software for automatic algorithm configuration (hyper-parameter optimization).

This package provides visualizations for:

• Parameter configurations using parallel coordinates with param_parallelcoord( ) or param_parallelcat( ).

• Configurations performance (training and testing) with qual_boxplot( ), qual_heatmap( ) and qual_scatter( ).

• Sampling distributions with param_boxplot( ), param_histplot( ), param_pairplot( ), param_heatmap( ), param_jointplot( ) and param_sunburst( ).

For more details about these functions, please check the user guide of the package and the documentation of the functions implemented in the package.

The aim of this package is to provide support for the analysis of the single crace run and its best parameter settings found, the assessment of the parameter space explored by crace and the overall performance of the configuration process.

Keywords: automatic configuration, offline tuning, parameter tuning, parameter visualization, crace.

Requisites

Python ($\text{version} \geq 3.8.0$) is required for running craceplot, but you don't need to know the Python language to use it. Python is freely available and you can download it from the Python project website. See Quick Start for a quick installation guide of Python.

User guide

A complete user guide comes with a IPython Notebook file. You can access it online guide or, after installing the craceplot package, invoking from the terminal console using the following command:

craceplot doc

The following is a quick-start guide. The user guide gives more detailed instructions.

Quick Start

• Install Python (with your favorite package manager, and see more details below).
• Install crace. The crace package can be installed automatically with conda, with pip or from the source code. We advise to use the automatic installation unless particular circumstances do not allow it.
• Once crace is installed, three commands, crace, crace_mpi and crace_guide, will be added to the active environment. These commands work in Bash shell (Linux and MacOS) as well as in Powershell (Windows).
• You can open the user guide with the following command. This command works on Bash shell (Linux and MacOS) and Powershell (Windows) with the active environment added to PATH (see detailed instructions below).

Installing Python

This section gives a quick Python installation guide that will work in most cases. The official instructions are available at official python site.

GNU/Linux

You should install Python from your package manager. On a Debian/Ubuntu system it will be something like

sudo apt-get install python3 python3-dev

Once Python is installed, you can use conda, pip or the source code to install the crace package. See instructions below.

OS X

For macOS 10.9 (Jaguar) up until 12.3 (Catalina) the operating system includes Python 2, which is no longer supported and is not a good choice for development. You should go to do the downloads page and download the installer.

For newer versions of macOS, Python is no longer included by default and you will have to download and install it. You can refer to the Python documentation for more details on the installation process and getting started.

You can install Python directly from a PyPI mirror (https://pypi.org/). Alternatively, you can just brew the Python formula from the science tap (unfortunately it does not come already bottled so you need to have Xcode (https://developer.apple.com/xcode/download/) installed to compile it):

# install homebrew if you don’t have it
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

# install python 3
brew tap homebrew/science
brew install python

Once Python is installed, you can use conda, pip or the source code to install the craceplot package.

Windows

You can install Python from the official website (https://www.python.org/downloads/windows). We recommend that you install Python on a filesystem path without spaces, special characters or long names, such as C:\Python.

Note that Python 3.5 - 3.8 cannot be used on Windows XP or earlier and Python 3.9 - 3.13 cannot be used on Windows 7 or earlier.

Once Python is installed, you can use conda, pip or the source code to install the craceplot package.

Installing craceplot

Requirements

• Python (>= 3.6.0) is required for running craceplot package.
• crace (>= 0.0.0) is required for running craceplot package, and you must have completed log files from one crace run in advance. See crace user guide for installing crace and obtaining log files.

The craceplot package can be installed automatically with conda, with pip or from the source. We advise to use the automatic installation unless particular circumstances do not allow it. The instructions to install craceplot with the three mentioned methods are the following:

Install automatically with conda

If you use Conda, execute the following line at the shell to install the package:

# Use the environment where you install crace
conda activate my-env
# If you want to install from conda-forge
conda config --env --add channels conda-forge
# The actual install command
conda install craceplot

Alternatively, within the Conda graphical interface (Anaconda Navigator), you may search and install craceplot.

Install automatically with pip

If you use Pip, execute the following line at the shell to install the package:

pip3 install craceplot

Also when using pip, it's good practice to use a virtual environment and here is the guide for using the virtual environment.

Manual download and installation from source

If the previous installation instructions fail because of insufficient permissions and you do not have sufficient admin rights to install craceplot system-wide, then you need to force a local installation. From the craceplot package PyPI website, search and download one of the two available versions depending on your operating system:

• craceplot-xxx.tar.gz (Unix/Linux/BSD/macOS)
• craceplot-xxx-py3-none-any.whl (python3, any platform)

Alternatively, you can use wget or curl from the command line to download the file:

# PyPI
wget https://files.pythonhosted.org/packages/source/c/craceplot/craceplot-xxx.tar.gz
# Github
wget https://github.com/race-autoconfig/craceplot/

To install the package on GNU/Linux, macOS or Windows using the formats .tar.gz, you should first extract the source files and then navigate to the directory containing the extracted files and run the following command to install the package:

python3 install .

To install the package on any platform using the Wheel format, you should run the following command to install the package:

pip3 install /path/to/craceplot-xxx-py3-none-any.whl

Testing the installation and invoking craceplot

Once craceplot has been installed, list by using command line at the shell

# List installed packages at the shell
pip3 list
# Show the information of craceplot at the shell
pip3 show craceplot

or load the package and test that the installation was successful by opening a Python console and executing:

# Load and test at the Python console
import craceplot
print(craceplot.__version__)

Checking the installation path of craceplot

Once craceplot is installed, the executable command will be added to the environment. To check the installation path of craceplot using command line at the shell:

which craceplot

The output of this line must be $PYTHON_HOME/bin/craceplot, which means the path before /bin/craceplot is $PYTHON_HOME. Here, $CRACEPLOT_HOME should be:

CRACEPLOT_HOME=$PYTHON_HOME/lib/python3.X/site-packages/craceplot

Also, you can check the installation path of craceplot opening a Python console and executing:

import importlib.util
print(importlib.util.find_spec('craceplot').submodule_search_locations)

This command must print out the filesystem path where craceplot is installed. In the remainder of this guide, the variable $CRACEPLOT_HOME is used to denote this path. When executing any provided command that includes the $CRACEPLOT_HOME variable, do not forget to replace it with the true installation path of craceplot.

On GNU/Linux or macOS, you can let the operating system know where to find craceplot by defining the $PYTHON_HOME variable and adding it to the system PATH. Append the following commands to ~/.bash_profile, ~/.bashrc or ~/.profile:

# Replace <CRACEPLOT_HOME> with the craceplot installation path
export CRACEPLOT_HOME=<CRACEPLOT_HOME>
# Tell operating system where to find craceplot
export PATH=${PYTHON_HOME}/bin/:$PATH

Then, open a new terminal and launch craceplot as follows:

craceplot --help

Alternatively, you may directly invoke craceplot within the Python console by executing:

import craceplot
craceplot.run("--help")

License

craceplot is free software (software libre): you can redistribute it and/or modify it under the terms of the MIT License.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

IMPORTANT NOTE: Please be aware that the fact that this program is released as Free Software does not excuse you from scientific propriety, which obligates you to give appropriate credit! If you write a scientific paper describing research that made substantive use of this program, it is your obligation as a scientist to (a) mention the fashion in which this software was used in the Methods section; (b) mention the algorithm in the References section.

The appropriate citation is:

• Yunshuang Xiao, Leslie Pérez Cáceres, Manuel López-Ibáñez, and Thomas Stützle. 2023. Algorithm Configuration via Continuously Racing: Preliminary Results. In Proceedings of the Companion Conference on Genetic and Evolutionary Computation (GECCO '23 Companion ). Association for Computing Machinery, New York, NY, USA, 1744–1752. doi: 10.1145/3583133.3596408

craceplot is built as an additional tool for the crace package: crace: Continuous Racing for Automatic Algorithm Configuration Copyright (c) 2026 crace developers