Benchmark your code
Project description
Lab is a Python package for evaluating solvers on benchmark sets. Experiments can run on a single machine or on a computer cluster. The package also contains code for parsing results and creating reports.
The Downward Lab Python package facilitates running experiments for the Fast Downward planning system. It uses the generic experimentation package Lab. Currently, Lab and Downward Lab are distributed together.
Code: https://github.com/aibasel/lab
Documentation: https://lab.readthedocs.io
Cite: please cite Downward Lab by using
@Misc{seipp-et-al-zenodo2017, author = "Jendrik Seipp and Florian Pommerening and Silvan Sievers and Malte Helmert", title = "{Downward} {Lab}", publisher = "Zenodo", year = "2017", howpublished = "\url{https://doi.org/10.5281/zenodo.790461}" }
Install Lab
Lab requires Python 3.7+ and Linux (e.g., Ubuntu). We recommend installing Lab in a Python virtual environment. This has the advantage that there are no modifications to the system-wide configuration, and that you can create multiple environments with different Lab versions (e.g., for different papers) without conflicts:
# Install required packages, including virtualenv. sudo apt install python3 python3-venv # Create a new directory for your experiments. mkdir experiments-for-my-paper cd experiments-for-my-paper # If PYTHONPATH is set, unset it to obtain a clean environment. unset PYTHONPATH # Create and activate a Python virtual environment for Lab. python3 -m venv --prompt my-paper .venv source .venv/bin/activate # Install Lab in the virtual environment. pip install -U pip wheel pip install lab # or preferably a specific version with lab==x.y # Store installed packages and exact versions for reproducibility. # Ignore pkg-resources package (https://github.com/pypa/pip/issues/4022). pip freeze | grep -v "pkg-resources" > requirements.txt
Please note that before running an experiment script you need to activate the virtual environment with:
source .venv/bin/activate
We recommend clearing the PYTHONPATH variable before activating the virtual environment.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.