lab 5.5

Benchmark your code

Lab is a Python package for running code on a large set of benchmarks. Experiments can be conducted on a single machine or on a cluster. The package also contains code for parsing results and creating reports. Currently, Lab and Downward Lab are distributed together.

The Downward Lab Python package facilitates running experiments for the Fast Downward planning system. It uses the generic experimentation package Lab.

Code: https://github.com/aibasel/lab

Documentation: https://lab.readthedocs.io

Cite: please cite Downward Lab by using

@Misc{seipp-et-al-zenodo2017,
  author =       "Jendrik Seipp and Florian Pommerening and
                  Silvan Sievers and Malte Helmert",
  title =        "{Downward} {Lab}",
  publisher =    "Zenodo",
  year =         "2017",
  doi =          "10.5281/zenodo.790461",
  url =          "https://doi.org/10.5281/zenodo.790461",
  howpublished = "\url{https://doi.org/10.5281/zenodo.790461}"
}

Install Lab

Lab supports Python 2.7 (deprecated) or Python >= 3.5. We recommend installing Lab in a Python virtual environment. This has the advantage that there are no modifications to the system-wide configuration, and that you can create multiple environments with different Lab versions (e.g., for different papers) without conflicts:

# Install required packages, including virtualenv.
sudo apt install python3 python3-venv

# Create and activate a Python 3 virtual environment for Lab.
python3 -m venv lab-venv  # or any other path instead of lab-venv
source lab-venv/bin/activate

# Install Lab in the virtual environment.
pip install lab  # or a specific version with lab==x.y

If you want to install the latest development version and/or need to change Lab itself, you can clone the Lab repo and install it in the virtual environment:

git clone https://github.com/aibasel/lab.git /path/to/lab
pip install --editable /path/to/lab

The --editable flag installs the project in “editable mode”, which makes any changes under /path/to/lab appear immediately in the installed package.

Please note that before running an experiment script you need to activate the virtual environment with:

source lab-venv/bin/activate

We recommend clearing the PYTHONPATH variable before activating the virtual environment.