sharktuner 3.5.0

IREE Dispatch Tuner

SHARK Tuner

libtuner.py is the core Python script that provides the fundamental functions for the tuning loop. It imports candidate_gen.py for candidate generation. To implement the full tuning loop, libtuner.py requires a separate Python script that uses the provided TuningClient API from libtuner.py.

Prerequisites

[Optional] Using virtual environments:

cd sharktuner
python -m venv .venv
source .venv/bin/activate

Install python dependencies:

pip install -r requirements.txt
pip install -r requirements-dev.txt

IREE's Python bindings setup:

Using the local IREE's Python bindings:

• Building with CMake

  Configure

  cmake -G Ninja -B ../iree-build/ -S . \
  -DCMAKE_BUILD_TYPE=RelWithDebInfo \
  -DCMAKE_C_COMPILER=clang \
  -DCMAKE_CXX_COMPILER=clang++ \
  -DIREE_HAL_DRIVER_HIP=ON -DIREE_TARGET_BACKEND_ROCM=ON \
  -DIREE_BUILD_PYTHON_BINDINGS=ON \
  -DPython3_EXECUTABLE="$(which python3)"
  

  Build

  cmake --build ../iree-build/
  

[!IMPORTANT] Make sure to enable the ROCM and HIP in your cmake configuration. See IREE documentation for the details.

• Set environment

  source ../iree-build/.env && export PYTHONPATH
  export PATH="$(realpath ../iree-build/tools):$PATH"
  

For more information, refer to the IREE documentation.

Using nightly IREE's Python bindings:

pip install --upgrade -r ../requirements-iree-unpinned.txt

Examples

For a concrete example, check the model_tuner directory for a sample tuner implemented with libtuner. The dispatch example should be a good starting point for most users.