ML SDK Model Converter

The ML SDK Model Converter is a command line application that translate TOSA ML Models to VGF files. A VGF file is a model file containing SPIR-V™ modules and constants that are required to execute the model through the ML extensions for Vulkan®. The ML SDK Model Converter supports several different TOSA encodings as inputs:

• TOSA FlatBuffers
• TOSA MLIR bytecode
• TOSA MLIR textual format

The ML SDK Model Converter can also produce TOSA FlatBuffers from its input, without performing any conversion.

You can also use the ML SDK Model Converter to check that all tensors specified in the input model are ranked and have fixed, non-dynamic shapes. If a dynamic tensor is detected, the program will exit with an error.

The suggested workflow for this tool as part of the ML SDK for Vulkan® is:

1. A TOSA MLIR file is converted to a VGF file using the ML SDK Model Converter (this project).
2. The generated VGF file and VGF library VGF Dump Tool is used to create a JSON scenario template file. The template file is edited with the correct filenames and paths.
3. Using the generated VGF file and scenario file, the ML SDK Scenario Runner then dispatches the contained SPIR-V™ modules to the ML extensions for Vulkan®.

Cloning the repository

To clone the ML SDK Model Converter as a stand-alone repository, you can use regular git clone commands. However, for better management of dependencies and to ensure everything is placed in the appropriate directories, we recommend using the git-repo tool to clone the repository as part of the ML SDK for Vulkan® suite. Repo tool.

For a minimal build and to initialize only the ML SDK Model Converter and its dependencies, run:

repo init -u https://github.com/arm/ai-ml-sdk-manifest -g model-converter

Alternatively, to initialize the repo structure for the entire ML SDK for Vulkan®, including the ML SDK Model Converter, run:

repo init -u https://github.com/arm/ai-ml-sdk-manifest -g all

After the repo is initialized, you can fetch the contents with:

repo sync --no-clone-bundle

Cloning on Windows®

To ensure nested submodules do not exceed the maximum long path length, you must enable long paths on Windows®, and you must clone close to the root directory or use a symlink. Make sure to use Git for Windows.

Using PowerShell:

Set-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem" -Name "LongPathsEnabled" -Value 1
git config --global core.longpaths true
git --version # Ensure you are using Git for Windows, for example 2.50.1.windows.1
git clone <git-repo-tool-url>
python <path-to-git-repo>\git-repo\repo init -u <manifest-url> -g all
python <path-to-git-repo>\git-repo\repo sync --no-clone-bundle

Using Git Bash:

cmd.exe "/c reg.exe add \"HKLM\System\CurrentControlSet\Control\FileSystem"" /v LongPathsEnabled /t REG_DWORD /d 1 /f"
git config --global core.longpaths true
git --version # Ensure you are using the Git for Windows, for example 2.50.1.windows.1
git clone <git-repo-tool-url>
python <path-to-git-repo>/git-repo/repo init -u <manifest-url> -g all
python <path-to-git-repo>/git-repo/repo sync --no-clone-bundle

After the sync command completes successfully, you can find the ML SDK Model Converter in <repo_root>/sw/model-converter/. You can also find all the dependencies required by the ML SDK Model Converter in :<repo_root>/dependencies/.

Building the ML SDK Model Converter from source

The build system must have:

• C/C++ 17 compiler: GCC, or optionally Clang on Linux and MSVC on Windows®.
• CMake 3.25 or later.
• Ninja 1.8.2 or later.
• Python 3.10 or later. Required python libraries for building are listed in tooling-requirements.txt.

The following dependencies are also needed:

• Argument Parser for Modern C++.
• LLVM.
• TOSA Tools.

For the preferred dependency versions see the manifest file.

Building with the script

Arm® provides a python build script to make build configuration options easily discoverable. When the script is run from a git-repo manifest checkout, the script uses default paths and does not require any additional arguments. Otherwise the paths to the dependencies must be specified.

To build on Linux, run:

SDK_PATH="path/to/sdk"
python3 ${SDK_PATH}/sw/model-converter/scripts/build.py -j $(nproc) \
    --vgf-lib-path ${SDK_PATH}/sw/vgf-lib \
    --flatbuffers-path ${SDK_PATH}/dependencies/flatbuffers \
    --argparse-path ${SDK_PATH}/dependencies/argparse \
    --tosa-tools-path ${SDK_PATH}/dependencies/tosa-tools \
    --external-llvm ${SDK_PATH}/dependencies/llvm-project

To build on Windows®, run:

$env:SDK_PATH="path\to\sdk"
$cores = [System.Environment]::ProcessorCount
python3 "$env:SDK_PATH\sw\model-converter\scripts\build.py" -j $cores `
    --vgf-lib-path "$env:SDK_PATH\sw\vgf-lib" `
    --flatbuffers-path "$env:SDK_PATH\dependencies\flatbuffers" `
    --argparse-path "$env:SDK_PATH\dependencies\argparse" `
    --tosa-tools-path "$env:SDK_PATH\dependencies\tosa-tools" `
    --external-llvm "$env:SDK_PATH\dependencies\llvm-project"

If the components are in their default locations, it is not necessary to specify the --vgf-lib-path, --flatbuffers-path, --argparse-path, --tosa-tools-path, and --external-llvm options.

Tests can be enabled and run with --test and linting by --lint. To enable tests and documentation building python dependencies must be installed:

pip install -r requirements.txt
pip install -r tooling-requirements.txt

The documentation can be built with --doc. To build the documentation, sphinx and doxygen must be installed on the machine.

You can install the project build artifacts into a specified location by passing the option --install with the required path.

To create an archive containing the build artifacts, pass the --package-type option with an archive type such as zip or tgz. Use --package-dir to choose where the archive is written; by default, packages are written to the build directory.

For more information, see the help output:

python3 scripts/build.py --help

Usage

To generate a VGF file, run:

./build/model-converter --input ${INPUT_TOSA} --output ${OUTPUT_VGF}

To generate a TOSA flatbuffer file, run:

./build/model-converter --tosa-flatbuffer --input ${INPUT_TOSA} --output ${OUTPUT_TOSA_FB}

To lower TOSA custom operations through the Arm.ExperimentalMLOperations CALL extended instruction, provide a domain to Opcode mapping:

./build/model-converter --input ${INPUT_TOSA} --output ${OUTPUT_VGF} \
    --custom-op-domain-to-opcode com.example.accel:42

The com.arm.bespoke domain is reserved for Opcode 0 and must be enabled with --enable-bespoke:

./build/model-converter --input ${INPUT_TOSA} --output ${OUTPUT_VGF} --enable-bespoke

For more information, see the help output:

./build/model-converter --help

PyPI

The ML SDK Model Converter is available on PyPI as the ai-ml-sdk-model-converter package.

Install the published package:

pip install ai-ml-sdk-model-converter

To build and install the host executable from an ML SDK checkout, run from this repository root:

pip install .

Known Limitations

• Usage of the patches/llvm.patch file is temporary until the required changes can be upstreamed to main LLVM Project

License

The ML SDK Model Converter is distributed under the software licenses in LICENSES directory.

Trademark notice

Arm® is a registered trademark of Arm Limited (or its subsidiaries) in the US and/or elsewhere.

Khronos® and Vulkan® are registered trademarks, and SPIR-V™ is a trademark of The Khronos Group Inc..