gaia-ml 0.6.0

GAIA-ML ONNX-to-AIE project generator

GAIA-ML

Global AI-Engine Architecture - ML (GAIA-ML) is a software tool hoping to automatically transform ML algorithms into AMD AI-Engine (AIE) optimal implementations. GAIA-ML is developed within the ATLAS Global Trigger group, as the ML layers implemented are designed to provide optimal computation for low latency and high-throughput environments

Main developer: I. Xiotidis.

GAIA-ML is developed by the NGT WP2.1 Group.

License: MIT.

Framework infrastructure

GAIA-ML follows a layers approach where each layer of the ML architecture is coded in a single AIE kernel. Having each layer coded in a single layer doesn't mean that the final result will be deployed in different AIE tiles (parametric graph implementation).

Project Status

This section is generated from the repository contents. Refresh it with:

cmake -S . -B build
cmake --build build --target readme-status

or directly:

python3 tools/update_readme_status.py

Layers

Layer	Submodule	Branch	SHA	CMake	Stream I/O	Kernel headers	Local graphs
activation	yes	master	bc7efd9	yes	no	activation.hpp	-
conv	yes	cnn_v2	794c567	yes	yes	activation_policy.hpp, conv2d.hpp, conv2d_depthwise.hpp	convGraph.cpp, depthwiseGraph.cpp
dense	yes	gaia_int	c2ae42f	yes	yes	activation_policy.hpp, dense.hpp, dense_aux.hpp	denseAuxGraph.cpp, denseGraph.cpp
tensor_op	yes	master	39e64de	yes	yes	flatten_chw.hpp	-

CMake Targets

aie
aie-conv2d
aie-dense
aie-dense-aux
aie-depthwise
aiesim
aiesim-conv2d
aiesim-dense
aiesim-dense-aux
aiesim-depthwise
data
data-conv2d
data-dense
data-depthwise
gaia_activation_kernel
gaia_conv_kernel
gaia_dense_kernel
gaia_kernels
gaia_tensor_op_kernel
layer_aie
layer_aiesim
layer_data
layer_x86
layer_x86sim
readme-status
x86
x86-conv2d
x86-dense
x86-dense-aux
x86-depthwise
x86sim
x86sim-conv2d
x86sim-dense
x86sim-dense-aux
x86sim-depthwise

Repository Layout

.
|-- examples
|   |-- conv_flatten_dense_onnx_generate.py
|   `-- dense_onnx_generate.py
|-- layers
|   |-- activation
|   |   |-- Kernel
|   |   |   `-- include
|   |   |-- .gitignore
|   |   |-- CMakeLists.txt
|   |   `-- README.md
|   |-- conv
|   |   |-- Graph
|   |   |   |-- convGraph.cpp
|   |   |   |-- convGraph.hpp
|   |   |   |-- depthwiseGraph.cpp
|   |   |   `-- depthwiseGraph.hpp
|   |   |-- Kernel
|   |   |   |-- include
|   |   |   `-- src
|   |   |-- testVectors
|   |   |   `-- makeDummyData.py
|   |   |-- .gitignore
|   |   |-- CMakeLists.txt
|   |   `-- README.md
|   |-- dense
|   |   |-- Graph
|   |   |   |-- denseAuxGraph.cpp
|   |   |   |-- denseAuxGraph.hpp
|   |   |   |-- denseGraph.cpp
|   |   |   `-- denseGraph.hpp
|   |   |-- Kernel
|   |   |   |-- include
|   |   |   `-- src
|   |   |-- testVectors
|   |   |   `-- makeDummyData.py
|   |   |-- .gitignore
|   |   |-- CMakeLists.txt
|   |   |-- notes.md
|   |   `-- README.md
|   `-- tensor_op
|       |-- Kernel
|       |   |-- include
|       |   `-- src
|       |-- CMakeLists.txt
|       `-- README.md
|-- python
|   |-- gaia_ml
|   |   |-- __init__.py
|   |   |-- _version.py
|   |   |-- cli.py
|   |   |-- limits.py
|   |   |-- model_ir.py
|   |   |-- onnx_frontend.py
|   |   `-- project.py
|   `-- gaia_ml.egg-info
|       |-- dependency_links.txt
|       |-- entry_points.txt
|       |-- PKG-INFO
|       |-- requires.txt
|       |-- SOURCES.txt
|       `-- top_level.txt
|-- tools
|   `-- update_readme_status.py
|-- .gitignore
|-- .gitmodules
|-- CMakeLists.txt
|-- pyproject.toml
|-- README.md
`-- setup.cfg

CMake hierarchy

Each layer repository can be configured and built on its own. In that mode, the layer-local graph is used for x86, aie, x86sim, and aiesim targets.

From the top-level package, layer repositories are added as kernel providers. The exported targets are:

gaia_activation_kernel
gaia_conv_kernel
gaia_dense_kernel
gaia_kernels

Enable GAIA_BUILD_LAYER_GRAPHS=ON only when you also want the top-level build to expose the layer-local smoke-test graph targets.

Notes

GAIA-ML v1.0 supports stream I/O only. Layer kernels use input_stream<int32> and output_stream<int32> ports so the top-level graph can be built around a single low-latency dataflow contract.

The current convolution stream kernels keep the existing frame-based compute path internally by reading one streamed frame into local AIE memory before computing. This preserves the stream interface for v1.0 while leaving room for a later line-buffer implementation that can reduce first-output latency.

Activations are header-only compile-time policies in v1.0. Layers default to gaia::activation::identity when the shared activation header is not on the include path, and the top-level package exposes activation.hpp so graphs can fuse policies such as linear, relu, sigmoid, tanh, or softmax into the producer kernel instead of adding a separate activation tile.

Python Automation

The first automation scaffold is available as a pip-installable package:

pip install -e .
gaia_ml version
gaia_ml inspect model.onnx
gaia_ml generate model.onnx --output build/generated_model --strategy throughput

As of gaia_ml v0.5.0, generated project structure is emitted from Jinja2 templates packaged under gaia_ml/templates/aie. Backend Python code now builds the graph/layer context and renders the project files from templates, so changes to generated Graph/, CMakeLists.txt, and generated README layout no longer need to be hard-coded directly into project.py.

As of gaia_ml v0.6.0, generated kernels keep the same stream interface but avoid full input-frame caching where possible. Depthwise Conv uses a K-row line buffer per channel; generic Conv and 1x1 pointwise Conv stream input rows and accumulate output partials; Dense and Dense-CHW stream inputs directly into output accumulators. This preserves the existing CHW padded stream contract while reducing local input storage and preparing the backend for more aggressive overlap.

Project generation supports two lowering strategies:

throughput  Use more AIE tiles and preserve layer-level graph pipelining.
compact     Fuse supported patterns to minimize tile count.

throughput is the default because AIE latency often benefits from multiple tiles running concurrently. compact is useful when fitting into the smallest number of tiles matters more than first-output latency.

The current generator parses ONNX Conv, depthwise Conv, 1x1 pointwise Conv, Flatten, and Dense/Gemm-style blocks, runs ONNX shape inference, checks what can be emitted, and creates a self-contained AIE project with Graph/, Kernel/, weights/, testVectors/, and CMakeLists.txt. Generation emits one packed-output tile per supported compute block and chains them in model order, unless a low-latency fused pattern is available:

input stream -> layer0 -> layer1 -> ... -> output stream

gaia_ml inspect always reports optimization suggestions, including patterns that are not generated yet. The CLI uses colored log levels for info, warning, critical warning, and error messages so compatibility problems and latency recommendations are easier to scan.

The current int32 backend pads dense vectors and convolution rows to 8-lane chunks. Dense blocks are rejected if either side exceeds 128 padded lanes. Convolution blocks are currently single-tile only: input spatial dimensions are limited to 32x32, kernels to 8x8, padded input/output rows to 128 lanes, and padded frame buffers to the current local-memory guardrails. Larger layers need an explicit tiling strategy before generation.

Flatten is represented in the Python IR as a tensor operation. When possible, it is fused away: Conv -> Flatten -> Dense lowers to a packed_dense_chw consumer that reads the padded CHW stream directly and skips row padding without an extra AIE tile. The tensor_op layer also provides an explicit flatten_chw kernel for future graph cases where a standalone reshuffle tile is actually required. ONNX depthwise Conv lowers to packed_depthwise_conv2d, and ONNX 1x1 Conv lowers to a specialized packed_pointwise_conv2d kernel. Weights are emitted as headers under weights/, and the generated testVectors/reference_output.csv contains the expected end-to-end inference stream for the generated test input. The first low-latency fused lowering is DepthwiseConv -> 1x1 Conv -> Flatten -> Dense(1), which emits packed_dw_pw_dense_scalar. The generic kernels are still emitted in generated projects so each kernel can continue to be developed and tested independently. Generated graphs write simulator output to testVectors/model_output.csv; after hardware AIE simulation this appears under aiesimulator_output/testVectors/model_output.csv, while the expected output remains in testVectors/reference_output.csv. The generated project can be built with:

cmake -S build/generated_model -B build/generated_model/build
cmake --build build/generated_model/build --target x86
cmake --build build/generated_model/build --target x86sim

For now, project emission supports sequential Conv, Flatten, and Dense/Gemm models with optional supported activations and zero/no bias. Branching, general transpose/concat/split reshuffles, softmax fusion, and larger-than-single-tile blocks still need explicit graph lowering rules before generation.

The examples/conv_flatten_dense_onnx_generate.py script builds a compact performance-check model:

Depthwise 3x3 over 6 channels -> ReLU -> 1x1 pointwise Conv ->
Flatten -> Dense -> ReLU -> Dense logits

It can also append ONNX Softmax with --include-softmax for semantic inspection. GAIA-ML recognizes that Softmax but rejects project generation until the backend has a whole-vector normalization kernel.