Skip to main content

aie4ml

License PyPI PyPI - Downloads arXiv

aie4ml is an end-to-end compiler that generates optimized AIE firmware automatically, which can be then built and simulated directly using AMD Vitis. It targets the AMD AI Engine (AIE) from model-level frontends and lowers supported operators into AIE graphs and kernels as a standalone AIE project.

  • Current hardware targets: AIE1, AIE-ML and AIE-MLv2 devices.
  • Current frontend paths: ONNX for explicit operator graphs, and the more user-friendly hls4ml frontend path.
  • Automatic firmware generation optimized for resource, throughput, or latency targets under a given AIE tile budget.

Supported Operators

The full constraints are in docs/support.md.

Operator AIE1 AIE-ML AIE-MLv2 Precision (inputs × weights) Notes
Dense (Gemm, MatMul with constant weights) ✅ ✅ ✅ int8 × int8, int16 × int8, float32 on all; int16 × int16 and bfloat16 on AIE-ML and AIE-MLv2; FP8 (E4M3) on AIE-MLv2 Optional bias, fused ReLU.
MatMul (two activations) ✅ ✅ ✅ As Dense The right operand is 2-D; it may be broadcast over the left operand's batch axes.
Conv2D (including grouped, depthwise and pointwise) ✅ ✅ ✅ int8 × int8 on all; int16 × int8 on AIE-ML and AIE-MLv2 (int8 or int16 output) Batch 1, kernels up to 7×7, padding smaller than the kernel, any stride (int16 inputs: horizontal stride 1), no dilation. Fuses bias, ReLU and a following MaxPool. Depthwise runs a channelwise kernel on AIE-ML and AIE-MLv2.
MaxPool ✅ ✅ ✅ As its Conv2D 2×2, stride 2, directly after a Conv2D (ReLU on either side).
BatchNormalization ✅ ✅ ✅ — Folded into the preceding Conv2D or Dense (QConv2DBatchnorm, Dense + BatchNormalization, or a folded ONNX export).
ReLU ✅ ✅ ✅ — Fused into the Dense or Conv2D.
Add ✅ ✅ ✅ Both inputs and the output of one type Same shapes; no broadcasting.
LayerNorm ✅ ✅ ✅ int8 Last axis.
Softmax ✅ ✅ ✅ int8 in, uint8 or int16 out Last axis. Accurate exponential (beta), or a faster surrogate for models trained with it.
Attention (Q·Kᵀ → Softmax → ·V) ✖️ ✅ ✅ int8 Q, K, V and output; uint8 probabilities One op per head, up to 128 tokens, head dimension a multiple of 8. Fuses an int8 or int16 bias added to the logits (a mask, for instance) and, where it ranks better, the Q, K, V projections. Multi-head exports are split head by head. Within about a code of the QDQ graph.
ReduceMean, ReduceSum ✖️ ✅ ✅ int8 Pooling over sets: over the axes between the first and the last, fused into the Dense before it.
Flatten / Reshape ✅ ✅ ✅ — One sample to [1, K], from a Conv2D into a Dense; or rows regrouped, the last axis kept, into a fused pool. No data is copied.
Transpose ✖️ ✅ ✅ — Of the last two axes.
Slice, Split, Concat ✅ ✅ ✅ — Along the boundaries of the producing layer's tiles; no data is copied.

8-bit inputs against 16-bit weights (int8 × int16) are not supported.

Prerequisites

  • AMD Vitis 2026.1.1 and a valid AIE tools license. (aie4ml tracks the newest AIE compiler; older releases may fail to compile some kernels.)
  • Python 3.10+.

Installation

pip install "aie4ml[onnx]"     # ONNX frontend
pip install "aie4ml[hls4ml]"   # hls4ml frontend

Keras 3 and QKeras v3 models need hls4ml 1.4. Until it is released, install hls4ml from upstream commit:

pip install "hls4ml @ git+https://github.com/fastmachinelearning/hls4ml.git@9e5f86caa503c2f49ac38841e174e7c06dba7533"

Documentation & Tutorials

Documentation and usage: https://github.com/dimdano/aie4ml

Tutorial 1: tutorials/tutorial_1.ipynb Tutorial 2: tutorials/tutorial_2.ipynb

General hls4ml concepts: https://fastmachinelearning.org/hls4ml

Frontends

Frontend Models
ONNX (preferred) Quantized operator graphs with QuantizeLinear/DequantizeLinear boundaries (QDQ).
hls4ml (user-friendly) Keras 3 and QKeras v3: Dense, Conv2D, DepthwiseConv2D, QConv2DBatchnorm, MaxPooling2D, Flatten, ReLU activations and LayerNormalization. Split a SeparableConv2D into its depthwise and pointwise layers.

Maintainer

aie4ml is developed and maintained by Dimitrios Danopoulos.

Citation

If aie4ml contributes to your research, please cite the corresponding publications:

@INPROCEEDINGS{11552717,
  author={Danopoulos, Dimitrios and Lupi, Enrico and Sun, Chang and Dittmeier, Sebastian and Kagan, Michael and Loncar, Vladimir and Pierini, Maurizio},
  booktitle={2026 IEEE 34th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM)},
  title={AIE4ML: An End-to-End Framework for Compiling Neural Networks for the Next Generation of AMD AI Engines},
  year={2026},
  volume={},
  number={},
  pages={176-184},
  keywords={Tiles;Modeling;Arrays;Kernel;Memory;Information rates;Throughput;System-on-chip;Loading;Engines;ai engines;hls4ml;aie4ml;versal;acceleration;inference},
  doi={10.1109/FCCM68464.2026.00035}}
@misc{danopoulos2026tamingexponentialfastsoftmax,
      title={Taming the Exponential: A Fast Softmax Surrogate for Integer-Native Edge Inference},
      author={Dimitrios Danopoulos and Enrico Lupi and Michael Kagan and Maurizio Pierini},
      year={2026},
      eprint={2604.02292},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2604.02292},
}

Metadata

Release files for aie4ml 0.3.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for aie4ml 0.3.0
File Size Uploaded Provenance Actions
aie4ml-0.3.0.tar.gz 882.5 kB Without provenance Details for aie4ml-0.3.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for aie4ml 0.3.0
File Interpreter ABI Platform Actions
aie4ml-0.3.0-py3-none-any.whl Python 3 none any Details for aie4ml-0.3.0-py3-none-any.whl

Total release size: 1.4 MB

Release files / aie4ml-0.3.0.tar.gz

Download URL aie4ml-0.3.0.tar.gz
Size 882.5 kB
Tags Source
SHA-256 checksum
How to use checksums
1a3322874dcffc46ee0e8673ecc7a61f6f31ff591f42a7051b409a7ec02c602a
BLAKE2b-256 checksum
How to use checksums
5dd9e792d98c32632d71b56f9584fbd808f84f0f779113265c78b85715085752
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.11.16

Release files / aie4ml-0.3.0-py3-none-any.whl

Download URL aie4ml-0.3.0-py3-none-any.whl
Size 518.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
5409495719a3324a35a11ff7225e78a21c0432320c4e6ba263af18640879a7ee
BLAKE2b-256 checksum
How to use checksums
e90c451939707b755ce0b697f0545d76b9c337f8a9369a85cb6dbfbcda50e351
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.11.16

Security for aie4ml 0.3.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page