Akida Custom Operators for Onnx Runtime

Custom ONNX Runtime operators for Akida neural network hardware acceleration.

Overview

This project provides custom ONNX operators that enable execution of neural network models on Brainchip Akida hardware through ONNX Runtime. It supports both INT8 and INT32 output types with 2 custom operators AkidaOpInt8 and AkidaOpInt32.

Requirements

Hardware

• Akida-compatible device (FPGA v2)

Software

• ONNX Runtime 1.23.0 (automatically downloaded during build)
• Akida Python package

Installation

1. Build from Source

# Clone the repository
git clone <repository-url>
cd AkidaORT

# Build the custom operator library
./build_akida_ort.sh python3.11

# The compiled library will be available at:
#     build/akida_ort_lib.so
# The wheel will be available at:
#     dist/akida_ort-*.whl

2. Install Package

# Install from wheel
pip install dist/akida_ort-*.whl

Usage

Basic Example with AkidaOpInt8

import numpy as np
import onnx
import onnxruntime as ort
import akida
import akida_ort

# 1. Create and compile your Akida model
model = akida.Model()
model.add(akida.InputConv2D((64, 64, 3), kernel_size=3, filters=16))
model.add(akida.Conv2D(kernel_size=3, filters=32, activation=akida.ActivationType.ReLU))
model.add(akida.Dense1D(units=10, output_bits=8))  # output_bits == 8 then use AkidaOpInt8

# 2. Map to hardware (first device found) and save program
hw_model = akida.Model(model.layers)
hw_model.map(akida.devices()[0], hw_only=True)
program = hw_model.sequences[0].program
# Save the binary program to send it to the operator
with open("program.bin", "wb") as f:
    f.write(program)

# 3. Create ONNX model with AkidaOpInt8 operator
onnx_model = onnx.parser.parse_model("""
    <ir_version: 8, opset_import: ["com.brainchip": 1]>
    model (uint8[1, 64, 64, 3] X) => (int8[1, ?, ?, ?] Y)
    {
        Y = com.brainchip.AkidaOpInt8<program_path="program.bin">(X)
    }
""")

# 4. Run inference with ONNX Runtime
so = ort.SessionOptions()
so.register_custom_ops_library(akida_ort.get_library_path())

sess = ort.InferenceSession(
    onnx_model.SerializeToString(),
    so,
    providers=["CPUExecutionProvider"]
)

# 5. Execute inference
inputs = {"X": np.random.randint(0, 255, (1, 64, 64, 3), dtype=np.uint8)}
outputs = sess.run(None, inputs)
print(f"Output shape: {outputs[0].shape}, dtype: {outputs[0].dtype}")