Python Bindings for ARA-2

Python bindings for the ARA-2 neural network accelerator client library, providing efficient NPU inference from Python via a proxy service running on NXP i.MX platforms with Kinara ARA-2 hardware.

Published to PyPI as edgefirst-ara2.

Architecture

Python Application ──(UNIX/TCP socket)──▶ ARA-2 proxy ──(PCIe)──▶ ARA-2 NPU
       │                                (system service)         (Kinara hardware)
       │
edgefirst-image ──(DMA-BUF fd)──▶ GPU preprocessing (zero-copy)

Your Python code connects to the proxy system service (not directly to the hardware). The proxy manages device access and must be running before your application starts — it is rt-sdk-ara2.service on EdgeFirst Yocto images shipping NXP's rt-sdk-ara2 integration:

systemctl status rt-sdk-ara2

Installation

From PyPI

pip install edgefirst-ara2

For zero-copy preprocessing with the EdgeFirst HAL:

pip install edgefirst-ara2[hal]

The hal extra pulls edgefirst-image (and, through it, edgefirst-tensor). The HAL ships as one wheel per library rather than a single edgefirst-hal wheel; add edgefirst-codec for JPEG/PNG decode and edgefirst-decoder for YOLO post-processing when you need them.

Prerequisites for Development

• Python 3.11 or higher
• Rust stable toolchain (edition 2024)
• maturin (pip install maturin)
• ARA-2 client library (libaraclient, see ara2::LIBRARY_NAMES for the names tried)

Development Install

cd crates/ara2-py
maturin develop --release --features abi3

Quick Start

import edgefirst_ara2

# Connect to ARA-2 proxy
session = edgefirst_ara2.Session.connect()

# Get version information
versions = session.versions()
print(f"Proxy version: {versions['proxy']}")

# List endpoints
endpoints = session.list_endpoints()
print(f"Found {len(endpoints)} endpoints")

# Check endpoint status
for endpoint in endpoints:
    state = endpoint.check_status()
    stats = endpoint.dram_statistics()
    print(f"State: {state}, Free DRAM: {stats.free_size / stats.dram_size * 100:.1f}%")

Inference with numpy

import numpy as np
import edgefirst_ara2

session = edgefirst_ara2.Session.connect()
endpoints = session.list_endpoints()
model = endpoints[0].load_model("model.dvm")

# Allocate tensors and run inference
model.allocate_tensors()
input_data = np.zeros(model.input_size(0), dtype=np.uint8)
model.set_input_tensor(0, input_data)
timing = model.run()

print(f"Inference: {timing.run_time_us} us")
output = model.get_output_tensor(0)
dequantized = model.dequantize(0)

Zero-Copy DMA-BUF Pipeline

For maximum throughput, use DMA-BUF tensors with edgefirst-image for GPU-accelerated preprocessing. This eliminates CPU memory copies between preprocessing and inference:

How it works: allocate_tensors("dma") allocates the model's input tensor in a DMA-BUF — a Linux kernel buffer accessible by multiple hardware devices. input_tensor_fd(0) returns a file descriptor to that buffer. You pass this FD to edgefirst.image.ImageProcessor.import_image(), which maps it as a GPU image surface. The GPU writes the preprocessed frame directly into the NPU's input buffer — no CPU copies involved.

import os

import edgefirst.codec as ef_codec
import edgefirst.image as ef_image
import edgefirst_ara2 as ara2

session = ara2.Session.connect()
endpoint = session.list_endpoints()[0]
processor = ef_image.ImageProcessor()

with endpoint.load_model("yolov8s.dvm") as model:
    model.allocate_tensors("dma")  # Must use "dma" for tensor FD access

    # Get DMA-BUF FD for the model's input tensor
    input_fd = model.input_tensor_fd(0)
    c, h, w = model.input_shape(0)
    try:
        # Import as PlanarRgb (CHW layout) to match ARA-2 tensor format
        dst = processor.import_image(input_fd, w, h, ef_image.PixelFormat.PlanarRgb)
    finally:
        os.close(input_fd)  # FD duplicated by import_image; close original

    # edgefirst-codec decodes into a buffer edgefirst-image allocated. The
    # decoder never converts, so a colour JPEG lands in its native NV12.
    info = ef_codec.Tensor.peek_image_info_file("image.jpg")
    src = processor.create_image(info.width, info.height, info.format)
    ef_codec.decode_file_into(src, "image.jpg")

    # GPU-accelerated convert: source frame -> model input (zero CPU copies).
    # `letterbox=` fits the source preserving aspect ratio and pads the rest.
    processor.convert(src, dst, letterbox=(114, 114, 114, 255))

    # Run inference — NPU reads from the same DMA-BUF
    timing = model.run()
    print(f"Inference: {timing.run_time_us} us")

Performance

Benchmarked on NXP FRDM i.MX 95 + ARA-2 with YOLOv8m-seg (640×640). The Python API adds minimal overhead over native Rust thanks to DMA-BUF zero-copy — GPU and NPU operate on the same physical memory buffers.

Steady-state mean over 30 iterations after warmup. Python overhead is under 1 ms across the entire pipeline.

Run the benchmark yourself. Create a virtual environment on the target and install the packages from PyPI:

python3 -m venv ~/venv
~/venv/bin/pip install edgefirst-ara2 'edgefirst-codec>=0.31' \
    'edgefirst-decoder>=0.31' 'edgefirst-image>=0.31'
~/venv/bin/python3 yolov8.py yolov8m-seg-int16.dvm zidane.jpg --benchmark 30 --save

Models

Official pre-trained models are published in the EdgeFirst model zoo on Hugging Face:

Each repository ships one directory per target — tflite/, imx95/, onnx/, hailo/, jetson/, qnn/. The ARA-2 builds are the int16 .dvm exports under ara240/, in n/s/m sizes:

curl -LO https://huggingface.co/EdgeFirst/yolov8-det/resolve/main/ara240/yolov8n-det-int16.dvm
curl -LO https://huggingface.co/EdgeFirst/yolov8-seg/resolve/main/ara240/yolov8n-seg-int16.dvm

Every export embeds an edgefirst.json schema and the class labels.txt in a ZIP trailer appended to the .dvm; read_metadata() and read_labels() below read them without loading the model onto the NPU.

DVM Metadata

Read model metadata without loading onto the NPU:

import edgefirst_ara2

metadata = edgefirst_ara2.read_metadata("model.dvm")
if metadata:
    print(f"Task: {metadata.task}")
    print(f"Classes: {metadata.classes}")
    if metadata.compilation and metadata.compilation.ppa:
        print(f"IPS: {metadata.compilation.ppa.ips}")

labels = edgefirst_ara2.read_labels("model.dvm")

Async Inference

The submit() / wait() API enables overlapping CPU work with NPU execution. This is the building block for pipeline parallelism — while the NPU runs inference on one frame, the CPU can preprocess the next.

import edgefirst_ara2 as ara2

session = ara2.Session.connect()
endpoint = session.list_endpoints()[0]
model = endpoint.load_model("model.dvm")
model.allocate_tensors()

# Submit — returns immediately while NPU works
request = model.submit()
print(f"Request #{request.request_id} submitted")

# CPU is free to do other work here...
# The GIL is NOT held during wait(), so other Python threads can run

timing = request.wait()  # blocks until NPU finishes
print(f"NPU inference: {timing.run_time_us} µs")

# Monitor pipeline depth
print(f"In-flight: {session.inflight_count()}")

Warning: Do not call model.allocate_tensors() while an InferRequest is still pending — the NPU is actively reading/writing the tensor buffers.

API Reference

Session

Connection to the ARA-2 proxy service.

Static Methods:

• connect() -> Session - Connect over the UNIX socket named by ARA2_SOCKET, falling back to DEFAULT_SOCKET
• create_via_unix_socket(socket_path: str) -> Session
• create_via_tcp_ipv4_socket(ip: str, port: int) -> Session

Methods:

• versions() -> dict[str, str] - Get component versions
• list_endpoints() -> list[Endpoint] - List available endpoints
• inflight_count() -> int - Number of pending async inference requests

Properties:

• socket_type: str - "unix" or "tcp"

Endpoint

Represents an ARA-2 accelerator device.

Methods:

• check_status() -> State - Get device state
• dram_statistics() -> DramStatistics - Get memory usage
• load_model(model_path: str) -> Model - Load a .dvm model

Model

Loaded neural network model.

Lifecycle:

• allocate_tensors(memory: str | None = None) - Allocate tensors ("dma", "shm", "mem", or None)
• set_timeout_ms(timeout_ms: int) - Set inference timeout
• run() -> ModelTiming - Execute inference synchronously
• submit() -> InferRequest - Submit inference asynchronously (returns immediately)

Tensor I/O (numpy):

• set_input_tensor(index: int, data: np.ndarray) - Copy data into input
• get_output_tensor(index: int) -> np.ndarray - Copy output data out
• dequantize(index: int) -> np.ndarray - Dequantize output to float32

DMA-BUF Zero-Copy:

• input_tensor_fd(index: int) -> int - Get input tensor FD (pass to edgefirst.image.ImageProcessor.import_image, which dups it — close after)
• output_tensor_fd(index: int) -> int - Get output tensor FD (pass to edgefirst.tensor.Tensor.from_fd, which takes ownership — do not close after)
• input_tensor_memory(index: int) -> str - Input memory type
• output_tensor_memory(index: int) -> str - Output memory type

Introspection:

• n_inputs: int, n_outputs: int - Tensor counts
• input_shape(i) -> (C, H, W), output_shape(i) -> (C, H, W)
• input_size(i) -> int, output_size(i) -> int - Size in bytes
• input_bpp(i) -> int, output_bpp(i) -> int - Bytes per element
• input_info(i) -> InputTensorInfo, output_info(i) -> OutputTensorInfo
• input_quants(i) -> InputQuantization, output_quants(i) -> OutputQuantization

InferRequest

Pending asynchronous inference request, created by Model.submit().

Methods:

• wait(timeout_ms: int = 1000) -> ModelTiming - Block until complete (GIL released)

Properties:

• request_id: int - Proxy-assigned ID for log correlation

Module Functions

• socket_path() -> str - Resolve the proxy socket path: ARA2_SOCKET if set, otherwise the first entry of SOCKET_PATHS that exists
• discover() -> list[Proxy] - Find every running ARA-2 proxy and where it listens

Metadata Functions

• read_metadata(path: str) -> DvmMetadata | None
• read_labels(path: str) -> list[str]
• has_metadata(path: str) -> bool

Supporting Types

• Proxy: pid, endpoints, config, exe, connect() — a running proxy from discover()
• ProxyEndpoint: kind ("unix"/"tcp"), path, host, port, connect()
• Abi (enum): V1_1, V1_3 — the DVAPI generation of the loaded libaraclient
• State (enum): Init, Idle, Active, ActiveSlow, ThermalUnknown, Inactive, Fault, Unknown, plus ActiveBoosted and ThermalInactive under DVAPI 1.1 or ThermalActiveSlow and FailSafe under DVAPI 1.3
• ModelOutputType (enum): Classification, Detection, SemanticSegmentation, Raw
• DramStatistics: dram_size, free_size, model_occupancy_size, ...
• ModelTiming: run_time_us, input_time_us, output_time_us
• InputQuantization: qn, scale, mean, is_signed
• OutputQuantization: qn, scale, offset, is_signed

Exceptions

Ara2Error (RuntimeError)
 +-- LibraryError       - libaraclient loading failures
 +-- HardwareError      - NPU faults, endpoint errors
 +-- ProxyError         - Proxy connection failures
 +-- ModelError         - Model load/inference failures
 +-- TensorError        - Tensor allocation, DMA-BUF errors
 +-- MetadataError      - DVM metadata parsing errors

Building Wheels

cd crates/ara2-py
maturin build --release --features abi3

Wheels are created in target/wheels/.

Stable ABI

The bindings use PyO3's stable ABI (abi3-py311):

• A single wheel works across Python 3.11, 3.12, 3.13, and future versions
• Minimum supported Python version is 3.11

Troubleshooting

"libaraclient not found" (or a specific libaraclient_*.so variant)

export LD_LIBRARY_PATH=/path/to/ara2/lib:$LD_LIBRARY_PATH

Verify Installation

python -c "import edgefirst_ara2; print(edgefirst_ara2.__version__)"

License

Licensed under the Apache License 2.0.

Copyright 2025 Au-Zone Technologies. All Rights Reserved.