AMEVA-Runtime

Next-Gen Unified On-Device Hardware Orchestration & 6-Modality AI Acceleration Runtime (with BitNet 1.58-bit Vulkan Compute) for Mobile & Edge

Architecture & Overview

AMEVA-Runtime is a hardware abstraction layer (HAL) and compute orchestration engine engineered specifically for mobile ARM64 devices (Android Termux, Linux Edge). It continuously inspects underlying silicon topology (/dev/kgsl-3d0, /dev/mali0) to route tensor execution across Qualcomm Adreno, ARM Mali, and ARM Cortex CPU-NEON backends.

6-Modality Acceleration Matrix

Empirical Physical Device Benchmarks

Tested on physical devices running Android 16 under Termux ARM64:

1. LLM Generation (Qwen2.5-0.5B-Instruct Q4_K_M)

Measured 2026-10-04 with llama-completion of the v2.8.0 llama bundle, 4 threads; the GPU route is the bundle's ggml-ameva backend with all layers on the GPU. After a warm-up run, two prompt runs and three generation runs alternate with the CPU. Prompt processing is a prompt of about 370 tokens evaluated at once; token generation is 64 tokens, one at a time. The text the GPU route generates is the CPU's on all five devices.

Galaxy S25 was running other work during the measurement, hence its wide ranges. Galaxy S20 was measured at a GPU clock cap of 441 MHz; at 587 MHz its token generation was 28-31 t/s on the CPU and 32.6-32.9 t/s on the GPU. On the Mali devices token generation is faster on the CPU, and Galaxy S21 and A53, whose drivers lack integer dot products, take the CPU route by default. Larger models and the measurement conditions are in the v2.8.0 release notes.

2. Speech-to-Text (Whisper Large-v3-Turbo Q5_0, 548MB)

3. Text-to-Speech (AMEVA Bionic Native Vulkan Fleet Benchmarks)

4. BitNet 1.58-bit LLM (Microsoft BitNet b1.58 2B-4T i2_s)

Root-Cause Driver Solutions

1. ARM Mali-G68 Valhall Integer Truncation: Enforced medium tile matmul kernel dispatch (loadstride_b = 4 > 0), permanently eliminating shader zero-stride infinite loops on subgroup-16 hardware.
2. Qualcomm Adreno 830 JIT Register Bug: Bounded vector column specialization (mul_mat_vec_max_cols = 2), preventing compiler crash VK_ERROR_UNKNOWN (-13).
3. Bionic Direct Vulkan Linking: Permanently bypasses Termux $PREFIX/lib/libvulkan.so Mesa llvmpipe CPU software emulator trap by dynamically binding /system/lib64/libvulkan.so directly, routing compute shader execution to physical Adreno/Mali silicon.
4. Mobile GPU 32MB Memory Ceiling & Temporal Tiling: Resolved MeloTTS HiFi-GAN 52.4MB buffer overflow (VK_ERROR_OUT_OF_DEVICE_MEMORY / kernel TDR kill) via temporal chunk slicing ($T_{\text{chunk}} \le 819$) and Piper VITS on-chip SRAM tiling.
5. Zero-Silent-Fallback & Strict Model Resolution: Guaranteed fail-fast architecture without silent CPU degradation. Explicitly raises AmbiguousModelMatchError on multi-model ambiguity and rejects silent guessing.

Two-Track Supply Chain Architecture

To simultaneously guarantee 100% intellectual property & source code protection and frictionless, tokenless binary deployment, AMEVA enforces a strict Two-Track Supply Chain Architecture:

[Track 1: Local Ground Truth & Offline Staging]
├── dev/ameva/ameva-runtime/releases/ (Consolidated 6-Modality SOTA Binaries)
└── dev/termux/termux-*/releases/     (Domain-Specific Compiled Binaries)
                     │
                     ▼ Synchronized via Git & Automated Release Workflows
[Track 2: Remote Public GitHub Releases  Cloud & CDN]
├── uno-km/ameva-runtime-releases    (Public Unified Binary Hub: Releases v2.7.9)
└── uno-km/termux-* (diffusion, stt) (Domain-Specific Public GitHub Releases)

1. Core Source Code (uno-km/ameva-runtime): Kept 100% Private. Architectural proprietary logic, HAL implementations, and optimization kernels are permanently protected.
2. Public Binary Distribution Hub (uno-km/ameva-runtime-releases): Public Registry. Hosts verified precompiled binaries, hardware acceleration compute shaders, and wheels for anonymous 1-Click installation without requiring personal GitHub tokens.

Installation & Engine Auto-Provisioning

1. Python SDK Installation

# Direct install from public release wheel
pip install --upgrade "https://github.com/uno-km/ameva-runtime-releases/releases/download/v2.7.9/ameva_runtime-2.7.9-py3-none-any.whl"

# Or standard PyPI install
pip install ameva-runtime

2. 1-Click Native Engine Provisioning

The NativeAssetManager automatically retrieves cryptographically verified native engine binaries from the public registry and establishes system links in $PREFIX/bin:

# Provision all hardware engines at once
ameva install --all

# Or provision specific engine modalities individually
ameva install --modality diffusion   # Stable Diffusion Vulkan Engine (sd-cli)
ameva install --modality stt         # Whisper STT Vulkan Engine (whisper-cli)
ameva install --modality llm         # LLaMA C++ Vulkan Engine (llama-cli)
ameva install --modality tts         # Sherpa-NCNN Neural Voice Engine
ameva install --modality bitnet      # BitNet 1.58-bit Acceleration Engine

# Validate system readiness with 12-stage hardware diagnostic
ameva doctor

Quickstart

Python SDK

import ameva_runtime as ameva
from ameva_runtime import vulkan

# 1. Inspect on-device silicon topology
profile = ameva.detect_hardware()
print(f"SoC: {profile.soc_name} | GPU: {profile.gpu_vendor}")

# 2. Run hardware self-test
doc = vulkan.Doctor()
report = doc.run_self_test()
print(f"GPU: {report.device_name} (Passed: {report.passed_stages}/{report.total_stages})")

Node.js / TypeScript

import { Doctor, createContext } from '@ameva/runtime';

const doc = new Doctor();
const report = await doc.runSelfTest();
console.log(`Vulkan GPU: ${report.deviceName}`);

Official Documentation & Benchmarks

• Official Architecture & API Reference
• Ecosystem Metrics & Registry Stats
• AMEVA Open-Source Foundation Portal

License

Licensed under the Apache-2.0 License. Copyright (c) 2026 Eunho Kim (@uno-km).