AMEVA-Runtime

Next-Gen Unified On-Device Hardware Orchestration & 6-Modality AI Acceleration Runtime (with BitNet 1.58-bit Vulkan Compute) for Mobile & Edge

Architecture & Overview

AMEVA-Runtime is a hardware abstraction layer (HAL) and compute orchestration engine engineered specifically for mobile ARM64 devices (Android Termux, Linux Edge). It continuously inspects underlying silicon topology (/dev/kgsl-3d0, /dev/mali0) to route tensor execution across Qualcomm Adreno, ARM Mali, and ARM Cortex CPU-NEON backends.

6-Modality Acceleration Matrix

Empirical Physical Device Benchmarks

Tested on physical devices running Android 16 under Termux ARM64:

1. LLM Generation (Qwen2.5-0.5B-Instruct Q4_K_M)

Measured 2026-10-04 with the v2.7.9 bundle through ameva-run exec, 4 threads, single runs; the generated text is compared with the CPU route's. The Vulkan rows run with flash attention off (-fa 0): with it the bundle's Vulkan backend returns corrupt text. The figures this table carried up to 2.7.8 (35.80 t/s on Galaxy S25, 4.44 t/s on Galaxy A35) are not reproduced with this bundle and are withdrawn.

For this model the bundle's Vulkan backend is slower than the CPU on both devices. Larger models, the other devices and the known defects are in the v2.7.9 release notes.

2. Speech-to-Text (Whisper Large-v3-Turbo Q5_0, 548MB)

3. Text-to-Speech (AMEVA Bionic Native Vulkan Fleet Benchmarks)

4. BitNet 1.58-bit LLM (Microsoft BitNet b1.58 2B-4T i2_s)

Root-Cause Driver Solutions

1. ARM Mali-G68 Valhall Integer Truncation: Enforced medium tile matmul kernel dispatch (loadstride_b = 4 > 0), permanently eliminating shader zero-stride infinite loops on subgroup-16 hardware.
2. Qualcomm Adreno 830 JIT Register Bug: Bounded vector column specialization (mul_mat_vec_max_cols = 2), preventing compiler crash VK_ERROR_UNKNOWN (-13).
3. Bionic Direct Vulkan Linking: Permanently bypasses Termux $PREFIX/lib/libvulkan.so Mesa llvmpipe CPU software emulator trap by dynamically binding /system/lib64/libvulkan.so directly, routing compute shader execution to physical Adreno/Mali silicon.
4. Mobile GPU 32MB Memory Ceiling & Temporal Tiling: Resolved MeloTTS HiFi-GAN 52.4MB buffer overflow (VK_ERROR_OUT_OF_DEVICE_MEMORY / kernel TDR kill) via temporal chunk slicing ($T_{\text{chunk}} \le 819$) and Piper VITS on-chip SRAM tiling.
5. Zero-Silent-Fallback & Strict Model Resolution: Guaranteed fail-fast architecture without silent CPU degradation. Explicitly raises AmbiguousModelMatchError on multi-model ambiguity and rejects silent guessing.

Two-Track Supply Chain Architecture

To simultaneously guarantee 100% intellectual property & source code protection and frictionless, tokenless binary deployment, AMEVA enforces a strict Two-Track Supply Chain Architecture:

[Track 1: Local Ground Truth & Offline Staging]
├── dev/ameva/ameva-runtime/releases/ (Consolidated 6-Modality SOTA Binaries)
└── dev/termux/termux-*/releases/     (Domain-Specific Compiled Binaries)
                     │
                     ▼ Synchronized via Git & Automated Release Workflows
[Track 2: Remote Public GitHub Releases  Cloud & CDN]
├── uno-km/ameva-runtime-releases    (Public Unified Binary Hub: Releases v2.7.9)
└── uno-km/termux-* (diffusion, stt) (Domain-Specific Public GitHub Releases)

1. Core Source Code (uno-km/ameva-runtime): Kept 100% Private. Architectural proprietary logic, HAL implementations, and optimization kernels are permanently protected.
2. Public Binary Distribution Hub (uno-km/ameva-runtime-releases): Public Registry. Hosts verified precompiled binaries, hardware acceleration compute shaders, and wheels for anonymous 1-Click installation without requiring personal GitHub tokens.

Installation & Engine Auto-Provisioning

1. Python SDK Installation

# Direct install from public release wheel
pip install --upgrade "https://github.com/uno-km/ameva-runtime-releases/releases/download/v2.7.9/ameva_runtime-2.7.9-py3-none-any.whl"

# Or standard PyPI install
pip install ameva-runtime

2. 1-Click Native Engine Provisioning

The NativeAssetManager automatically retrieves cryptographically verified native engine binaries from the public registry and establishes system links in $PREFIX/bin:

# Provision all hardware engines at once
ameva install --all

# Or provision specific engine modalities individually
ameva install --modality diffusion   # Stable Diffusion Vulkan Engine (sd-cli)
ameva install --modality stt         # Whisper STT Vulkan Engine (whisper-cli)
ameva install --modality llm         # LLaMA C++ Vulkan Engine (llama-cli)
ameva install --modality tts         # Sherpa-NCNN Neural Voice Engine
ameva install --modality bitnet      # BitNet 1.58-bit Acceleration Engine

# Validate system readiness with 12-stage hardware diagnostic
ameva doctor

Quickstart

Python SDK

import ameva_runtime as ameva
from ameva_runtime import vulkan

# 1. Inspect on-device silicon topology
profile = ameva.detect_hardware()
print(f"SoC: {profile.soc_name} | GPU: {profile.gpu_vendor}")

# 2. Run hardware self-test
doc = vulkan.Doctor()
report = doc.run_self_test()
print(f"GPU: {report.device_name} (Passed: {report.passed_stages}/{report.total_stages})")

Node.js / TypeScript

import { Doctor, createContext } from '@ameva/runtime';

const doc = new Doctor();
const report = await doc.runSelfTest();
console.log(`Vulkan GPU: ${report.deviceName}`);

Official Documentation & Benchmarks

• Official Architecture & API Reference
• Ecosystem Metrics & Registry Stats
• AMEVA Open-Source Foundation Portal

License

Licensed under the Apache-2.0 License. Copyright (c) 2026 Eunho Kim (@uno-km).