AMEVA-Runtime (Python)

모바일 및 엣지 환경을 위한 차세대 통합 온디바이스 하드웨어 오케스트레이션 및 6대 멀티모달 가속 런타임
Next-Gen Unified On-Device Hardware Orchestration & 6-Modality AI Acceleration Runtime for Mobile & Edge

Installation

pip install ameva-runtime

Quickstart

import ameva_runtime as ameva
from ameva_runtime import vulkan

# 1. Execute LLM inference directly with optimal on-device hardware dispatch
result = ameva.run(
    model="qwen2.5-0.5b",
    prompt="Space in Korean is:",
    max_tokens=32
)
print(f"Generated text: {result.text}")
print(f"Hardware backend: {result.backend_used} ({result.tokens_per_second:.2f} t/s)")

# 2. Hardware diagnostic inspection via Vulkan engine
doc = vulkan.Doctor()
report = doc.run_self_test(verbose=False)
print(f"GPU Target: {report.device_name} (Passed: {report.passed_stages}/{report.total_stages})")

Description

AMEVA Runtime is a unified on-device hardware orchestration and AI acceleration engine engineered for mobile and edge systems. It dynamically inspects SoC topology and driver environments, routing compute graphs between Qualcomm Adreno, ARM Mali, and ARM Cortex CPU-NEON.

1. Empirical Real-Device LLM Benchmarks (Qwen2.5-0.5B-Instruct, GGUF Q4_K_M)

2. Empirical Real-Device STT Benchmarks (Whisper Large-v3-Turbo Q5_0, 548MB)

• Test Device: Samsung Galaxy A35 5G (Exynos 1380, ARM Mali-G68 MP5, 8GB RAM, Android 16 Termux)
• Audio Source: John F. Kennedy 1-minute speech sample (jfk_1min.wav)

• Galaxy S25 (Adreno 830) STT: Completed 4,401 ms inference on speech input (JIT compiler bug resolved).

3. Root-Cause Defect Resolution (Ground Truth)

(1) ARM Mali-G68 Valhall Integer Truncation Infinite Loop Elimination

• Defect: Executing mul_mm.comp on Mali-G68 (subgroup size 16) caused GPU hangs and hardware watchdog TDR resets (VK_ERROR_DEVICE_LOST).
• Root Cause: The stride calculation loadstride_b = gl_WorkGroupSize.x * LOAD_VEC_B / BK = 16 * 1 / 32 = 0 truncated to zero in integer division, producing an infinite loop for (uint l = 0; l < BN; l += 0).
• Resolution: Enforced Medium MatMul kernels (_m, workgroup size 128, loadstride_b = 4 > 0) via enforce_medium_matmul: true, enabling stable 25/25 layer GPU offloading.

(2) Qualcomm Adreno 830 JIT Compiler Bug Isolation

• Defect: Whisper STT pipeline compilation failed on Snapdragon 8 Elite with VK_ERROR_UNKNOWN (-13) during mul_mat_vec dispatch.
• Root Cause: Qualcomm's Adreno JIT compiler failed register allocation when Specialization Constant NUM_COLS >= 3.
• Resolution: Bound mul_mat_vec_max_cols = 2 for Adreno 830, achieving stable GPU inference in 4,401 ms on speech input.

4. 6-Modality Acceleration Roadmap

5. Architectural Principles

• Fail-Fast & Zero-Silent-Fallback: Never disguise GPU failures as CPU success. If a hardware backend fails, immediate explicit exceptions and telemetry are raised.
• Consolidated Single Package: Distributed cleanly via pip install ameva-runtime and npm install @ameva/runtime.

Documentation

• Official Documentation & API Reference
• GitHub Repository

License

Apache-2.0 License. Copyright (c) 2026 Eunho Kim (@uno-km).