qai-appbuilder 2.48.40

AppBuilder is Python & C++ extension that simplifies the process of developing AI prototype & App on WoS. It provides several APIs for running QNN models in WoS CPU & HTP, making it easier to manage AI models.

---

QAI AppBuilder

The Quick AI Application Builder (this repository) is also referred to as "QAI AppBuilder" in the source code and documentation. It is an extension of the Qualcomm® AI Runtime SDK that substantially lowers the complexity of on-device AI model deployment for developers.

With QAI AppBuilder, a single natural language prompt drives the full AI application build pipeline — no coding required:

• Path A — Pre-built models from AI Hub: The AI Hub Model Run Skill automatically downloads pre-exported QNN models directly from Qualcomm AI Hub and runs inference on the Snapdragon NPU without any conversion step.
• Path B — Custom model conversion: The Model Builder Skill takes your own PyTorch or ONNX model and automatically handles the full pipeline — export, quantization (FP16 / W8A16 / W8A8 / W4A8), and context binary generation via QAIRT SDK — producing an optimized .bin/.dlc ready for on-device inference.

Both paths converge on the same NPU inference engine (QNNContext via qai-appbuilder), running fully on-device on Snapdragon series — no internet connection required at inference time.

What can QAI AppBuilder do?

Capability	Description
Run AI Models on NPU / GPU / CPU	Load QNN context binaries (.bin), model libraries (.dll), or SNPE DLC (.dlc) onto the Snapdragon NPU, GPU, or CPU and perform high-performance inference
C++ & Python APIs	Full-featured bindings for both C++ and Python, so you can integrate on-device AI into any project regardless of language
Cross-Platform Support	Runs on Windows on Snapdragon (WoS) ARM64 and Linux (e.g. QCS8550, QCM6490), with a unified API surface across both platforms
AI-Driven Model Conversion	Convert PyTorch / ONNX models to QNN format at multiple precisions (FP16 / W8A16 / W8A8 / W4A8) through natural language chat — the AI handles export, quantization, context binary generation, and accuracy validation end-to-end
Pre-built Models from AI Hub	Automatically download and run pre-exported QNN models from Qualcomm AI Hub with no conversion step required
On-Device Model Workbench	Run ready-to-use Model Packs (SR / OCR / ASR / TTS / CV…) on the local Snapdragon NPU with one click; supports multi-variant precision switching, benchmarking, and side-by-side comparison
LLM Agent Pipeline	Orchestrate multiple local NPU models in a single task via natural language — e.g. classify images then upscale matches, all driven by one prompt
Large Language Models (Genie)	Run Llama 3 / Qwen 3 / Gemma 2 and other quantized LLMs fully offline on the NPU via GenieAPIService (OpenAI-compatible API); switch between local and cloud models in the same chat UI
Multimodal & Speech & Vision	Supports multimodal LLMs (e.g. Qwen2.5-VL), ASR (Whisper, Zipformer), TTS (MeloTTS), OCR (PP-OCR), super-resolution, object detection, and more
WebUI Applications	Bundled WebUI apps (StableDiffusionApp, ImageRepairApp, GenieWebUI) and a streaming chat WebUI — all running on-device, no internet required
Skill Plugin System	Extend the AI with hot-reloadable Skill plugins; write your own in a single SKILL.md file
Native & Float I/O / Multi-Graph / LoRA	Native (quantized) and float I/O for maximum throughput; multiple model graphs in one session; LoRA adapter support

Supports ARM64 Windows, Linux and Ubuntu (e.g. X Elite Windows, QCS8550 Linux, QCM6490 Ubuntu). Use "native" mode I/O to improve data throughput — see User Guide and Whisper sample for reference.

---

QAI Skills

QAI AppBuilder ships two built-in AI Agent Skills that together cover the full on-device model lifecycle — from sourcing a model all the way to running inference on the Snapdragon NPU.

Skill 1 — AI Hub Model Run

Use this skill when the model you need already exists on Qualcomm AI Hub.

The AI Hub Model Run skill downloads pre-exported QNN models directly from AI Hub and runs them on the Snapdragon NPU via qai_appbuilder (QNNContext) — no conversion, no Visual Studio, no QAIRT SDK required.

What it does	Details
Supported formats	QNN_CONTEXT_BINARY (.bin), QNN_DLC (.dlc), ONNX (CPU baseline only), VOICE_AI, TFLITE
Inference engine	Always qai_appbuilder.QNNContext loading .bin / .dlc on the NPU/HTP; onnxruntime is used only for optional CPU accuracy comparison
Target devices	Snapdragon X Elite, Snapdragon X2 Elite, Snapdragon X Plus 8-Core
Typical workflow	Detect chipset → fetch download link from AI Hub → curl download → extract ZIP → run inference script
No conversion needed	Pre-compiled .bin / .dlc packages load directly — first-run graph compilation takes 5–60 s, subsequent runs are fast
CPU baseline	Optionally compare NPU output against onnxruntime CPU baseline (cosine similarity > 0.999 for float models)

Example trigger prompts:

"Download Inception V3 from AI Hub and run inference on my photo"
"Run YOLOv8 object detection on this image using the NPU"
"Use the pre-built Whisper model to transcribe my audio file"

---

Skill 2 — Model Builder

Use this skill when you have a custom PyTorch or ONNX model that is NOT available on AI Hub.

The Model Builder skill automates the full QNN conversion pipeline for custom models — from ONNX export all the way to a validated .bin context binary running on the Snapdragon NPU.

What it does	Details
Input formats	PyTorch (.pt / .pth) → ONNX export, or existing ONNX (.onnx)
Output formats	QNN context binary (.bin), QNN model library (.dll), SNPE DLC (.dlc)
Supported precisions	FP16, FP32, W8A16, W8A8, W8A8B8, W4A16, W4A8
Auto pipeline	Export → ONNX inspection → operator patching → QNN conversion → context binary generation → inference → accuracy validation vs. ONNX CPU baseline
Operator patching	Automatically detects and patches unsupported operators (Einsum, GridSample, ScatterND, Mod, Floor…) with QNN-compatible equivalents
Accuracy validation	Computes cosine similarity between QNN output and ONNX CPU baseline; threshold ≥ 0.99 (FP16/FP32) or ≥ 0.95 (INT8/quantized)
Auto-generated inference code	Saves a standalone infer_<model>.py and inference_manifest.json after successful validation — ready for App Builder Pack export
Requires	QAIRT SDK 2.45+, Visual Studio 2022 Community, Python x64 3.10 (all auto-installed by Setup.bat)
Scope	Best suited for small-to-medium models (recommended < 2 GB); LLM conversion is not yet supported

Example trigger prompts:

"Convert my ResNet ONNX model to QNN FP16 and validate the accuracy"
"Convert my custom YOLOv8 model to W8A8 with calibration data and compare against the original"
"Export my PyTorch model to QNN, run inference, and generate an App Builder pack"

---

Quick Start

Python

pip install qai-appbuilder

C++

Download the prebuilt binary package from Releases:

QAI_AppBuilder-win_arm64-{Qualcomm® AI Runtime SDK version}-Release.zip

Refer to User Guide for full API usage, or follow tutorial.ipynb to set up and run a CV model step by step.

Environment Setup

Refer to python.md for instructions on setting up the Python (x64) environment to use QAI AppBuilder on Windows on Snapdragon (WoS) platforms.

You can also run the batch file from QAI AppBuilder Launcher to set up the environment automatically — enabling you to experience the core functionalities within an hour.

---

Diagram

---

WebUI AI Application

We have developed several WebUI AI applications based on QAI AppBuilder, allowing you to experience them quickly. All these applications run on a local PC, requiring no internet connection and are completely free. You can run WebUI AI applications through the batch file 4.Start_WebUI.bat.

Note: Before trying other functions, we suggest that you try these WebUI AI applications first.

App	Description
ImageRepairApp	An image restoration tool designed to repair old or damaged photographs.
StableDiffusionApp	A text-to-image generation tool that creates images based on user input.
GenieWebUI	A large language model (LLM) interface that enables interactive conversations.

---

OpenAI Compatible API Service (GenieAPIService)

Considering that the current mainstream method for invoking LLMs is based on OpenAI-compatible APIs, we have implemented such interfaces in both C++ and Python. This allows application developers to interact with the local large language model running on NPU in a familiar way.

Many third-party applications that support the OpenAI API can seamlessly switch to the local NPU-based model by simply changing the API endpoint.

We have also implemented client sample code for GenieAPIService in both C++ and Python for developer reference.

1. Python based service: Guide to run OpenAI compatible API services developed with Python.
2. C++ based service: Guide to run OpenAI compatible API services developed with C++.

---

Samples

We have a rich set of samples covering multiple categories. All models are sourced from AI Hub and automatically downloaded on first run.

Use the interactive launcher to run any sample without writing code:

cd qai-appbuilder\samples
python run_inference.py              # interactive menu
python run_inference.py --list       # list all available models
python run_inference.py --model whisper_base_en --args "--audio_file input.wav"

Category	Description	Link
Audio	TTS (PiperTTS), ASR (Whisper Base/Tiny), Audio Classification (YAMNet)	audio/
Computer Vision	Image classification, object detection, segmentation, depth estimation, pose estimation, face analysis, super-resolution, inpainting	computerVision/
Generative AI	Stable Diffusion v1.5 / v2.1 / v3.5 (text → image)	generativeAI/
Multimodal	OCR (EasyOCR), text embedding (NomicEmbed), CLIP, Chinese→English translation (OpusMT), VLM (Qwen-VL)	multimodal/
WebUI Apps	Gradio-based apps: ImageRepairApp, StableDiffusionApp, GenieWebUI	webui/
Apps	StorySeed (AI story + image → Xiaohongshu), FletUI desktop app	apps/
Genie LLM Service	OpenAI-compatible LLM API service (Python + C++) for Llama, Qwen, Phi, Granite	genie/
Android	GenieChat (LLM/VLM) and SuperResolution Android apps	android/
C++	C++ inference samples for Real-ESRGAN, BEiT	c++/

See samples/README.md for the full guide including environment setup, model download instructions, and run examples.

---

Tools

1. QAI AppBuilder Launcher

QAI AppBuilder Launcher — enables you to experience the core functionalities of QAI AppBuilder within an hour.

2. DLC2BIN

DLC2BIN — converts the general DLC model format into the BIN format optimized for a specific platform.

3. ONNX2BIN

ONNX2BIN — converts the ONNX model format into the BIN format optimized for a specific platform.

4. ONNXWRAPPER

ONNXWRAPPER — a wrapper to run ONNX inference code with a QNN model, which switches to the QNN runtime automatically.

5. SKILLS

SKILLS includes 3 skills:

• genie_api_service — GenieAPIService technical documentation retrieval
• qai_app_builder — QAI AppBuilder technical documentation retrieval
• qai-runner-skill — QAIRT model conversion & inference on Qualcomm devices

---

Models

Model Hub

Hub	Link
AI Hub	aihub.qualcomm.com
AI Dev Home	aidevhome.com

LLM Models

Model	Link
Qwen2 7B SSD	Download
DeepSeek-R1-Distill-Qwen-7B	Download

---

Blog & Documentation

Official Docs

Guide	Links
QAI AppBuilder Guide	English | 中文
GenieAPIService (OpenAI Compatible API)	English | 中文
Qwen2.5-VL-3B On-Device Deployment	English | 中文
QAI AppBuilder WoS PDF	PDF
QAI AppBuilder on Linux (QCS6490)	English

Blog Posts

The following blog posts are in Chinese (中文).

Title	Link
3分钟上手，在骁龙AI PC上部署DeepSeek	中文
本地 OpenAI 兼容 API 服务的配置与部署	中文
Qwen2.5-VL-3B 多模态模型端侧部署	中文
BGE-Base-Zh-V1.5 端侧使用教程	中文
Qwen3-Reranker-0.6B 使用指南	中文
Qwen3-embedding-0.6B 使用指南	中文
Qwen3-8B-8K 模型端侧部署指南	中文
高通平台大语言模型精选	中文
Qwen2 7B SSD 使用教程	中文
Qwen2.5 3B 使用教程	中文
Genie API Service 配置与使用	中文
GenieChat：Genie API Service 安卓应用开发	中文
SuperResolutionApp：图片超分 Android 开发示例	中文

---

Third-Party App List

App	Description
stable-diffusion-webui Extension	Stable Diffusion WebUI plugin accelerated by QAIRT
Blender ControlNet Plugin	Blender image generation plugin for Snapdragon
无痕修图软件 (Inpainting App)	AI-powered photo inpainting tool
图片超分器 (Super-Resolution Tool)	Image super-resolution upscaler
图片超分应用 (Super-Resolution App)	Super-resolution application
视频超分应用 (Video Super-Resolution App)	Video super-resolution upscaler
图片消除器 (Object Removal Tool)	AI-powered object removal from images
图片搜索应用 (Image Search App)	AI-powered image search application

---

QAI AppBuilder Components

There are two ways to use QAI AppBuilder:

1. C++ Library

Download the prebuilt binary package from Releases and link the headers and .lib/.so files into your C++ project:

QAI_AppBuilder-win_arm64-{Qualcomm® AI Runtime SDK version}-Release.zip

2. Python Binding

Install via pip (see Quick Start) or download a specific wheel for your Python version from Releases.

---

Build

You can use the pre-compiled version directly — download the version you need from Releases. To build other wheel variants, refer to BUILD.md.

---

License

QAI AppBuilder is licensed under the BSD 3-clause "New" or "Revised" License. Check out the LICENSE for more details.

---

Star History

---

Disclaimer

This software is provided "as is," without any express or implied warranties. The authors and contributors shall not be held liable for any damages arising from its use. The code may be incomplete or insufficiently tested. Users are solely responsible for evaluating its suitability and assume all associated risks.

Note: Contributions are welcome. Please ensure thorough testing before deploying in critical systems.