Qualcomm AI Runtime Development (QAIRT-DEV) package

[Docs] [Discord Forums]

The Qualcomm AI Runtime (QAIRT) Development Python API provides a simple interface for executing ML models on QAIRT runtimes

It mirrors select capabilities and extends the features of existing QAIRT command line tools, while also providing an intuitive Pythonic API for easy integration into ML workflows

Features

• Framework Model Conversion

  • Convert ONNX, Pytorch (1.x), TFLite framework models into DLC
  • Includes support for quantization and application of quantization encodings generated from AIMET

• Compilation

  • Perform AOT compilation on QAIRT backends to generate optimized binaries.
  • Perform compiler optimization using tuning API on HTP
  • Supports compilation on HTP, HTP MCP and AIC backends.

• Model Execution

  • Execute models on python native targets via Pybind wrappers on QAIRT APIs
  • Execute models on other targets (e.g android) via helper APIs that abstract platform specific details

• Model Analysis

  • Generate profiling reports on all supported backends
  • Generate Op Trace and Qualcomm Hexagon Analysis Summary (QHAS) reports on HTP

• Gen AI Model Building and Execution

  • Convert, optimize, and compile Gen AI models for on-device inference using a builder object with a single API call.
  • Perform text generation and obtain metrics via Generative AI Inference Engine
  • Construct Gen AI applications natively in python using simplified python bindings on Genie APIs.

Install

QAIRT Dev is available via pip:

pip install qairt-dev

Getting Started

QAIRT Dev documentation can be found here

Need help?

• Discord Forums: Qualcomm Developer Network - AI
• Support: qairt-dev-support@qti.qualcomm.com

LICENSE

Qualcomm AI Runtime Development (QAIRT-DEV) package is Proprietary licensed. See LICENSE.pdf for further details.