nndeploy 3.0.10

nndeploy: An Easy-to-Use, and High-Performance AI Deployment Framework

Introduction

nndeploy is an easy-to-use, and high-performance AI deployment framework. Based on the design concepts of visual workflows and multi-backend inference, developers can quickly develop SDKs for specified platforms and hardware from algorithm repositories, significantly saving development time. Furthermore, the framework has already deployed numerous AI models including LLM, AIGC generation, face swap, object detection, image segmentation, etc., ready to use out-of-the-box.

Simple and Easy to Use

• Visual Workflow: Deploy AI algorithms through drag-and-drop operations. Visually adjust all node parameters of the AI algorithm in the frontend and quickly preview the effect after parameter tuning.
• Custom Nodes: Support Python/C++ custom nodes, seamlessly integrated into the visual interface without frontend code.
• Algorithm Combination: Flexibly combine different algorithms to quickly build innovative AI applications.
• One-Click Deployment: The completed workflow can be exported as a JSON configuration file with one click, supporting direct calls via Python/C++ API, achieving seamless transition from development to production environments, and fully supporting platforms like Linux, Windows, macOS, Android, iOS, etc.

High Performance

• Parallel Optimization: Supports execution modes like serial, pipeline parallel, task parallel, etc.

• Memory Optimization: Optimization strategies like zero-copy, memory pool, memory reuse, etc.

• High-Performance Optimization: Built-in nodes optimized with C++/CUDA/Ascend C/SIMD, etc.

• Multi-Backend Inference: One workflow, multiple backend inference. Integrates 13 mainstream inference frameworks with zero abstraction cost, covering all platforms including cloud, desktop, mobile, edge, etc.

  Inference Framework	Application Scenario	Status
  ONNXRuntime	Cross-platform inference	✅
  TensorRT	NVIDIA GPU high-performance inference	✅
  OpenVINO	Intel CPU/GPU optimization	✅
  MNN	Mobile inference engine by Alibaba	✅
  TNN	Mobile inference engine by Tencent	✅
  ncnn	Mobile inference engine by Tencent	✅
  CoreML	iOS/macOS native acceleration	✅
  AscendCL	Huawei Ascend AI chip inference framework	✅
  RKNN	Rockchip NPU inference framework	✅
  SNPE	Qualcomm Snapdragon NPU inference framework	✅
  TVM	Deep learning compiler stack	✅
  PyTorch	Rapid prototyping / Cloud deployment	✅
  Self-developed Inference Framework	Default inference framework	✅

Out-of-the-Box Algorithms

List of deployed models, with 100+ nodes created. We will continue to deploy more high-value AI algorithms. If you have algorithms you need deployed, please let us know via issue.

See more details in the Deployed Model List Details