VapourSynth-MLRT-NCNN

This package contains the NCNN backend implementation of the vs-mlrt plugin.

Installation

pip install vapoursynth-mlrt-ncnn

Building from source

Requirements

• C++ Compiler: C++20 compatible (e.g. MSVC 2019+, GCC, Clang)
• Build Tools: uv, CMake, Ninja
• Dependencies (must be built/installed before the plugin):
  • ncnn (built from source in CI with Vulkan support enabled)
  • ONNX (built from source in CI)
  • Vulkan SDK (LunarG)
• Platforms: Windows, Linux, macOS (uses Metal via MoltenVK on macOS)

Compilation

1. Initialize the submodule:

   git submodule update --init --recursive vsmlrt/ncnn/vs-mlrt
   

2. Install the Vulkan SDK from LunarG.

3. Build NCNN from source (with Vulkan enabled) and build ONNX from source. Both must be discoverable by CMake (via CMAKE_PREFIX_PATH or installed system-wide).

4. Build the wheel:

   uv build --package vapoursynth-mlrt-ncnn
   

Detailed parameter information from the parent project follows.

vs-mlrt

This project provides VapourSynth ML filter runtimes for a variety of platforms:

• x86 CPUs: vsov-cpu, vsort-cpu
• Intel GPU (both integrated & discrete): vsov-gpu, vsncnn-vk
• NVidia GPU: vsort-cuda, vstrt & vstrt_rtx, vsncnn-vk
• AMD GPU: vsncnn-vk, vsmigx
• Apple SoC: vsort-coreml

Please refer to the wiki for supported models & usage information.

vsov: OpenVINO-based Pure CPU & Intel GPU Runtime

OpenVINO is an AI inference runtime developed by Intel, mainly targeting x86 CPUs and Intel GPUs.

The vs-openvino plugin provides optimized pure CPU & Intel GPU runtime for some popular AI filters. Intel GPU supports Gen 8+ on Broadwell+ and the Arc series GPUs.

Please visit the vsov directory for details.

vsort: ONNX Runtime-based CPU/GPU Runtime

ONNX Runtime is an AI inference runtime with many backends.

The vs-onnxruntime plugin provides optimized CPU and CUDA GPU runtime for some popular AI filters.

Please visit the vsort directory for details.

vstrt: TensorRT-based GPU Runtime

TensorRT is a highly optimized AI inference runtime for NVidia GPUs. It uses benchmarking to find the optimal kernel to use for your specific GPU, and so there is an extra step to build an engine from ONNX network on the machine you are going to use the vstrt filter, and this extra step makes deploying models a little harder than the other runtimes. However, the resulting performance is also typically much much better than the CUDA backend of vsort.

TensorRT-RTX is a specialization of TensorRT for NVIDIA RTX GPUs, which compiles engines faster with comparable performance with TensorRT.

Please visit the vstrt directory for details.

vsmigx: MIGraphX-based GPU Runtime

MIGraphX is a highly optimized AI inference runtime for AMD GPUs. It also uses benchmarking to find the optimal kernel, similar to vstrt.

Please visit the vsmigx directory for details.

vsncnn: NCNN-based GPU (Vulkan) Runtime

ncnn is a popular AI inference runtime. vsncnn provides a vulkan based runtime for some AI filters. It includes support for on-the-fly ONNX to ncnn native format conversion so as to provide a unified interface across all runtimes provided by this project. As it uses the device-independent Vulkan interface for GPU accelerated inference, this plugin supports all GPUs that provides Vulkan interface (NVidia, AMD, Intel integrated & discrete GPUs all provide this interface.) Another benefit is that it has a significant smaller footprint than other GPU runtimes (both vsort and vstrt CUDA backends require >1GB CUDA libraries.) The main drawback is that it's slower.

Please visit the vsncnn directory for details.