AraViQ6 - NDArray-based Video processing with Qt6

AraViQ6 is a Python package which integrates NDArray-based image processing with video pipeline of Qt6.

It provides:

• QVideoFrame processor based on array processing
• Converters between NDArray and QVideoFrame
• Convenience classes and widgets for array displaying

The following Qt bindings are supported:

• PySide6
• PyQt6

How to use

There are two ways to use AraViQ6; using QVideoFrame-based pipeline, or using NDArray-based pipeline.

Frame-based pipeline

Frame-based pipeline is a high-level approach that works well with Qt Multimedia scheme.

QVideoFrame pipeline with AraViQ6

Frame-based pipeline consists of:

1. VideoFrameWorker
2. VideoFrameProcessor

QVideoFrame comes from and goes to Qt6's QVideoSink. AraViQ6's VideoFrameWorker converts QVideoFrame to numpy array, performs processing, and sends the results to downstream in both QVideoFrame and NDArray. User may subclass the worker to define own processing.

VideoFrameProcessor wraps the worker and provides API around it. Worker is mulithreaded in the processor.

Array-based pipeline

Array-based pipeline is a low-level approach. It can be useful when third-party package provides video frame in numpy array format.

NDArray pipeline with AraViQ6

Array-based pipeline consists of:

1. FrameToArrayConverter
2. ArrayWorker
3. ArrayProcessor
4. ArrayToFrameConverter

FrameToArrayConverter and ArrayToFrameConverter performs conversion between frame pipeline and array pipeline. To retain the metadata (e.g., timestamp) of QVideoFrame, these classes includes the original frame for the array.

ArrayWorker performs processing on incoming array and sends the result to downstream in NDArray. User may subclass the worker to define own processing.

ArrayProcessor wraps the worker and provides API around it. Worker is mulithreaded in the processor.

Convenicence classes

AraViQ6 also provides various convenience classes to make building the pipeline easier.

The following classes help setting array pipeline with the video source and the display.

• NDArrayVideoPlayer
• NDArrayMediaCaptureSession
• NDArrayLabel

The following classes are plug-and-play widgets where user can process the video with minimal boilerplate.

• PlayerProcessWidget
• CameraProcessWidget

Examples

Use cases are provided in examples directory. They can be found in documentation as well.

Installation

Before you install, be careful for other Qt-dependent packages installed in your environment. For example, non-headless OpenCV-Python modifies the Qt dependency thus can make other Qt bindings unavailable.

araviq6 can be installed using pip.

$ pip install araviq6

Documentation

AraViQ6 is documented with Sphinx. Documentation can be found on Read the Docs:

https://araviq6.readthedocs.io/

If you want to build the document yourself, clone the source code and install with [doc] option. Go to doc directory and build the document.

$ pip install araviq6[doc]
$ cd doc
$ make html

Document will be generated in build/html directory. Open index.html to see the central page.