visiongraph 0.1.60.1

Visiongraph is a high level computer vision framework.

Visiongraph

Visiongraph is a high level computer vision framework that includes predefined modules to quickly create and run algorithms on images. It is based on opencv and includes other computer vision frameworks like Intel openVINO and Google MediaPipe.

Here an example on how to start a webcam capture and display the image:

import visiongraph as vg
vg.create_graph(vg.VideoCaptureInput()).then(vg.ImagePreview()).open()

The main goal is to implement a platform independent and high performance framework for day-to-day computer vision tasks.

Installation

Visiongraph supports python 3.10 and 3.11. Other versions might work as well but are not officially supported.

To install visiongraph with all dependencies call pip like this:

pip install "visiongraph[all]"

🚨 Please note that visiongraph is in an early alpha phase and the API will still undergo changes.

It is also possible to only install certain packages depending on your needs:

pip install "visiongraph[realsense, openvino, mediapipe, onnx, media, azure, numba, opencv-contrib]"

Development

To develop visiograph itself it is recommended to clone this repository and install the dependencies like this:

# in the visiongraph directory
pip install -e ".[all]"

Build

To build a new wheel package of visiongraph run the following command in the root directory.

python setup.py bdist_wheel

Examples

To demonstrate the possibilities of visiongraph there are already implemented examples ready for you to try out. Here is a list of the current examples:

• SimpleVisionGraph - SSD object detection & tracking of live webcam input with 5 lines of code.
• VisionGraphExample - A face detection and tracking example with custom events.
• InputExample - A basic input example that determines the center if possible.
• RealSenseDepthExample - Display the RealSense or Azure Kinect depth map.
• FaceDetectionExample - A face detection pipeline example.
• FindFaceExample - A face recognition example to find a target face.
• CascadeFaceDetectionExample - A face detection pipeline that also predicts other feature points of the face.
• HandDetectionExample - A hand detection pipeline example.
• PoseEstimationExample - A pose estimation pipeline which annotates the generic pose keypoints.
• ProjectedPoseExample - Project the pose estimation into 3d space with the RealSense camera.
• ObjectDetectionExample - An object detection & tracking example.
• InstanceSegmentationExample - Intance Segmentation based on COCO80 dataset.
• InpaintExample - GAN based inpainting example.
• MidasDepthExample - Realtime depth prediction with the midas-small network.
• RGBDSmoother - Smooth RGB-D depth map videos with a one-euro filter per pixel.

There are even more examples where visiongraph is currently in use:

• Spout/Syphon RGB-D Example - Share RGB-D images over spout or syphon.
• WebRTC Input - WebRTC input example for visiongraph

Documentation

This documentation is intended to provide an overview of the framework. A full documentation will be available later.

Graph

The core component of visiongraph is the BaseGraph class. It contains and handles all the nodes of the graph. A BaseGraph can run on the same thread as called or a new thread or process. The nodes in the graph are just a list, the graph itself is created by nesting nodes into each other.

Graph Node

A GraphNode is a single step in the graph. It has a input and output type and processes the data within the process() method.

Graph Builder

The graph builder helps to create new graphs on a single line in python. It creates a VisionGraph object which is a child of the BaseGraph. The following code snippet is an example of the graph builder which creates a smooth pose estimation graph.

import visiongraph as vg

graph = (vg.create_graph(name="Smooth Pose Estimation",
                         input_node=vg.VideoCaptureInput(0),
                         handle_signals=True)
         .apply(ssd=vg.sequence(vg.OpenPoseEstimator.create(), vg.MotpyTracker(), vg.LandmarkSmoothFilter()),
                image=vg.passthrough())
         .then(vg.ResultAnnotator(image="image"), vg.ImagePreview())
         )
graph.open()

Input

Supported are image, video, webcam, RealSense and Azure Kinect input types.

Estimator

Usually an estimator is a graph node which takes an image as an input and estimates an information about the content. This could be a pose estimation or a face detection. It is also possible to have a transformation of the image, for example de-blurring it or estimate the depth map.

Object Detection Tracker

Object detection trackers allow a detected object to be assigned an id that remains the same across successive frames.

DSP (Digital Signal Processing)

To filter noisy estimations or inputs, the DSP package provides different filters which can be applied directly into a graph.

Recorder

To record incoming frames or annotated results, multiple frame recorders are provided.

Assets

Most estimators use big model and weight descriptions for their neural networks. To keep visiongraph small and easy to install, these assets are hosted externally on github. Visiongraph provides a system to directly download and cache these files.

Argparse

To support rapid prototyping many graph and estimator options are already provided to add to the argparse parser.

Logging

To enable logging for visiongraph imports please set the following environment variable:

# zsh / bash
export VISIONGRAPH_LOGLEVEL=INFO

# cmd
set VISIONGRAPH_LOGLEVEL=INFO

# powershell
$env:VISIONGRAPH_LOGLEVEL="INFO"

Roadmap

Next roadmap points:

• Async input and network model (run when ready)

About

Copyright (c) 2023 Florian Bruggisser

Included Libraries

Parts of these libraries are directly included and adapted to work with visiongraph.

• motpy - simple multi object tracking library (MIT License)
• motrackers - Multi-object trackers in Python (MIT License)
• OneEuroFilter-Numpy - (MIT License)

For more information about the dependencies have a look at the requirements.txt.