vidgear 0.1.9

High-performance cross-platform Video Processing Python framework powerpacked with unique trailblazing features.

Releases   |   Gears   |   Documentation   |   Installation   |   License

 

VidGear is a high-performance Video Processing python library that provides an easy-to-use, highly extensible, Multi-Threaded + Asyncio wrapper around many state-of-the-art specialized libraries like OpenCV, FFmpeg, ZeroMQ, picamera, starlette, pafy and python-mss at its backend, and enable us to flexibly exploit their internal parameters and methods, while silently delivering robust error-handling and unparalleled real-time performance.

VidGear primarily focuses on simplicity, and thereby lets programmers and software developers to easily integrate and perform Complex Video Processing Tasks, in just a few lines of code.

 

The following functional block diagram clearly depicts the generalized functioning of VidGear APIs:

 

Table of Contents

• TL;DR
• Getting Started
• Gears: What are these?
  • CamGear
  • PiGear
  • VideoGear
  • ScreenGear
  • WriteGear
  • StreamGear
  • NetGear
  • WebGear
  • NetGear_Async
• Documentation
• Community Channel
• Contributions & Support
  • Support
  • Contributors
• Citation
• Copyright

 

 

TL;DR

What is vidgear?

"VidGear is a High-Performance Framework that provides an all-in-one complete Video Processing solution for building real-time applications in python."

What does it do?

"VidGear can read, write, process, send & receive video frames from/to various devices in real-time."

What is its purpose?

"Built with simplicity in mind, VidGear lets programmers and software developers to easily integrate and perform complex Video Processing tasks in their existing or new applications, in just a few lines of code. Beneficial for both, if you're new to programming with Python language or already a pro at it."

 

 

Getting Started

If this is your first time using VidGear, head straight to the Installation >> to install VidGear.

Once you have VidGear installed, checkout its well-documented Gears >>

Also, if you're already familar with OpenCV library, then see Switching from OpenCV >>

Or, if you're just getting started with OpenCV with Python, then see here >>

 

 

Gears: What are these?

VidGear is built with multiple Gears (APIs), each with some unique functionality.

Each of these APIs is exclusively designed to handle/control different device-specific video streams, network streams, and media encoders. These APIs provide an easy-to-use, highly extensible, multi-threaded and asyncio layer above state-of-the-art libraries under the hood to exploit their internal parameters and methods flexibly while providing robust error-handling and unparalleled performance.

These Gears can be classified as follows:

A. VideoCapture Gears:

• CamGear: Multi-threaded API targeting various IP-USB-Cameras/Network-Streams/YouTube-Video-URLs.
• PiGear: Multi-threaded API targeting various Raspberry Pi Camera Modules.
• ScreenGear: Multi-threaded ultra-fast Screencasting.
• VideoGear: Common API with internal Video Stabilizer wrapper.

B. VideoWriter Gears:

• WriteGear: Handles Flexible Lossless Video Encoding and Compression.

C. Streaming Gears:

• StreamGear: Handles Ultra-Low Latency, High-Quality, Dynamic & Adaptive Streaming Formats.

D. Network Gears:

• NetGear: Handles high-performance video-frames & data transfer between interconnecting systems over the network.

• Asynchronous I/O Network Gears:

  • WebGear: ASGI Video Server that can send live video-frames to any web browser on the network.
  • NetGear_Async: Immensely Memory-efficient Asyncio video-frames network messaging framework.

 

 

CamGear

CamGear can grab ultra-fast frames from diverse range of devices/streams, which includes almost any IP/USB Cameras, multimedia video file format (upto 4k tested), various network stream protocols such as http(s), rtp, rstp, rtmp, mms, etc., plus support for live Gstreamer's stream pipeline and YouTube video/live-streams URLs.

CamGear provides a flexible, high-level multi-threaded wrapper around OpenCV's VideoCapture class with access almost all of its available parameters and also employs pafy python APIs for live YouTube streaming. Furthermore, CamGear relies exclusively on Threaded Queue mode for ultra-fast, error-free and synchronized frame handling.

CamGear API Guide:

>>> Usage Guide

 

 

VideoGear

VideoGear API provides a special internal wrapper around VidGear's exclusive Video Stabilizer class.

VideoGear also act as a Common API, that provides an internal access to both CamGear and PiGear APIs and their parameters, with a special enablePiCamera boolean flag.

VideoGear is basically ideal when you need to switch to different video sources without changing your code much. Also, it enables easy stabilization for various video-streams (real-time or not) with minimum effort and using way fewer lines of code.

Below is a snapshot of a VideoGear Stabilizer in action (See its detailed usage here):

_{Original Video Courtesy @SIGGRAPH2013}

Code to generate above result:

# import required libraries
from vidgear.gears import VideoGear
import numpy as np
import cv2

# open any valid video stream with stabilization enabled(`stabilize = True`)
stream_stab = VideoGear(source = "test.mp4", stabilize = True).start()

# open same stream without stabilization for comparison
stream_org = VideoGear(source = "test.mp4").start()

# loop over
while True:

    # read stabilized frames
    frame_stab = stream_stab.read()

    # check for stabilized frame if Nonetype
    if frame_stab is None:
        break

    # read un-stabilized frame
    frame_org = stream_org.read()

    # concatenate both frames
    output_frame = np.concatenate((frame_org, frame_stab), axis=1)

    # put text over concatenated frame
    cv2.putText(
        output_frame, "Before", (10, output_frame.shape[0] - 10), cv2.FONT_HERSHEY_SIMPLEX,
        0.6, (0, 255, 0), 2,
    )
    cv2.putText(
        output_frame, "After", (output_frame.shape[1] // 2 + 10, output_frame.shape[0] - 10),
        cv2.FONT_HERSHEY_SIMPLEX,
        0.6, (0, 255, 0), 2,
    )

    # Show output window
    cv2.imshow("Stabilized Frame", output_frame)

    # check for 'q' key if pressed
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break

# close output window
cv2.destroyAllWindows()

# safely close both video streams
stream_org.stop()
stream_stab.stop()

VideoGear API Guide:

>>> Usage Guide

 

 

PiGear

PiGear is similar to CamGear but made to support various Raspberry Pi Camera Modules (such as OmniVision OV5647 Camera Module and Sony IMX219 Camera Module).

PiGear provides a flexible multi-threaded wrapper around complete picamera python library, and also provides us the ability to exploit almost all of its parameters like brightness, saturation, sensor_mode, iso, exposure, etc. effortlessly. Furthermore, PiGear supports multiple camera modules, such as in case of Raspberry Pi Compute module IO boards.

Best of all, PiGear provides excellent error-handling with features like a Threaded Internal Timer - that keeps active track of any frozen-threads/hardware-failures robustly, and exit safely if it does occurs, i.e. If you're running PiGear API in your script, and someone accidentally pulls Camera module cable out, instead of going into possible kernel panic, PiGear will exit safely to save resources.

Code to open picamera stream with variable parameters in PiGear API:

# import required libraries
from vidgear.gears import PiGear
import cv2

# add various Picamera tweak parameters to dictionary
options = {"hflip": True, "exposure_mode": "auto", "iso": 800, "exposure_compensation": 15, "awb_mode": "horizon", "sensor_mode": 0}

# open pi video stream with defined parameters
stream = PiGear(resolution = (640, 480), framerate = 60, logging = True, **options).start() 

# loop over
while True:

    # read frames from stream
    frame = stream.read()

    # check for frame if Nonetype
    if frame is None:
        break


    # {do something with the frame here}


    # Show output window
    cv2.imshow("Output Frame", frame)

    # check for 'q' key if pressed
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break

# close output window
cv2.destroyAllWindows()

# safely close video stream
stream.stop()

PiGear API Guide:

>>> Usage Guide

 

 

ScreenGear

ScreenGear is designed exclusively for ultra-fast Screencasting, that means it can grab frames from your monitor in real-time, either by define an area on the computer screen, or full-screen, at the expense of inconsiderable latency. ScreenGear also seamlessly support frame capturing from multiple monitors.

ScreenGear implements a multi-threaded wrapper around python-mss python library API and also supports a easy and flexible direct internal parameter manipulation.

Below is a snapshot of a ScreenGear API in action:

Code to generate the above results:

# import required libraries
from vidgear.gears import ScreenGear
import cv2

# open video stream with default parameters
stream = ScreenGear().start()

# loop over
while True:

    # read frames from stream
    frame = stream.read()

    # check for frame if Nonetype
    if frame is None:
        break


    # {do something with the frame here}


    # Show output window
    cv2.imshow("Output Frame", frame)

    # check for 'q' key if pressed
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break

# close output window
cv2.destroyAllWindows()

# safely close video stream
stream.stop()

ScreenGear API Guide:

>>> Usage Guide

 

 

WriteGear

WriteGear handles various powerful Writer Tools that provide us the freedom to do almost anything imagine with multimedia files.

WriteGear API provides a complete, flexible and robust wrapper around FFmpeg, a leading multimedia framework. With WriteGear, we can process real-time frames into a lossless compressed video-file with any suitable specification in just few easy lines of codes. These specifications include setting video/audio properties such as bitrate, codec, framerate, resolution, subtitles, etc., and also performing complex tasks such as multiplexing video with audio in real-time (see this doc), while handling all errors robustly.

Best of all, WriteGear grants the complete freedom to play with any FFmpeg parameter with its exclusive Custom Commands function (see this doc), without relying on any Third-party library.

In addition to this, WriteGear also provides flexible access to OpenCV's VideoWriter API which provides some basic tools for video frames encoding but without compression.

WriteGear primarily operates in the following two modes:

• Compression Mode: In this mode, WriteGear utilizes powerful FFmpeg inbuilt encoders to encode lossless multimedia files. This mode provides us the ability to exploit almost any parameter available within FFmpeg, effortlessly and flexibly, and while doing that it robustly handles all errors/warnings quietly. You can find more about this mode here >>

• Non-Compression Mode: In this mode, WriteGear utilizes basic OpenCV's inbuilt VideoWriter API tools. This mode also supports all parameters manipulation available within VideoWriter API, but it lacks the ability to manipulate encoding parameters and other important features like video compression, audio encoding, etc. You can learn about this mode here >>

WriteGear API Guide:

>>> Usage Guide

 

 

StreamGear

StreamGear automates transcoding workflow for generating Ultra-Low Latency, High-Quality, Dynamic & Adaptive Streaming Formats (such as MPEG-DASH) in just few lines of python code.

StreamGear provides a standalone, highly extensible and flexible wrapper around FFmpeg - a leading multimedia framework, for generating chunked-encoded media segments of the content.

SteamGear API automatically transcodes source videos/audio files & real-time frames, and breaks them into a sequence of multiple smaller chunks/segments (typically 2-4 seconds in length) at different quality levels (i.e. different bitrates or spatial resolutions). It also creates a Manifest file (such as MPD in-case of DASH) that describes these segment information (timing, URL, media characteristics like video resolution and bit rates), and is provided to the client prior to the streaming session. Thereby, segments are served on a web server and can be downloaded through HTTP standard compliant GET requests. This makes it possible to stream videos at different quality levels, and to switch in the middle of a video from one quality level to another one – if bandwidth permits – on a per segment basis.

SteamGear currently only supports MPEG-DASH (Dynamic Adaptive Streaming over HTTP, ISO/IEC 23009-1) , but other adaptive streaming technologies such as Apple HLS, Microsoft Smooth Streaming, will be added soon.

StreamGear primarily works in two independent modes for transcoding which serves different purposes:

• Single-Source Mode: In this mode, StreamGear transcodes entire video/audio file (as opposed to frames by frame) into a sequence of multiple smaller chunks/segments for streaming. This mode works exceptionally well, when you're transcoding lossless long-duration videos(with audio) for streaming and required no extra efforts or interruptions. But on the downside, the provided source cannot be changed or manipulated before sending onto FFmpeg Pipeline for processing. This mode can be easily activated by assigning suitable video path as input to -video_source attribute, during StreamGear initialization. Learn more about this mode here >>

• Real-time Frames Mode: When no valid input is received on -video_source attribute, StreamGear API activates this mode where it directly transcodes video-frames (as opposed to a entire file), into a sequence of multiple smaller chunks/segments for streaming. In this mode, StreamGear supports real-time numpy.ndarray frames, and process them over FFmpeg pipeline. But on the downside, audio has to added manually (as separate source) for streams. Learn more about this mode here >>

StreamGear API Guide:

>>> Usage Guide

 

 

NetGear

NetGear is exclusively designed to transfer video frames synchronously and asynchronously between interconnecting systems over the network in real-time.

NetGear implements a high-level wrapper around PyZmQ python library that contains python bindings for ZeroMQ - a high-performance asynchronous distributed messaging library that provides a message queue, but unlike message-oriented middleware, its system can run without a dedicated message broker.

NetGear provides seamless support for Bidirectional data transmission between receiver(client) and sender(server) through bi-directional synchronous messaging patterns such as zmq.PAIR (ZMQ Pair Pattern) & zmq.REQ/zmq.REP (ZMQ Request/Reply Pattern).

NetGear also supports real-time Frame Compression capabilities for optimizing performance while sending the frames directly over the network, by encoding the frame before sending it and decoding it on the client's end automatically in real-time.

For security, NetGear implements easy access to ZeroMQ's powerful, smart & secure Security Layers, that enables Strong encryption on data, and unbreakable authentication between the Server and the Client with the help of custom certificates/keys and brings easy, standardized privacy and authentication for distributed systems over the network.

Best of all, NetGear can robustly handle Multiple Server-Systems and Multiple Client-Systems and at once, thereby providing access to seamless Live Streaming of the multiple device in a network at the same time.

NetGear as of now seamlessly supports three ZeroMQ messaging patterns:

• zmq.PAIR (ZMQ Pair Pattern)
• zmq.REQ/zmq.REP (ZMQ Request/Reply Pattern)
• zmq.PUB/zmq.SUB (ZMQ Publish/Subscribe Pattern)

Whereas supported protocol are: tcp and ipc.

NetGear API Guide:

>>> Usage Guide

 

 

WebGear

WebGear is a powerful ASGI Video-streamer API, that is built upon Starlette - a lightweight ASGI framework/toolkit, which is ideal for building high-performance asyncio services.

WebGear API provides a highly extensible and flexible asyncio wrapper around Starlette ASGI application, and provides easy access to its complete framework. Thereby, WebGear API can flexibly interact with the Starlette's ecosystem of shared middleware and mountable applications, and its various Response classes, Routing tables, Static Files, Templating engine(with Jinja2), etc.

In layman's terms, WebGear can acts as powerful Video Streaming Server that transfers live video-frames to any web browser on a network. It addition to this, WebGear API also provides a special internal wrapper around VideoGear, which itself provides internal access to both CamGear and PiGear APIs thereby granting it exclusive power for streaming frames incoming from any device/source, such as streaming Stabilization enabled Video in real-time.

Below is a snapshot of a WebGear Video Server in action on the Mozilla Firefox browser:

_{WebGear Video Server at http://localhost:8000/ address.}

Code to generate the above result:

# import required libraries
import uvicorn
from vidgear.gears.asyncio import WebGear

#various performance tweaks
options = {"frame_size_reduction": 40, "frame_jpeg_quality": 80, "frame_jpeg_optimize": True, "frame_jpeg_progressive": False}

#initialize WebGear app  
web = WebGear(source = "foo.mp4", logging = True, **options)

#run this app on Uvicorn server at address http://localhost:8000/
uvicorn.run(web(), host='localhost', port=8000)

#close app safely
web.shutdown()

WebGear API Guide:

>>> Usage Guide

 

 

NetGear_Async

NetGear_Async can generate double performance as compared to NetGear API at about 1/3rd of memory consumption, and also provide complete server-client handling with various options to use variable protocols/patterns similar to NetGear, but it doesn't support any NetGear's Exclusive Modes yet.

NetGear_Async is an asyncio videoframe messaging framework, built on zmq.asyncio, and powered by high-performance asyncio event loop called uvloop to achieve unmatchable high-speed and lag-free video streaming over the network with minimal resource constraints. Basically, this API is able to transfer thousands of frames in just a few seconds without causing any significant load on your system.

NetGear_Async provides complete server-client handling and options to use variable protocols/patterns similar to NetGear API but doesn't support any NetGear Exclusive modes yet. Furthermore, NetGear_Async allows us to define our own custom Server Source to manipulate frames easily before sending them across the network(see this doc example).

NetGear_Async as of now supports all four ZeroMQ messaging patterns:

• zmq.PAIR (ZMQ Pair Pattern)
• zmq.REQ/zmq.REP (ZMQ Request/Reply Pattern)
• zmq.PUB/zmq.SUB (ZMQ Publish/Subscribe Pattern)
• zmq.PUSH/zmq.PULL (ZMQ Push/Pull Pattern)

Whereas supported protocol are: tcp and ipc.

NetGear_Async API Guide:

>>> Usage Guide

 

 

Documentation

The complete documentation for all VidGear APIs can be found in the link below:

• Documentation - English

 

 

Contributions & Support

Contributions are welcome, We'd love your contribution to VidGear in order to fix bugs or to implement new features!

Please see our Contribution Guidelines for more details.

Support

VidGear is free, but rely on your support.

Sending a donation using link below is extremely helpful in keeping VidGear development alive:

Contributors

 

 

Community Channel

If you've come up with some new idea, or looking for the fastest way troubleshoot your problems, then join our Gitter community channel >>

 

 

Citation

Here is a Bibtex entry you can use to cite this project in a publication:

@misc{vidgear,
    author = {Abhishek Thakur},
    title = {vidgear},
    howpublished = {\url{https://github.com/abhiTronix/vidgear}},
    year = {2019-2020}
  }

 

 

Copyright

Copyright (c) abhiTronix 2019-2020

This library is released under the Apache 2.0 License.