pyds_ext

python bindings for nvidia deepstream

pybind11 bindings to access Nvidia DeepStream metadata from Python - uses patched deepstream_python_apps. This package offers the same functionality as the default nvidia bindings, as well as some additional funcitonality:

• Allow to install with pip: wheels for x86-64 (GPU) and aarch64 (jetson) are available
• Patch pybind bindings to allow maskrcnn masks extraction

Previous version:

The following patches were included in the previous version (<2), and are no longer required because they are included in the vanilla bindings:

• Tracker meta info (NvDsPastFrame... classes)
• Detector and tracker bbox info (NvDsObjectMeta.[tracker/detector]_bbox_info... attrs) from Python.
• Analytics Metadata (NvDsAnalyticsFrameMeta and NvDsAnalyticsObjInfo) from Python.

Installation

Prerequisites

1. python3.8

2. Jetson (Xavier only tested) or Nvidia GPU

3. Deepstream>=6 (6.1 only tested)

4. [Option A] pep-517 compatible pip:

   pip install "pip>=10"
   

5. [Option B] Only necessary for old pip<10:

   • pybind11:
     • [Option B.1] You might try simply pip install pybind11.
     • [Option B.2] The recommended way is to build it from source

Install package

pip install --upgrade pip>=10
pip install pyds_ext

Usage

This meta-package provides a single package:

Standard pyds

1. pyds: Patched pyds from /opt/nvidia/deepstream/deepstream/lib See oficial documentation here

Maskrcnn extraction

Use the following as a reference to extract the np.ndarray of the mask:

import pyds

def osd_sink_pad_buffer_probe(pad, info, u_data):

    # ... code to acquire frame_meta
    l_obj = frame_meta.obj_meta_list
    while l_obj is not None:
        try:
            obj_meta = pyds_bbox_meta.NvDsObjectMeta.cast(l_obj.data)
        except StopIteration:
            break
        mask_array = obj_meta.mask_params.data
        print(f"mask_array: {mask_array}")
        try: 
            l_obj = l_obj.next
        except StopIteration:
            break
        ...

To convert the mask to an uint8 bidimensional matrix, see this code.

For this and other utilities, consider using the pythia package, which builds on this and adds pythonic iteration and common deepstream boilerplate.

References

• mask patch