Skip to main content

A Python API For wiki.sipeed.com/m3axpi Pipeline

Project description

what this ?

MIT PyPI version

This project is a Python implementation of ax-pipeline.

  • pip3 install ax-pipeline-api -U

Based on AX620A Debian11 system. Docs at wiki.sipeed.com/m3axpi

Python apis for pybind11 and ctypes are implemented here, but please do not mix them as there is only one screen and one camera.

  • pybind11 has a friendlier usage

  • ctypes have a more stable effect

new pybind11 code (1.0.9+)

It need update debian11(202202+) for pillow ImageFont(freetype). (sudo apt install libxft-dev && pip3 uninstall pillow -y && pip3 install pillow --no-cache-dir -U)

https://user-images.githubusercontent.com/32978053/218361047-6a72b294-cd66-4f0e-8bed-152a6661d458.mp4

The new API is still in an unstable state, so don't mind if you encounter strange bugs.

import m3axpi

# m3axpi.camera(SysCase=0) # switch os04a10
# m3axpi.camera(SysCase=2) # default gc4653
# m3axpi.load("/home/config/yolov8.json")

from PIL import Image, ImageDraw, ImageFont

lcd_width, lcd_height, lcd_channel = 854, 480, 4

fnt = ImageFont.truetype("/home/res/sans.ttf", 20)
img = Image.new('RGBA', (lcd_width, lcd_height), (255,0,0,200))
ui = ImageDraw.ImageDraw(img)
ui.rectangle((20, 20, lcd_width-20, lcd_height-20), fill=(0,0,0,0), outline=(0,0,255,100), width=20)

logo = Image.open("/home/res/logo.png")
img.paste(logo, box=(lcd_width-logo.size[0], lcd_height-logo.size[1]), mask=None)

while True:
    rgba = img.copy()

    tmp = m3axpi.capture()
    rgb = Image.frombuffer("RGB", (tmp[1], tmp[0]), tmp[3])
    rgba.paste(rgb, box=(0, 0), mask=None) ## camera 320x180 paste 854x480

    res = m3axpi.forward()
    if 'nObjSize' in res:
        ui = ImageDraw.ImageDraw(rgba)
        ui.text((0, 0), "fps:%02d" % (res['niFps']), font=fnt)
        for obj in res['mObjects']:
            x, y, w, h = int(obj['bbox'][0]*lcd_width), int(obj['bbox'][1]*lcd_height), int(obj['bbox'][2]*lcd_width), int(obj['bbox'][3]*lcd_height)
            ui.rectangle((x,y,x+w,y+h), fill=(255,0,0,100), outline=(255,0,0,255))
            ui.text((x, y), "%s:%02d" % (obj['objname'], obj['prob']*100), font=fnt)
            rgba.paste(logo, box=(x+w-logo.size[1], y+h-logo.size[1]), mask=None)

    m3axpi.display([lcd_height, lcd_width, lcd_channel, rgba.tobytes()])

old ctypes code

yolov5s

https://user-images.githubusercontent.com/32978053/204093040-179e35d0-8bfa-4626-b4cc-46f3f148eb71.mp4

import time
from ax import pipeline

pipeline.load([
    'libsample_vin_ivps_joint_vo_sipy.so',
    '-p', '/home/config/yolov5s.json',
    '-c', '2',
])

while pipeline.work():
    time.sleep(0.001)
    tmp = pipeline.result()
    if tmp and tmp['nObjSize']:
        for i in tmp['mObjects']:
            x, y, w, h = i['bbox']['x'], i['bbox']['y'], i['bbox']['w'], i['bbox']['h']
            objname, objprob = i['objname'], i['prob']
            print(objname, objprob, x, y, w, h)
        # if tmp['nObjSize'] > 10: # try exit
        #     pipeline.drop()
pipeline.drop()

yolov5s_face

https://user-images.githubusercontent.com/32978053/204444795-635299e9-89f1-4c76-9536-1c6dd9915b72.mp4

import time
from ax import pipeline

pipeline.load([
    'libsample_vin_ivps_joint_vo_sipy.so',
    '-p', '/home/config/yolov5s_face.json',
    '-c', '2',
])

while pipeline.work():
    time.sleep(0.001)
    tmp = pipeline.result()
    if tmp and tmp['nObjSize']:
        for i in tmp['mObjects']:
            print(i)
        # if tmp['nObjSize'] > 10: # try exit
        #     pipeline.drop()
pipeline.drop()

other demo

https://user-images.githubusercontent.com/32978053/204150061-779c2443-5416-4a5e-a10f-e684a035510d.mp4

https://user-images.githubusercontent.com/32978053/204150065-6977de65-423a-4895-a970-59ef914f9184.mp4

more ?

change sensor

  • camera os04a10 is '-c', '0', and gc4653 is '-c', '2',.
    pipeline.load([
        'libsample_vin_ivps_joint_vo_sipy.so',
        '-p', '/home/config/ax_pose.json',
        '-c', '0',
    ])

change libxxx*.so

    pipeline.load([
        'libsample_vin_ivps_joint_venc_rtsp_vo_sipy.so',
        '-p', '/home/config/yolov5_seg.json',
        '-c', '0',
    ])
  • Package with many inputs and outputs
libsample_h264_ivps_joint_vo_sipy.so                # input h264 video to ivps joint output screen vo
libsample_v4l2_user_ivps_joint_vo_sipy.so           # input v4l2 /dev/videoX to ivps joint output screen vo
libsample_rtsp_ivps_joint_rtsp_vo_sipy.so           # input video from rtsp to ivps joint output rtsp and screen vo
libsample_vin_ivps_joint_vo_sipy.so                 # input mipi sensor to ivps joint output screen vo
libsample_vin_ivps_joint_venc_rtsp_sipy.so          # input mipi sensor to ivps joint output rtsp
libsample_vin_ivps_joint_venc_rtsp_vo_sipy.so       # input mipi sensor to ivps joint output rtsp and screen vo
libsample_vin_ivps_joint_vo_h265_sipy.so            # input mipi sensor to ivps joint output screen vo and save h265 video file
libsample_multi_rtsp_ivps_joint_multi_rtsp_sipy.so  # input multi rtsp video to ivps joint output multi rtsp video
libsample_rtsp_ivps_joint_rtsp_sipy.so              # input video from rtsp to ivps joint output rtsp
libsample_rtsp_ivps_joint_rtsp_vo_sipy.so           # input video from rtsp to ivps joint output rtsp and screen vo
libsample_rtsp_ivps_joint_vo_sipy.so                # input video from rtsp to ivps joint output screen vo

change ai model

pipeline.load([
    'libsample_vin_ivps_joint_vo_sipy.so',
    '-p', '/home/config/yolov5s_face.json',
    '-c', '0',
])
  • In-system models on /home/config.
ax_bvc_det.json		    hrnet_pose_yolov8.json	    yolov5s_face_recognition.json
ax_person_det.json	    license_plate_recognition.json  yolov5s_license_plate.json
ax_pose.json		    nanodet.json		    yolov6.json
ax_pose_yolov5s.json	    palm_hand_detection.json	    yolov7.json
ax_pose_yolov8.json	    pp_human_seg.json		    yolov7_face.json
crowdcount.json		    scrfd.json			    yolov7_palm_hand.json
hand_pose.json		    yolo_fastbody.json		    yolov8.json
hand_pose_yolov7_palm.json  yolopv2.json		    yolov8_seg.json
hrnet_animal_pose.json	    yolov5_seg.json		    yolox.json
hrnet_pose.json		    yolov5s.json
hrnet_pose_ax_det.json	    yolov5s_face.json

pypi

  • python3 setup.py sdist
  • python3 setup.py build && cp build/lib*/m3axpi.*.so m3axpi.so
  • rm -rf build && pip3 uninstall ax-pipeline-api -y && python3 setup.py build && pip3 install .

pip3 install twine

  • twine upload dist/* --verbose

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ax-pipeline-api-1.1.2.tar.gz (68.4 MB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page