matrice_streaming — Streaming Gateway KT Docs
This folder is the knowledge-transfer reference for matrice_streaming, the
Streaming Gateway (SG). It is written to be walked through in a live KT session:
a presenter takes each chapter top to bottom with a new engineer. Every chapter is
self-contained. Chapters 01–06 follow the same shape — an "In one minute" summary, a
code-free high-level walkthrough, a "Going deeper" section with file/line anchors into
the source, and a configuration table; 07 is a flat reference and 08 an appendix.
What the Streaming Gateway is
SG is the frame producer of the Matrice video pipeline. Given a set of cameras, it opens each stream (RTSP fronted by MediaMTX, or file/HTTP sources), decodes frames at a target FPS, and publishes every decoded frame into a per-camera shared-memory ring buffer that downstream processes — primarily the inference engine — attach to and read zero-copy. It runs hundreds of cameras per host and manages the camera set fully dynamically: cameras are added, updated, and removed at runtime without restarting the process, driven by Kafka events plus a periodic backend poll.
There are two decode backends, chosen at construction:
- NVDEC (GPU) — the production path. Demuxed compressed packets are decoded by NVIDIA's hardware decoder; frames stay on the GPU as NV12 surfaces and are handed to the inference engine through CUDA IPC, never touching host RAM. This path gets the depth in these docs (chapter 03).
- OpenCV (CPU) — a
cv2.VideoCapture-based fallback for hosts without a usable GPU decode stack. It is covered briefly in chapter 05.
SG does not run inference and does not force preprocessing: frames are published at native resolution and the inference engine owns resize/letterbox/color conversion.
Where SG sits
Cameras (RTSP via MediaMTX / file / HTTP)
│
▼
┌─────────────────────────────────────────────────┐
│ Streaming Gateway (this package) │
│ demux → decode (NVDEC on GPU in prod) → │
│ publish into per-camera ring buffers │
└─────────────────────────────────────────────────┘
│ /dev/shm DataBus ring buffers
│ (NVDEC: CUDA IPC handles to GPU-resident NV12 frames)
▼
Inference Engine ──► results ──► analytics / results-agg
▲
│ control plane
┌─────────────────────────────────────────────────┐
│ Matrice backend: REST API (camera list, Kafka │
│ connection info, status) + Kafka (camera │
│ add/remove/update events, stop commands, │
│ heartbeats, metrics) │
└─────────────────────────────────────────────────┘
Three external dependencies, and only three: the backend REST API (via the
matrice_common SDK's Session.rpc), Kafka, and shared memory via
matrice_common.stream (DataBus / CUDA IPC ring buffers). The matrice_common SDK
has its own KT doc set in the py_common repo; chapter 04 here explains the publish
boundary from SG's side so you do not need that set to follow these docs.
Session reading order
Read the chapters in order; each builds on the picture established by the previous one, but none requires flipping back.
| # | Chapter | One line |
|---|---|---|
| 01 | 01-architecture-overview.md | The big map: StreamingAction → StreamingGateway → one decode backend, the process/thread model, lifecycle states, and where every output goes. |
| 02 | 02-camera-management.md | How the camera set stays in sync at runtime: InstanceEventListener (Kafka + periodic poll), the dynamic camera manager, config diffing, and phantom-camera self-healing. |
| 03 | 03-nvdec-pipeline.md | The production decode path in depth: demuxers, codec handling, one worker process per GPU, the decoder pool, GPU placement, and the watchdog. |
| 04 | 04-publishing-and-matrice-common.md | The publish boundary: DataBus addresses, ring-buffer semantics, CUDA IPC handle sharing, frame counters, and what SG cleans up (or deliberately leaves) on stop. |
| 05 | 05-opencv-path.md | The CPU fallback, briefly: an async worker-process pool sharding cameras, JPEG/BGR publishing through the same DataBus addresses. |
| 06 | 06-metrics-and-heartbeat.md | Health reporting: per-camera metrics collection/aggregation, heartbeat payloads, and the Kafka topics the backend watches. |
| 07 | 07-configuration-and-ops.md | Environment variables and constructor parameters in one place, plus production symptoms → causes → knobs. |
| 08 | 08-appendix-other.md | Everything that is real but not on the main path: LocalDecoder, the frame-optimizer seam, camera tampering (Case 1), and other supporting utilities. |
| — | 11-motion-optimizer-test-results.md | Motion frame optimizer validation (unit, E2E, threshold tuning). |
| — | 14-camera-tampering-test-results.md | Blank-screen tampering detector validation on labelled clips + NVDEC E2E. |
Suggested emphasis for a session: 01 and 03 carry most of the weight; 02 and 04 are where the subtle bugs have historically lived; 05 is a skim; 06–08 are reference.
Public API in one glance
Everything external lives under matrice_streaming.streaming_gateway:
StreamingAction (the production entry point — turns an action_id into a running,
monitored, auto-restarting gateway), StreamingGateway (the orchestrator underneath,
used directly in tests or custom control loops), InstanceStreamingGatewayUtil /
StreamingGatewayUtil (backend API clients), InputStream (the per-camera config
dataclass), and InstanceEventListener (Kafka-driven camera reconciliation).
Chapter 01 walks the relationships between them.
Glossary
- Stream key — the gateway's internal handle for one camera stream; maps 1:1 to a
camera_idin the gateway's camera↔stream-key map. - Camera config /
InputStream— the per-camera configuration dataclass (source URL, target FPS, codec, camera ids/keys, location, topic).width=0/height=0means native resolution — SG does not resize by default. - Demuxer — the component that splits a container or transport (RTSP, MP4) into raw compressed video packets (H.264/H.265 NAL units) ready for the decoder.
- NVDEC — NVIDIA's fixed-function hardware video decoder; the production decode backend. Decoding costs essentially no CPU and no CUDA compute.
- NV12 — a YUV 4:2:0 pixel format (full-resolution Y plane plus interleaved UV at
half resolution); the native output of NVDEC. As a buffer it is
(H*1.5, W)uint8. - DataBus address — the name a producer publishes under and a consumer attaches
to,
{camera_id}__sg__frames, backed by a/dev/shm/databus__…file. Both decode backends publish frames under the same address scheme. - CUDA IPC — the CUDA mechanism for sharing a GPU memory buffer between processes without copying it to host RAM. On the NVDEC path the shared-memory ring buffer carries CUDA IPC handles; the actual NV12 pixels never leave the GPU.
- Phantom camera — a camera the gateway believes it is streaming but which has no
live frame ring buffer in
/dev/shm. The event listener detects phantoms and self-heals by re-adding the camera (chapter 02).
1 maintainer
Release files for matrice-streaming 0.18.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| matrice_streaming-0.18.0.tar.gz | 834.5 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| matrice_streaming-0.18.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 1.9 MB
Release files / matrice_streaming-0.18.0.tar.gz
| Download URL | matrice_streaming-0.18.0.tar.gz |
|---|---|
| Size | 834.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
a46332e7bd70792e972c362fa817e488a13aafa383052d033922c720f875ee0f
|
|
BLAKE2b-256 checksum How to use checksums |
ed27097affa5cf7a77ce5f6259aebb67bdfe52926267bee1a7bc87d5ce9b1334
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.14
|
Release files / matrice_streaming-0.18.0-py3-none-any.whl
| Download URL | matrice_streaming-0.18.0-py3-none-any.whl |
|---|---|
| Size | 1.1 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
29acefeacd6e8b3f1fad52960587f773a42540f0194c1c4ae2223bebf162b458
|
|
BLAKE2b-256 checksum How to use checksums |
993036a5ac96e857f480577eedc7b7a023525f3b0acec3d0086d8257044e4893
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.14
|
