sima-vision

Live YOLO26 on the MLA of a SiMa.ai Modalix DevKit 3.0.
Three apps, one pipeline, one command.

pip install sima-vision
sima-vision detect          # sample clip and model fetched for you, then run

Inference needs the board. Nothing else does. Checking a config and seeing exactly what the overlay will look like both run on your laptop, so you can set the whole thing up before you own any hardware.


Install	pip, and what each extra buys you
Quickstart	Without a board, then with one
The three tasks	What each does, and its own flags
Commands	Every subcommand in one table
Settings	Flags, Python keywords, `config.yaml`
Python API	The same three verbs, importable
Driving the board from your PC	`push`, `remote`, `pull`
Set up a new DevKit	One time, about two hours
Troubleshooting	Symptom to fix, in one table
How it works	The pipeline, and why it is shaped that way

Install

Python 3.10 or later. No compiler, and the only dependency is PyYAML.

pip install sima-vision

Where	Install	Why
On the DevKit	`~/pyneat/bin/pip install sima-vision`	Into the venv that has `pyneat`. See below
Your laptop	`pip install "sima-vision[preview]"`	Adds numpy and OpenCV so `preview` can draw
Contributing	`pip install -e ".[dev,preview]"`	Also ruff and pytest

[!IMPORTANT] On the DevKit, install into the pyneat venv. sima-cli sdk setup puts pyneat in a virtualenv of its own at ~/pyneat, and pip installs into whichever Python you ran pip with. Install anywhere else and a run stops at ModuleNotFoundError: pyneat.

If you already installed it elsewhere, it still works: the run looks for that venv and uses it, printing [pyneat] using pyneat from ~/pyneat when it does. Set SIMA_VISION_PYNEAT if yours is somewhere unusual. sima-vision doctor says which of these applies to you.

[!CAUTION] On the board, never let pip pull numpy 2.x. pyneat and every simaai-* package need numpy<2. sima-vision depends on neither numpy nor OpenCV precisely so that installing it cannot upgrade them, which is also why installing it into the pyneat venv is safe. If something already broke it: ~/pyneat/bin/pip install "numpy>=1.24,<2" "opencv-python>=4.7,<5"

Check what you have:

sima-vision doctor

Quickstart

Without a board

sima-vision preview --task segment -o blur.png    # draw the overlay your config makes
sima-vision init segment                          # write a documented config.yaml
sima-vision segment --validate                    # check it

preview runs no model. It draws synthetic detections through the real overlay code, so what you are judging is styling, not accuracy. Nothing here touches the network.

On the DevKit

~/pyneat/bin/pip install sima-vision      # the venv that has pyneat
sima-cli login                            # once, for the model packs
~/pyneat/bin/sima-vision detect           # that is the whole thing

No arguments. Each task has a default sample clip and model archive, and the first run puts both in ./assets/: the clip comes straight from a public GitHub release, the model through sima-cli, which holds your login. Every run after that reuses them.

Out comes detections.mp4 and a frames/ directory. Bring them back with sima-vision pull.

Your own clip or model, as a path or an https URL:

sima-vision detect --source my-clip.h264 --model my-model.tar.gz
sima-vision detect --source https://example.com/my-clip.h264

A URL is downloaded into assets/ once and reused. There is one assets/ for all three tasks, not one per task: they share the clips, and detect and fall share the model archive outright.

[!IMPORTANT] Video must be raw H.264, not .mp4. The board decodes H.264 in hardware, and containers hit a demuxer bug in Neat 0.3.0. Convert once, losslessly:
ffmpeg -i clip.mp4 -c:v copy -bsf:v h264_mp4toannexb -f h264 clip.h264
Renaming an .mp4 does not work and is caught at startup, with that command in the error. Leave source.fps, source.width and source.height at 0: the real geometry is read out of the stream's SPS.

New board? Bring-up is a one-time job of about two hours, mostly downloading. That is Set up a new DevKit. Nothing above needs it.

The three tasks

Task	What it does	Model	Output
`detect`	Boxes, class names and confidence on every frame	YOLO26 detect	`detections.mp4`
`segment`	Per-pixel masks, and a background blur that keeps the subject sharp	YOLO26 segment	`segmentation.mp4`
`fall`	Tracks people and emails when one of them goes down	YOLO26 detect	`falls.mp4`, `alerts/`

All three share one pipeline: the same source handling, Neat graph, sample decoding, drawing and sinks. A task supplies only what is genuinely its own.

detect · boxes, labels and confidence

The simplest thing that proves the whole chain works. No task-specific flags: everything it understands is in Settings.

sima-vision detect
sima-vision detect --conf 0.5 --frames 200
sima-vision detect --source rtsp://cam/live --source-type rtsp

Problem	Fix
No detections at all	Check `model.family` is `yolo26`, then lower `--conf`
Scores all near zero	`model.family` does not match the archive, so a raw-logit head is being decoded wrong

segment · masks, and a background blur

Needs a -seg model pack. A plain detect head carries no mask data, and the run says so rather than guessing.

sima-vision segment                                        # masks and the default blur
sima-vision segment --blur-strength 81                     # a stronger blur
sima-vision segment --blur-method pixelate                 # pixelate instead
sima-vision segment --keep-classes person                  # only people stay sharp
sima-vision segment --anonymise --keep-classes person      # the other way round
sima-vision segment --no-blur                              # masks, no background effect

Flag	What it does
`--blur` / `--no-blur`	Whether the background is treated at all
`--blur-method`	`gaussian`, `pixelate` or `none`
`--blur-strength PX`	Gaussian kernel width for a 1080p frame. Default 41
`--keep-classes`	Class names or ids that stay sharp. Default: everything detected
`--anonymise`	Blur the instances instead of the background
`--mask-threshold T`	Mask cut-off as a probability. Default 0.5; lower grows instances
`--no-masks`	Blur around plain boxes. Works with a detect head
`--minimal`	Pull frames and do nothing else. See below

--minimal is the diagnostic. If a run that stalls part-way through a clip completes with --minimal, the cause is how much work the app does per frame. If it stalls at the same frame, the cause is the graph.

Problem	Fix
`model.family ... is a detect head`	Point `--model` at a `-seg` pack, or pass `--no-masks`
Whole frame blurred	Nothing was detected. Lower `--conf`
Mask edges jagged	Raise `blur.feather`, then `segmentation.blur_mask`
Masks in the wrong place	Pin `segmentation.space` to `net` or `box` instead of `source`
Too slow at 1080p	`blur.downscale: 4` is the biggest win, then `--save-every 30`

fall · tracking, a fall state machine, SMTP alerts

Tracks people across frames and watches three signals from the plain bounding box: the box turning wide-and-short, its height collapsing, and how fast its centre is dropping. A fall has to hold for --confirm seconds before an alert fires, which is what keeps someone crouching from setting it off.

sima-vision fall                                       # track and judge, no email
sima-vision fall --no-fall                             # track only, to tune tracking first
sima-vision fall --confirm 0.8                         # confirm faster
sima-vision fall --alert-to ops@example.com --send     # actually email
sima-vision fall --test-alert                          # one fake alert now, needs no board

Flag	What it does
`--classes`	Class names or ids that can fall. Default `person`
`--confirm S`	How long a fall signal must hold. Default 1.5
`--no-fall`	Track without judging
`--alert-to EMAIL`	Recipients. Implies `--alerts`
`--alert-from EMAIL`	From address
`--alerts`	Enable alerts, still a dry run
`--send`	Actually connect to the SMTP server
`--smtp-host` / `--smtp-port` / `--smtp-user`	Server, 587 for STARTTLS or 465 for SSL, and the login
`--site NAME`	Human name for this camera, used in the subject
`--test-alert`	Send one fake alert and exit

[!WARNING] Alerts stay a dry run until --send. The SMTP password is only ever read from $FALL_ALERT_SMTP_PASSWORD, never from a config file, because config files get committed.

export FALL_ALERT_SMTP_PASSWORD='your-app-password'    # macOS, Linux, the DevKit

$env:FALL_ALERT_SMTP_PASSWORD = "your-app-password"    # PowerShell

Gmail needs an app password, not your account password. --test-alert proves the whole path synchronously and needs no board.

Problem	Fix
`falls=0` on footage that has one	Lower `--confirm` and `fall.aspect_ratio`; first check the person is tracked at all with `--no-fall`
Alerts fire constantly	Raise `--confirm` first, then `alerts.cooldown_seconds`
Track ids change every few frames	Lower `tracking.iou_threshold`, raise `tracking.max_age`
Gmail rejects the login	App password, not the account password. `--test-alert` prints the SMTP error verbatim

Commands

Command	Board?	What it does
`sima-vision detect`	yes	Run detection on the MLA
`sima-vision segment`	yes	Run segmentation, with the optional blur
`sima-vision fall`	yes	Run fall detection, with SMTP alerts
`sima-vision preview`	no	Draw the overlay your config produces, to a PNG
`sima-vision init`	no	Write a documented `config.yaml` here
`sima-vision doctor`	no	What is installed, and what it lets you do
`sima-vision fetch`	no	Download the sample clips up front
`sima-vision push`	no	Copy files to the DevKit
`sima-vision pull`	no	Copy results back
`sima-vision remote`	no	Run a task on the DevKit over SSH

Add --validate to any task to parse and check a config, print what it resolved to, and exit. It loads neither pyneat nor the model, so it runs anywhere:

sima-vision segment --conf 0.5 --blur-strength 81 --validate

config OK: config.yaml
  model: assets/models/yolo26m-seg-bf16-mla_tess-b1.tar.gz
  family=yolo26-seg -> BoxDecodeType.YoloV26Seg
  source: type=video uri=assets/videos/people-walking-outside-mall.h264
  decode: conf=0.5 iou=0.6 max_det=50
  segmentation: masks=on source=auto space=auto threshold=0.5
  blur: background | method=gaussian kernel=81 sigma=auto down=2 feather=9
  output: video=segmentation.mp4 stills=frames/ every=10

sima-vision <command> --help lists every flag.

Settings

Three layers, each beating the one above it:

built-in defaults   ->   config.yaml   ->   flags / Python keywords

So everything is optional. With no file and no flags you get this task's sample clip and model. For a setup you keep:

sima-vision init detect     # a documented config.yaml, right here
sima-vision detect          # picks up ./config.yaml on its own

Every setting has a flag and a Python keyword under the same name. Both write the same config key, and both go through the same validation.

What people actually change

I want	Flag	Python	Config key
Fewer spurious boxes	`--conf 0.5`	`conf=0.5`	`decode.score_threshold`
To catch more, noisily	`--conf 0.15`	`conf=0.15`	`decode.score_threshold`
A short test run	`--frames 100`	`frames=100`	`runtime.frames`
No video file	`--no-video`	`video=False`	`output.video.enable`
No stills	`--no-save`	`save=False`	`output.save.enable`
Fewer stills	`--save-every 30`	`save_every=30`	`output.save.every`
To see where the time goes	`--profile`	`profile=True`	`runtime.profile`
A live Insight view	`--insight`	`insight=True`	`output.insight.enable`
A stronger blur	`--blur-strength 81`	`blur_strength=81`	`blur.kernel`
A pixelated background	`--blur-method pixelate`	`blur_method="pixelate"`	`blur.method`
Only people kept sharp	`--keep-classes person`	`keep_classes=["person"]`	`blur.keep_classes`
People blurred, scene sharp	`--anonymise`	`anonymise=True`	`blur.invert`
Masks, but no blur	`--no-blur`	`blur=False`	`blur.enable`
To track something else	`--classes forklift`	`classes=["forklift"]`	`tracking.classes`
Falls confirmed faster	`--confirm 0.8`	`confirm=0.8`	`fall.confirm_seconds`
An email on a fall	`--alert-to me@x.com --send`	`alert_to=[...], send=True`	`alerts.*`

Shared flags

Flag	Config key	What it does
`--source`, `-s`	`source.uri`	File, `https` URL, RTSP URL, or nothing for the sample clip
`--source-type`	`source.type`	`video`, `rtsp` or `usb`
`--fps` / `--width` / `--height`	`source.*`	Leave at 0. Read from the stream
`--model`, `-m`	`model.path`	Compiled model archive, or an `https` URL to one
`--labels`	`model.labels`	Class names. Defaults to the packaged COCO list
`--family`	`model.family`	Detection head. Must match the model
`--conf` / `--iou` / `--max-det`	`decode.*`	Confidence, NMS IoU, top-K
`--frames`, `-n`	`runtime.frames`	Stop after N frames
`--timeout` / `--queue-depth`	`runtime.*`	Pull timeout, and how far ahead of the sinks to run
`--profile`	`runtime.profile`	Per-stage timings
`--video-path` / `--no-video`	`output.video.*`	The annotated recording
`--save-dir` / `--save-every` / `--no-save`	`output.save.*`	Annotated stills
`--no-hud`	`output.video.hud`	Leave the frame-rate badge off
`--insight` / `--insight-host`	`output.insight.*`	The live Neat Insight feed
`--config`, `-c` / `--no-config`		Which config file, or none
`--validate`		Check and print, then exit

Cameras and streams

sima-vision detect --source-type usb                            # the DevKit camera
sima-vision detect --source rtsp://cam/live --source-type rtsp

For a live source, raise --queue-depth and leave runtime.overflow_policy on auto. For a file, auto resolves to block, which keeps every frame so the recording matches the input length.

Environment

Variable	What it does
`SIMA_VISION_ASSETS`	Where clips and models are downloaded. Default `./assets`
`SIMA_VISION_PYNEAT`	The `pyneat` virtualenv, when it is not at `~/pyneat`
`SIMA_VISION_DEVKIT`	The board, as `user@address`, so `push`, `pull` and `remote` stop asking
`FALL_ALERT_SMTP_PASSWORD`	The only place the SMTP password is ever read from

Python API

from sima_vision import run, preview, validate

# No board: draw the overlay a setting produces, and write a PNG.
preview("segment", out="blur.png", blur_strength=81, keep_classes=["person"])

# No board: resolve and check a config, then look at what it became.
cfg = validate("detect", conf=0.5, max_det=20)
print(cfg.score_threshold, cfg.video_path)

# On the DevKit: run it. Every argument is optional, exactly like the CLI.
run("detect")
run("detect", source="clip.h264", model="det.tar.gz", conf=0.5, frames=200)

Every keyword is derived from the CLI's own flags, so --blur-strength 81 and blur_strength=81 cannot drift apart. Anything the keywords do not cover is still reachable by its config path:

run("segment", **{"runtime.output_buffers": 2})

Driving the board from your PC

Three wrappers around ssh and scp, so the awkward parts stop being yours.

sima-vision push my-clip.h264               # PC -> board
sima-vision remote -- detect --frames 200   # run it there, watch it here
sima-vision pull                            # board -> PC

Say the address once:

export SIMA_VISION_DEVKIT=sima@192.168.137.50     # macOS, Linux

$env:SIMA_VISION_DEVKIT = "sima@192.168.137.50"   # PowerShell

or pass --host to any of the three. Authentication is ssh's own business: an agent, a key, or it asks you. Nothing here handles or stores a password.

Command	Notes
`sima-vision push FILE...`	Folders are copied whole. `--dest` changes where they land, default `~/`
`sima-vision pull`	With no names, takes whatever a run of any task left: the video, `frames/`, `alerts/`, `config.yaml`. `--into` chooses where they land here
`sima-vision pull detections.mp4`	Or name exactly what you want
`sima-vision remote -- ARGS`	Everything after `--` runs as `sima-vision ARGS` on the board

Three things these get right that a hand-written scp usually does not:

On Windows, scp D:\clips\a.h264 sima@ip:~ fails with could not resolve hostname d:, because scp reads everything before the first colon as a host. push never passes a full local path: it groups files by folder and runs scp from inside each one. pull does the same at the other end.
ssh host cmd without a pty means Ctrl-C never reaches the task. It keeps running, keeps the MLA, and your next launch fails with a busy device. remote always passes -tt.
pull with no arguments cannot be one scp, because scp fails the whole transfer on a name that is not there and the outputs depend on which task ran. So the names are listed over ssh first and only what exists is fetched.

They need an OpenSSH client, which macOS and Linux ship and Windows 10/11 has under Settings > Apps > Optional features > OpenSSH Client.

Set up a new DevKit

One time, about two hours, mostly downloading. Skip this if your board already runs pyneat.

Written on Windows with WSL2, which is the path SiMa's own tooling expects. Every warning below marks somewhere real time was lost.

   WINDOWS PC          WSL2 / UBUNTU              MODALIX DEVKIT 3.0
   ----------          -------------              ------------------
1  cable up      ----------------------------->   DHCP address
2  wsl --install ---->  Ubuntu ready
3  .wslconfig    ---->  WSL takes .137.1    --->  reachable both ways
                  4    sima-cli in a venv
                  5    docker + nfs
                  6    sdk setup           --->   pyneat on the board

Step 3 is load-bearing. Step 6 installs onto the board over the network and fails silently if networking is not fixed first, which is the usual way to lose an afternoon.

1. Cable up

USB (serial console) plus Ethernet straight to your PC. Open the serial tool and set the DevKit to DHCP.

arp -a | Select-String "192.168.137"     # find the board
ping <devkit-ip>

Must reply. Nothing else works until it does. The board's address changes between reboots; your PC keeps 192.168.137.1.

2. WSL2

wsl --install -d Ubuntu      # PowerShell as Administrator
wsl -l -v                    # want: Ubuntu, Running, 2

3. Mirrored networking

WSL sits behind NAT by default and cannot see your DevKit.

@"
[wsl2]
networkingMode=mirrored
"@ | Set-Content -Path "$env:USERPROFILE\.wslconfig" -Encoding utf8

wsl --shutdown

Wait ten seconds, open a WSL terminal, then verify both of these:

wsl -- hostname -I                  # must list 192.168.137.1
wsl -- ping -c 2 <devkit-ip>        # must reply

If they fail, check .wslconfig was not saved as .wslconfig.txt.

4. sima-cli

Become root first. sudo su - is a login shell, so it drops you in /root.

sudo su -
apt update && apt install -y git python3-venv python3-pip
python3 -m venv sima
source sima/bin/activate
pip install sima-cli
sima-cli login                  # needs a community.sima.ai account

5. Docker and NFS

The Neat SDK is a Docker container. No Docker, no SDK. Install Docker Engine with Docker's own instructions for Ubuntu, which stay current in a way a copy here would not, then add what the SDK needs on top:

sudo apt install -y nfs-kernel-server nfs-common

Two things are specific to WSL. Docker needs systemd to survive a restart:

grep -q 'systemd=true' /etc/wsl.conf 2>/dev/null || sudo tee -a /etc/wsl.conf <<'EOF'

[boot]
systemd=true
EOF

Then wsl --shutdown in PowerShell, reopen WSL, and confirm:

sudo systemctl enable --now docker
sudo docker run hello-world      # must print "Hello from Docker!"

6. The Neat SDK

sudo su -
source sima/bin/activate
sima-cli install ghcr:sima-neat/sdk
sima-cli sdk setup --devkit <devkit-ip>

Answer every prompt. The ones that matter:

Prompt	Answer
`Some system checks failed. Continue?`	`y`. The Firewall row says Unverified, not failed
`Install Model Compiler extension?`	`Y`. Adds 9 GB, only needed to compile your own models
`Install VSCode Extensions?`	`y` lowercase. A bare Enter is rejected
`Apply passwordless sudo on DevKit?`	`y`. Required for workspace sync
everything else	`Y` or Enter

mount.nfs: Connection timed out is fine; setup falls back to rsync and carries on.

Then confirm the board half actually happened, because that is the part that fails quietly:

ssh sima@<devkit-ip> "~/pyneat/bin/python3 -c 'import pyneat; print(pyneat.__version__)'"

A version means you are done, and it also tells you where pyneat lives: that venv is what to install into.

ssh sima@<devkit-ip>
~/pyneat/bin/pip install sima-vision
~/pyneat/bin/sima-vision doctor      # every row should say yes

No such file or directory from the check above means pairing never installed it, almost always because networking was not fixed first. Re-run sima-cli sdk setup --devkit <devkit-ip> from WSL now that it works.

Five rules that prevent most problems

#	Rule	Because
1	Networking before pairing	Pairing installs over the network. No route means a silent no-op
2	Docker before the SDK	The SDK is a container
3	`cd` after `sudo su -`	`-` is a login shell, so it drops you in `/root`
4	Raw `.h264`, never `.mp4`	Containers hit a demuxer bug in Neat 0.3.0
5	Never leave the only copy on the board	A firmware update wipes its home directory

Firmware version mismatch

ERROR: DevKit/SDK version mismatch. DevKit 2.0.0, SDK 2.1.2

New boards often ship older firmware. eLxr cannot be updated remotely, so this runs on the board, which already has internet over your Ethernet cable:

ssh sima@<devkit-ip>
sima-cli login
sima-cli update            # menu, then "Update all packages to the latest"

Budget 15 to 40 minutes plus a reboot, then re-run sima-cli sdk setup. If SSH complains the host key changed, that is expected: ssh-keygen -R <devkit-ip>.

Setup errors

Symptom	Fix
`sima-cli: command not found`	The venv is not active: `sudo su -`, then `source sima/bin/activate`
Venv landed in `/root/sima`	You ran `cd` before `sudo su -`
`externally-managed-environment`	Create the venv first
`Error: No such command 'sdk'`	You ran it on the board. `sdk` is PC-side
WSL cannot ping the DevKit	`.wslconfig` missing, saved as `.txt`, or WSL not restarted
`Cannot connect to the Docker daemon`	`sudo systemctl start docker`
Docker dead after every restart	systemd not enabled in `/etc/wsl.conf`
`ssh: Could not resolve hostname d:`	A Windows path went to `scp`, which read `D:` as a host. Use `sima-vision push`
Copy hangs	The board's IP changed. Find it again with `arp -a`
`DevKit/SDK version mismatch`	Firmware recovery, above

Troubleshooting

Symptom to fix, for a running task.

Symptom	Fix
`ModuleNotFoundError: pyneat` on the board	You installed into a Python that is not the `pyneat` venv. `~/pyneat/bin/pip install sima-vision`, or set `SIMA_VISION_PYNEAT`. `sima-vision doctor` confirms which
`ModuleNotFoundError: pyneat` on your PC	Expected. Inference only happens on the board; use `sima-vision remote -- detect` to drive it from here
`pyneat` missing after pairing	Pairing never installed it, almost always because networking was not fixed first. Re-run `sima-cli sdk setup --devkit <ip>` from WSL
`pyneat requires numpy<2`	`~/pyneat/bin/pip install "numpy>=1.24,<2" "opencv-python>=4.7,<5"`
`model archive not found`	`sima-cli login`, then run again. It fetches the pack itself
`sima-cli download did not produce ...`	Not logged in. `sima-cli login`, or pass `--model` with a path or URL
`source file not found`	The error lists what is actually in the folder. Paths are relative to where you launch
`is not a raw H.264 elementary stream`	You renamed an `.mp4` instead of converting it. The error carries the ffmpeg command
`No src-element named "nN_demux"`	The `.mp4` demuxer bug. Convert to `.h264`
Device busy	An orphaned run still holds the MLA: `sima-vision remote -- doctor` first, then `ssh sima@<devkit-ip> pkill -f sima-vision`
Stuck after `loading model`	The first load unpacks the archive. Give it a minute
First run seems to hang before anything prints	That is the 13 MB clip and the model downloading. It only happens once
`processed=0` and a 20 second timeout	The source caps are not negotiating. Leave `--fps`, `--width` and `--height` at 0
Output video shorter than the input, plays fast	Frames are being dropped. Set `runtime.overflow_policy: auto`
Recording only a few frames long	Usually `output.insight`. Its encoder shares the codec daemon with the decoder
`timed out waiting for instances` after a few frames	The graph starved the decoder's buffer pool. Lower `runtime.output_buffers`, and use `--minimal` to tell "too slow" from "wrong graph"
Dropped frames on a live source	Raise `--queue-depth`, keep `overflow_policy: auto`
`has unknown class`	A typo. The error suggests near matches from the labels file

Known issues

groups.video_input cannot play .mp4, Neat 0.3.0.

gst_parse_launch failed: No src-element named "n1_demux" - omitting link

VideoTrackSelect builds its fragment from one variable, so what it emits is correct. The graph then appends an instance suffix, but the renamer only rewrites name=<x> declarations, so the pad reference is never fixed. Any non-empty suffix breaks it, and reordering does not help.

The fix is no container, so no demuxer. sima-vision detects .h264, .264, .avc and .bin and builds the chain by hand:

FileInput -> H264Parse -> Queue -> SimaDecode -> CapsRaw

A container input still goes through groups.video_input and prints the conversion command.

How it works

The pipeline, and why it is shaped that way.

The pipeline is a Neat Graph, not a single Model.run, because it has several stages, named public endpoints and a branch with a fan-in:

source --> branch --> frame ---------------+
              |                            +--> combine("<task>_output")
              +----> model --> results ----+

<task>_output --> parse --> overlay --> video file + stills
                        |-> MetadataSender  (JSON over UDP)
                        +-> VideoSender     (H.264 RTP over UDP)

Frames come off the hardware decoder, whose buffer pool is small: the boot log prints BufferNum=8. Everything expensive therefore happens on a sink thread, so the pull loop hands each buffer back immediately. Holding one across a pull() is what deadlocks the decoder part-way through a clip, and it looks exactly like the app being slow.

Boxes arrive as one UInt8 tensor tagged BBOX, packed as a uint32 count followed by 24-byte records of x, y, w, h, score, class_id in source-image pixels. A segment head packs its masks into the tail of that same buffer.

sima_vision/
  cli.py        the command line          api.py      the Python API
  config.py     loading and validation    scene.py    the preview scene
  assets.py     clips and model archives  devkit.py   push, pull, remote
  media.py      H.264 and geometry        neat.py     graph assembly
  samples.py    decoding a sample         masks.py    masks and compositing
  draw.py       the overlay               sinks.py    video, stills, Insight
  runloop.py    the pull loop
  tasks/        detect.py   segment.py   fall.py

assets.py and devkit.py are the only modules that reach the network, and assets.py only from a run. A --validate or a preview resolves the same paths and fetches nothing.

Contributing

git clone https://github.com/RizwanMunawar/sima-projects.git
cd sima-projects
pip install -e ".[dev,preview]"

ruff check sima_vision tests
pytest -q

The tests need no board. Mask decoding, compositing, the overlay, the tracker and the fall rules are plain numpy and OpenCV, so they run anywhere, and the ssh and scp wrappers are tested against a fake subprocess. CI covers Python 3.10 to 3.13 on Linux, macOS and Windows, builds the wheel and installs it clean.

License

The models used here for testing are Ultralytics YOLO26, under AGPL-3.0. All other parts of this repository are under Apache-2.0. See LICENSE.

Credits

SiMa.ai for Modalix, the Palette SDK and Neat
Ultralytics for the YOLO26 models

Built by Muhammad Rizwan Munawar. If this saved you an afternoon, star the repo and pass it on to someone else bringing up a DevKit.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sima_vision-0.0.2.tar.gz (173.0 kB view details)

Uploaded Sep 2, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

sima_vision-0.0.2-py3-none-any.whl (145.7 kB view details)

Uploaded Sep 2, 2026 Python 3

File details

Details for the file sima_vision-0.0.2.tar.gz.

File metadata

Download URL: sima_vision-0.0.2.tar.gz
Upload date: Sep 2, 2026
Size: 173.0 kB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for sima_vision-0.0.2.tar.gz
Algorithm	Hash digest
SHA256	`33e3fd075b5a569c5677ae97c871fee228d75486c610efe4d2d7f07bc4c7a165`
MD5	`cf63bf51835202c39c8c761cc6d54520`
BLAKE2b-256	`7bbd573963d6ae25d15e61bc549187ad656ba8f6ac101b9b9a24a88b8e0b4faa`

See more details on using hashes here.

Provenance

The following attestation bundles were made for sima_vision-0.0.2.tar.gz:

Publisher: release.yml on RizwanMunawar/sima-projects

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: sima_vision-0.0.2.tar.gz
- Subject digest: 33e3fd075b5a569c5677ae97c871fee228d75486c610efe4d2d7f07bc4c7a165
- Sigstore transparency entry: 2685240555
- Sigstore integration time: Sep 2, 2026
Source repository:
- Permalink: RizwanMunawar/sima-projects@195d934296805a707f76dec9d45be7cdfeccae26
- Branch / Tag: refs/heads/main
- Owner: https://github.com/RizwanMunawar
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: release.yml@195d934296805a707f76dec9d45be7cdfeccae26
- Trigger Event: push

File details

Details for the file sima_vision-0.0.2-py3-none-any.whl.

File metadata

Download URL: sima_vision-0.0.2-py3-none-any.whl
Upload date: Sep 2, 2026
Size: 145.7 kB
Tags: Python 3
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for sima_vision-0.0.2-py3-none-any.whl
Algorithm	Hash digest
SHA256	`ba67c0a80ad9f10208d631e67d21a22d5ec9dd3965affc45f5b8851084fd8ae7`
MD5	`5b21ad96f43c7c0091a49e93a5fff847`
BLAKE2b-256	`2477f52b9ccfe9f1d27bcca1283349346e1acd52af9e0884ac162d467b4a4345`

See more details on using hashes here.

Provenance

The following attestation bundles were made for sima_vision-0.0.2-py3-none-any.whl:

Publisher: release.yml on RizwanMunawar/sima-projects

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: sima_vision-0.0.2-py3-none-any.whl
- Subject digest: ba67c0a80ad9f10208d631e67d21a22d5ec9dd3965affc45f5b8851084fd8ae7
- Sigstore transparency entry: 2685240568
- Sigstore integration time: Sep 2, 2026
Source repository:
- Permalink: RizwanMunawar/sima-projects@195d934296805a707f76dec9d45be7cdfeccae26
- Branch / Tag: refs/heads/main
- Owner: https://github.com/RizwanMunawar
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: release.yml@195d934296805a707f76dec9d45be7cdfeccae26
- Trigger Event: push

Release history Release notifications | RSS feed

0.1.0

Sep 2, 2026 2 files

This release

0.0.2 This release

Sep 2, 2026 2 files

sima-vision 0.0.2

Live YOLO26 on the MLA of a SiMa.ai Modalix DevKit 3.0.Three apps, one pipeline, one command.

Contents

Install

Quickstart

Without a board

On the DevKit

The three tasks

Commands

Settings

What people actually change

Shared flags

Cameras and streams

Environment

Python API

Driving the board from your PC

Set up a new DevKit

1. Cable up

2. WSL2

3. Mirrored networking

4. sima-cli

5. Docker and NFS

6. The Neat SDK

Five rules that prevent most problems

Firmware version mismatch

Setup errors

Troubleshooting

Known issues

How it works

Contributing

License

Credits

Project links

Key dates

1 maintainer

Credits

GitHub Statistics

License expression

Requires

Provides Extra

Tags

Classifiers

Download files

Source Distribution

Built Distribution

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

Release history Release notifications | RSS feed

Live YOLO26 on the MLA of a SiMa.ai Modalix DevKit 3.0.
Three apps, one pipeline, one command.