Computer vision applications on the SiMa.ai Modalix DevKit 3.0. Object detection, instance segmentation, fall detection and retail analytics: one pipeline, no setup step. Needs a Modalix DevKit 3.0 and Python 3.10 or later.
| Object detection | Instance segmentation | Fall detection | Retail analytics |
detect |
segment |
fall |
ret-analytics |
Quickstart
Every command here is typed on the board.
pip install sima-cli
# Login via SiMa.ai Portal
sima-cli login
# Install Neat core, once
sima-cli neat install core@v0.3.0
# Install sima-vision
pip install sima-vision
# Run object detection
sima-vision detect
The first run fetches a pretrained YOLO26 pack and a demo clip into ./assets, then runs.
Every run after that reuses them. Both come from one public GitHub release, so getting
them needs no login and no sima-cli.
| App | Fetched for you | Writes |
|---|---|---|
detect |
yolo26n-det-bf16-mla_tess-b1.tar.gz (21 MB) + a 1080p demo clip (13 MB) |
detections.mp4 |
segment |
yolo26n-seg-bf16-mla_tess.tar.gz (24 MB) + the same clip |
segmentation.mp4 |
fall |
the detection pack again + a shorter clip (1.2 MB) | falls.mp4 |
ret-analytics |
the detection pack + shopping-trolley-items.mp4 |
retail.mp4 |
Each app writes annotated video. Stills are off unless you ask for them
with --save.
Instance segmentation · per-pixel masks, with the background blurred
sima-vision segment # background blurred, instances sharp
sima-vision segment --keep-classes person # only people stay sharp
sima-vision segment --anonymise # the other way round: blur the people
sima-vision segment --no-blur # a plain overlay, nothing blurred
Fall detection · tracks people and relabels the box when one goes down
sima-vision fall
sima-vision fall --classes person --confirm 2.0
The frame is the one detect draws: same palette, same boxes, same captions. A confirmed
fall changes one thing, the class the box is labelled with, so a person reading person 0.93 a second earlier now reads FALL 0.93. Each fall is also printed on the console and
counted in the run summary.
A confirmed fall re-arms only after sustained recovery.
Retail analytics · displays items in a shopping trolley
sima-vision ret-analytics
sima-vision ret-analytics --source shopping-trolley-items.mp4
sima-vision ret-analytics --classes bottle banana apple
Displays detected trolley items with boxes, names and confidence, and writes
retail.mp4. Uses the same source, model and recording flags as detect.
The white side menu uses a table with Qty, Item, Unit $, and Total $ columns,
larger text, alternating rows, and an estimated basket total.
Each product row is bold and has its own consistent color.
Products enter the menu after three consecutive detections and stay in first-added
order. Quantities increase only after a higher simultaneous count is confirmed for
three frames. Brief misses never remove rows or reduce counts, and repeated frames
never add purchases. The menu retains confirmed items for the run; removals are not
inferred, and new items hidden behind existing ones cannot be counted reliably.
All model classes are displayed by default; --classes optionally filters them.
The default sample is shopping-trolley-items.mp4, fetched automatically.
The default model recognizes COCO categories such as bottle, banana, apple,
and orange. Product-specific recognition needs a trained model and matching labels.
Point the camera at the trolley so the displayed detections represent its contents.
The optional retail.classes config setting selects item names or ids;
an empty list displays all model classes.
Prices are rough per-item US estimates, not live quotes. Produce estimates use
US retail listings as a reference.
Unpriced classes show price N/A and are excluded from the known subtotal.
Defaults include cherry ($0.10 each), cantaloupe ($3.00 each), banana ($0.20 each),
cucumber ($0.75 each), bread ($2.00 per loaf), snack ($1.50 per small packet),
orange ($1.00 each), avocados ($1.00 each), egg ($0.25 each), and milk
($4.00 per gallon container). Both avocado and avocados labels are supported.
Override estimates for your products in config.yaml:
retail:
classes: []
prices:
banana: 0.20
tomato: 0.25
apple: 1.00
The COCO model does not have a tomato class; tomato detection requires a model trained for tomatoes and a matching labels file.
If an automatically discovered config.yaml contains fall settings without a
retail section, retail uses its own sample and output paths and prints a warning.
CLI flags still win; an explicit --config uses the named file as written.
Apps arguments
Every flag, and which apps take it. All of them are for the four apps, which run on the
DevKit. sima-vision <app> --help prints the same list.
| Flag | Apps | What it does |
|---|---|---|
--source, -s URI |
all | File, https URL, RTSP URL, or empty for the sample clip |
--source-type |
all | video, rtsp or usb. Default video |
--fps N |
all | Source frame rate. Default 0, read from the stream |
--width PX |
all | Source width. Default 0, read from the stream's SPS |
--height PX |
all | Source height. Default 0, read from the stream's SPS |
--model, -m PATH |
all | Compiled pack, or an https URL to one. Empty fetches this app's default |
--labels PATH |
all | Newline-separated class names. Defaults to the packaged COCO list |
--family NAME |
all | Detection head. Must match the model or every box is noise |
--conf T |
all | Minimum confidence. Default 0.30 |
--iou T |
all | Non-max suppression IoU. Default 0.60 |
--max-det N |
all | Top-K cap per frame. Default 50 |
--frames, -n N |
all | Stop after N frames. Default 0, runs until interrupted |
--timeout MS |
all | How long to wait for a frame before giving up. Default 20000 |
--video-path PATH |
all | Where the annotated recording is written |
--no-video |
all | Do not record |
--video-encoder |
all | sima (hardware H.264, default) or opencv (software, about ten times slower) |
--video-bitrate KBPS |
all | Target bitrate for the hardware encoder. Default 12000 |
--save |
all | Also write annotated stills. Off by default, so only the video is written |
--save-dir DIR |
all | Where annotated stills are written. Implies --save |
--save-every N |
all | Write every Nth still. Default 10 once stills are on; implies --save. 0 disables |
--no-save |
all | Do not write stills. Already the default; use it to override a config file |
--show-fps |
all | Show the FPS badge on the overlay (hidden by default) |
--no-hud |
all | Leave the frame-rate badge off the overlay |
--hud-scale N |
all | Frame-rate badge font size. Default 2.4, which is 1.5x the caption scale; 0 follows the caption scale exactly |
--hud-thickness N |
all | Badge stroke weight. Default 6, which is 1.5x the caption thickness; 0 follows it exactly |
--hud-bg B,G,R |
all | Badge fill colour. Default 132,28,193 (#C11C84) |
--hud-color B,G,R |
all | Badge text colour. Default 255,255,255 |
--hud-padding PX |
all | Gap between badge text and its edge, which is what sizes the badge. Default 22; 0 follows the caption padding |
--config, -c PATH |
all | Config file. Defaults to ./config.yaml |
--no-config |
all | Ignore any config file and use defaults plus these flags |
--validate |
all | Resolve and check the settings, then stop. Needs no board |
--quiet, -q |
all | Warnings, errors and the closing report only |
--profile |
all | Per-stage timings, when a run is slower than it should be |
--queue-depth N |
all | Neat's own queue depth. Below 4 the graph drops frames when the recorder falls behind. Default 4 |
--sink-queue-depth N |
all | Finished frames that may wait for the recorder. Host memory only. Default 12 |
--sink-queue-mb MB |
all | Memory budget for that backlog, which grows it to fit a known clip |
--output-buffers N |
all | Buffers each public output may hold. Default 1 |
--decoder-buffers N |
all | Buffers to ask the decoder for. Default 0, sized from the stream's reference frames |
--decoder-tuning |
all | Hardware decoder preset. Default default, which keeps every frame; auto drops frames in bursts |
--segment-frames N |
all | Frames per piece when a clip is too long for one decode. Default 150; 0 runs it whole |
--blur / --no-blur |
segment |
Blur the background and keep instances sharp. On by default; --no-blur draws a plain overlay |
--blur-method |
segment |
gaussian, pixelate or none. Default gaussian |
--blur-strength PX |
segment |
Gaussian kernel width at 1080p. Default 41 |
--keep-classes CLASS... |
segment |
Names or ids that stay sharp. Default: every detected class |
--anonymise, --anonymize |
segment |
Blur the instances instead of the background |
--mask-threshold T |
segment |
Mask cut-off as a probability. Default 0.5; lower grows instances |
--no-masks |
segment |
Blur around plain boxes instead. Needs no segment head |
--minimal |
segment |
Pull frames and do nothing else. Tells a slow app from a stalled graph |
--classes CLASS... |
fall, ret-analytics |
Fall classes (default person) or retail products (default all model classes) |
--confirm S |
fall |
How long a fall signal must hold before it counts. Default 1.5 |
--no-fall |
fall |
Track without judging falls, which is how you tune tracking first |
A config.yaml in the working directory is picked up on its own. Flags beat it, and it
beats the built-in defaults.
Your own model
A trained .pt has to be compiled into a .tar.gz pack before the board can run it. The
compiler is the afe package and exists only inside the Palette Model SDK container,
which is x86 Docker, which on Windows means WSL2. So the whole of this runs on your PC,
needs a community.sima.ai account and ~10 GB of disk, and
takes about 30 minutes the first time. Quickstart needs none of it.
Converting a trained .pt into a DevKit pack
# --- PowerShell, as Administrator -------------------------------------------
wsl --install -d Ubuntu
wsl -l -v # want: Ubuntu, Running, 2
# Install Docker Desktop, then Settings > Resources > WSL integration > Ubuntu.
# Without that toggle docker works in PowerShell and is missing inside Ubuntu.
wsl -d Ubuntu # everything below is Ubuntu. exit comes back
# --- Ubuntu ------------------------------------------------------------------
docker run hello-world # must print "Hello from Docker!"
sudo apt update && sudo apt install -y python3-venv python3-pip
python3 -m venv ~/sima # Ubuntu refuses a bare pip install
source ~/sima/bin/activate # new shells start outside it. Look for (sima)
pip install sima-cli
sima-cli login
sima-cli install ghcr:sima-neat/sdk:v2.0.0 # several GB, once per machine
cd /path/to/your/model # the folder holding best.pt
sima-cli sdk setup --workspace . # . is what gets mounted. 15-20 minutes
activate-model-compiler
# --- Model SDK shell ---------------------------------------------------------
pip install "sima-vision[compile]" # ultralytics, onnx, onnxsim
sima-vision compile best.pt # ONNX, bfloat16, tessellate, ELF. 10-15 minutes
# -> build/best_mpk.tar.gz
# --- on the DevKit -----------------------------------------------------------
sima-vision push build/best_mpk.tar.gz
sima-vision detect --model best_mpk.tar.gz
sima-vision detect --model https://example.com/my-model.tar.gz # a URL works too
Moving files
Output lands beside the run, on the board, and both commands are typed on your PC. Name the board once and neither command needs
--host:
$env:SIMA_VISION_DEVKIT="sima@<devkit-ip>"
sima-vision pull # everything the run left
sima-vision pull --into results/
sima-vision push my-clip.h264
The default pull includes retail.mp4 and retail-report.json when present,
along with the other apps' outputs. Missing files are skipped.
Environment
| Variable | What it does |
|---|---|
SIMA_VISION_DEVKIT |
The board, as user@address, so push and pull stop asking |
SIMA_VISION_ASSETS |
Where clips and models are downloaded. Default ./assets |
SIMA_VISION_PYNEAT |
The pyneat virtualenv, when the search does not find it |
SIMA_VISION_PYNEAT_INDEX |
A pip index carrying a pyneat wheel, if your site publishes one |
SIMA_VISION_AUTO_INSTALL |
0 to look but never install |
SIMA_VISION_QUIET |
Non-empty is --quiet for every command |
SIMA_VISION_COLOR |
0 or 1 to force colour off or on. NO_COLOR also works |
Contributing
git clone https://github.com/RizwanMunawar/sima-projects.git
cd sima-projects
pip install -e ".[dev]"
ruff check sima_vision tests
pytest -q
The tests need no board.
License
The models used here for testing are Ultralytics YOLO26, under AGPL-3.0. All other parts of this repository are under Apache-2.0. See LICENSE.
Credits
- SiMa.ai for Modalix, the Palette SDK and Neat
- Ultralytics for the YOLO26 models
- Timothy Do for the Neat 0.4.0 port, frame-complete recordings and hardware H.264 recording
Metadata
Release files for sima-vision 26.0.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| sima_vision-26.0.4.tar.gz | 250.5 kB | With provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| sima_vision-26.0.4-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 429.7 kB
Release files / sima_vision-26.0.4.tar.gz
| Download URL | sima_vision-26.0.4.tar.gz |
|---|---|
| Size | 250.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
bfaefbe2bfbd489ec4d4b273fcd6a3c671eec88df3cf95292318c927e36bd288
|
|
BLAKE2b-256 checksum How to use checksums |
2a7beb0baf6195d74460d46c4fa2cacd761cbcbbbe972a0f2e25bc80b3fb9786
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 21, 2026.
Transparency logRelease files / sima_vision-26.0.4-py3-none-any.whl
| Download URL | sima_vision-26.0.4-py3-none-any.whl |
|---|---|
| Size | 179.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
24af5060465ebaaaefe61545eefc2d128b3319f7ba850248541f36eeadb4e9c5
|
|
BLAKE2b-256 checksum How to use checksums |
ccfd357f9be4d8301b790f9ea1270ce92967074742b2b9cf2f58be1b53877809
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 21, 2026.
Transparency logSecurity for sima-vision 26.0.4
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
All 2 files in this release have been uploaded with provenance metadata.
Attestation for sima_vision-26.0.4.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 21, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.