mrg-iot 1.3.2

IoT Testbed Experiment Automation CLI for MergeTB / Sphere Testbed

mrg-iot

IoT testbed experiment automation for the Sphere / Merge Testbed platform.

mrg-iot is a command-line tool that drives the entire lifecycle of an IoT experiment on the Merge Testbed — so you don't have to stitch together a dozen mergetb and ssh calls by hand. From a single command it will:

1. Create an experiment and generate its network model for the devices you select.
2. Realize the model (allocate physical hardware).
3. Materialize the realization (bring the allocation online).
4. Create an XDC (eXperiment Development Container) and attach it to the materialization.
5. SSH into the XDC and open SSH tunnels for RTSP video streams (:8554), file downloads (:9000), and the experiment's ExperimentControl gRPC channel (:17000).
6. Give you an interactive control loop (or a persistent background daemon) to drive devices, watch camera feeds in VLC, and pull experiment artifacts.
7. Clean up every resource it created — even on partial failure.

---

Table of contents

• Requirements
• Installation
• Quick start
• Authentication
• Two ways to run
  • One-shot mode: run
  • Persistent daemon mode: connect
• The spiot_ctl control language
• Command reference
• Input validation rules
• Configuration & on-disk state
• Architecture
• Troubleshooting
• Development
• Release process
• License

---

Requirements

Requirement	Notes
Python 3.9+	3.9, 3.10, 3.11, and 3.12 are supported and CI-tested.
A Merge Testbed account	With access to at least one IoT project (neuiot, sphereiotpostlude, or iotdev).
Network access	To the gRPC portal (grpc.sphere-testbed.net:443) and the SSH jump host (jump.sphere-testbed.net:2022).
VLC	Only needed for show video / RTSP playback. pip cannot install it — install it from your OS package manager. The vlc binary must be on your PATH.

---

Installation

The supported, production install path is pipx — it puts mrg-iot on your PATH in its own isolated environment:

pipx install mrg-iot

Or, into a regular virtual environment:

python3 -m venv .venv && source .venv/bin/activate
pip install mrg-iot

Verify the install:

mrg-iot --version
mrg-iot --help

Installing VLC (for video)

Platform	Command
macOS	brew install --cask vlc
Debian/Ubuntu	sudo apt-get install vlc
Fedora	sudo dnf install vlc
Windows	Download from videolan.org

---

Quick start

# 1. Authenticate once — credentials are cached under ~/.mrg-iot/session.json
mrg-iot login

# 2. Spin up an experiment end-to-end, interactively
mrg-iot run

# ...or fully scripted, no prompts:
mrg-iot run --non-interactive \
    --project neuiot \
    --devices s-echodot-1 s-googlenest-1 \
    --exp-name myexp \
    --exp-desc "Smart-speaker capture run" \
    --realization realiot \
    --duration 1w \
    --xdc myxdc

Inside the interactive control loop you can issue spiot_ctl commands:

spiot_ctl > exp devices
spiot_ctl > dev s-echodot-1 click_button
spiot_ctl > exit

---

Authentication

mrg-iot login     # prompts for username + password, stores a bearer token
mrg-iot logout    # clears the stored session

Credentials are written to ~/.mrg-iot/session.json with mode 0600. Every authenticated command transparently re-validates the stored token and re-prompts only if it has expired — you rarely need to run login more than once.

---

Three ways to run

mrg-iot offers three execution models. run and connect drive the full workflow from your laptop (SSH + tunnels); ctl is the lightweight in-XDC client. Pick based on where you are and how long you need the session to stay alive.

One-shot mode: run

mrg-iot run owns the SSH session, the spiot_ctl channel, and the tunnels for the lifetime of the process. When you type exit (or send Ctrl-C / SIGTERM), it tears everything down. This is the simplest mode and the default if you type mrg-iot with no subcommand.

mrg-iot run                       # interactive, prompts for everything
mrg-iot run --simplified          # interactive but skips optional prompts
mrg-iot run --non-interactive ... # fully scripted (see Quick start)

Useful run flags:

Flag	Purpose
--non-interactive, -n	No prompts; all required args must be supplied.
--simplified, -s	Interactive, but uses sensible defaults for optional steps.
--commands-file, -cf <path>	Run a newline-delimited file of spiot_ctl commands non-interactively.
--download-files, -df	Download the experiment archive on exit.
--delete-xdc, -dx	Delete the XDC during cleanup (default: keep).
--delete-exp, -de	Delete the experiment during cleanup (default: keep).
--network, -net <name>	Model network name (default: mrg-iot-net).

Tip: By default run keeps the XDC and experiment on exit so you can reconnect later with connect. Pass --delete-xdc/--delete-exp to do a full teardown.

Persistent daemon mode: connect

For long-running experiments you don't want tied to a single terminal, mrg-iot connect spawns a detached background daemon that holds the SSH session and tunnels open. Your shell returns immediately, and you drive the daemon from any terminal with send, show, download, and disconnect.

# Resolve + connect (interactive pickers if flags are omitted)
mrg-iot connect
mrg-iot connect --project neuiot --experiment myexp --realization realiot --xdc myxdc

# Drive the running experiment from anywhere
mrg-iot send "exp devices"
mrg-iot send "dev s-echodot-1 click_button"
mrg-iot send "exp cred s-echodot-1" "exp sleep 10"   # multiple commands, in order

# Open VLC viewers for every camera on the XDC
mrg-iot show video

# Pull the experiment traffic archive
mrg-iot download traffic                       # -> ~/Downloads/<enclave>.tar.gz
mrg-iot download traffic --output /tmp/run.tar.gz

# Shut the daemon down
mrg-iot disconnect

How it works: connect validates the enclave, then launches python -m mrg_iot.services.daemon.server detached via setsid. The daemon binds a Unix socket at ~/.mrg-iot/connection.sock and advertises readiness in ~/.mrg-iot/connection.json. Client commands exchange line-delimited JSON over that socket:

request:   {"cmd": "<name>", "args": <any JSON>}
response:  {"out": "<text>"} | {"err": "<text>"} ... {"done": <exit_code>}

Access to spiot_ctl is serialized with a lock, so concurrent send calls never interleave their output. Daemon logs land in ~/.mrg-iot/daemon.log.

In-XDC mode: ctl

When you're already inside an XDC (e.g. SSH'd in yourself), you don't need the SSH/tunnel machinery — the experiment's ExperimentControl server is reachable directly over WireGuard. mrg-iot ctl connects to it and gives you the same interactive spiot_ctl > prompt, replacing the standalone spiot_ctl script. Install mrg-iot in the XDC and run:

mrg-iot ctl              # connect to the experiment (default host 192.168.254.1)
mrg-iot ctl 10.0.0.5     # override the ExperimentControl host

Each line you type is sent as one RunCommand gRPC call; type exit (or Ctrl-C / Ctrl-D) to quit. Prompt history is saved to ~/.spiot_history.

---

The spiot_ctl control language

Whether you're in the interactive run loop or using mrg-iot send, you speak to devices through spiot_ctl. The built-in help command lists everything; the most common verbs:

Command	Description
help	Show all available commands.
exp devices	List the devices in the experiment.
exp cred <device>	Get a device's credentials (e.g. exp cred s-echodot-1).
exp sleep <seconds>	Pause for N seconds.
exp read <filepath>	Read a command file on the XDC and run every line in it.
dev <device> help / dev <device> commands	Show the commands a device supports.
dev <device> <command> help	Detailed help for one device command.
dev <device> [-s <id>] <command> [args]	Execute a device command (optionally storing a job id).
query state <device> <command> <id>	Check the state of a running command.
query get_result <device> <command> <id>	Fetch the result of a finished command.
query wait_result <device> <command> <id>	Block until a command finishes, then return its result.
exit	Tear down spiot_ctl, optionally download files, and clean up.

The device-commands/ directory in this repo contains a per-device reference of supported commands (one *-cmds.txt file per device, e.g. s-echodot-1-cmds.txt).

---

Command reference

mrg-iot exposes resource-CRUD subcommands so you can manage individual testbed objects without running the full flow. Add --help to any subcommand for its exact flags, and --debug/-d to any command for verbose logging.

Experiments

mrg-iot exp list [--project p]                              # list (all projects if omitted)
mrg-iot exp info   --project p --name n                     # details + realizations
mrg-iot exp create --project p --name n [--description d] \
        [--network net] [--realization r] [--duration 1w]   # create + realize + materialize
mrg-iot exp delete --project p --name n

Realizations (alias: rz)

mrg-iot realization list   --project p --experiment e
mrg-iot realization info   --project p --experiment e --name r
mrg-iot realization status --project p --experiment e --name r
mrg-iot realization create --project p --experiment e --devices d1 d2 \
        [--name r] [--network net] [--duration 1w]          # build model, compile, push, realize
mrg-iot realization relinquish --project p --experiment e --name r

Materializations (alias: mtz)

mrg-iot materialization list          --project p --experiment e
mrg-iot materialization info          --project p --experiment e --realization r
mrg-iot materialization status        --project p --experiment e --realization r
mrg-iot materialization create        --project p --experiment e --realization r
mrg-iot materialization dematerialize --project p --experiment e --realization r

XDCs

mrg-iot xdc list [--project p]
mrg-iot xdc info   --project p --name n
mrg-iot xdc create --project p [--name n] [--duration 1w] [--no-wait]
mrg-iot xdc create --project p --experiment e --realization r        # create + attach
mrg-iot xdc attach --project p --name n --experiment e --realization r
mrg-iot xdc detach --project p --name n --experiment e --realization r
mrg-iot xdc delete --project p --name n

Connection / daemon

mrg-iot connect    [--project p --experiment e --realization r --xdc x]
mrg-iot send       "<cmd>" ["<cmd>" ...]
mrg-iot show video
mrg-iot download   traffic [--output <path>]
mrg-iot disconnect

In-XDC client

mrg-iot ctl [host]      # interactive spiot_ctl prompt, run from inside the XDC

Typical cleanup sequence

mrg-iot xdc detach --project neuiot --name myxdc --experiment myexp --realization realiot
mrg-iot xdc delete --project neuiot --name myxdc
mrg-iot materialization dematerialize --project neuiot --experiment myexp --realization realiot
mrg-iot realization relinquish --project neuiot --experiment myexp --name realiot
mrg-iot exp delete --project neuiot --name myexp

---

Input validation rules

Inputs are validated up front so bad arguments fail fast with a clear [mrg-iot] Invalid <label>: … message (exit code 2) rather than deep inside a portal call:

• Names (exp-name, realization, xdc, network): must match ^[a-z][a-z0-9]*$ — start with a lowercase letter, lowercase letters and digits only, max 32 chars.
• Description: letters, digits, spaces, commas, periods, hyphens; max 256 chars.
• Duration: minimum 4 days. Accepts forms like 1w, 4d, 1w2d3h, 1 week. Anything under 4 days is rejected.

---

Configuration & on-disk state

Everything user-scoped lives under ~/.mrg-iot/:

File	Purpose
session.json	{"username", "token"} — your cached login (mode 0600).
connection.json	Active daemon descriptor (project, experiment, realization, xdc, pid, socket, started_at). Stale entries are auto-cleared when the pid is dead.
connection.sock	The daemon's Unix socket (mode 0600).
daemon.log	Daemon stdout/stderr after daemonization.

--debug additionally writes verbose logs to debug.log in the current directory (and to stderr).

Endpoints & known projects

• gRPC portal: grpc.sphere-testbed.net:443 (via mergetbapi==1.3.41)
• SSH jump host: jump.sphere-testbed.net:2022
• Tunneled ports: 8554 (RTSP), 9000 (file download)
• Known projects: neuiot, sphereiotpostlude, iotdev

---

Architecture

The codebase follows an MVC + services layout under the mrg_iot/ package. Controllers translate argparse args into service calls; services hold business logic (portal gRPC client, SSH/tunnel/daemon plumbing); models hold data structures and on-disk state; views own all user-facing output.

mrg_iot/
├── cli.py                  # argparse tree + main() dispatcher
├── constants.py            # pure data: paths, endpoints, model template, help text
├── controllers/            # argparse args -> service calls (@require_login)
│   ├── experiment.py  realization.py  materialization.py
│   ├── xdc.py  run.py  connection.py  _base.py
├── models/                 # dataclasses + on-disk state
│   ├── enclave.py  validators.py  session.py  connection.py  memory.py
├── views/                  # all user-facing output
│   ├── output.py  formatters.py  prompts.py  log_manager.py
├── services/               # business logic
│   ├── auth.py
│   ├── experiment_workflow.py   # the `run` state machine + spiot session lifecycle
│   ├── reconnect.py             # interactive pickers + readiness checks for `connect`
│   ├── ssh.py                   # XDC SSH connect + spiot_ctl bootstrap + framed reader
│   ├── tunnel.py                # Paramiko port forwarding (RTSP 8554, download 9000)
│   ├── api/                     # gRPC client (mergetbapi), retry wrapper, error handling
│   └── daemon/                  # detached background daemon (server/client/protocol)
└── utils/                  # duration parsing, file helpers

The experiment workflow (services/experiment_workflow.py):

Login/Session → Select Project → Select Devices → Create Experiment →
Build+Compile+Push Model → Realize → Materialize → Create/Attach XDC →
SSH to XDC + start spiot_ctl + open RTSP/download tunnels → Interactive loop → Cleanup

State is captured in an ExpState dataclass so cleanup can tear down exactly what was created, even on partial failure. Both run and the connect daemon share the same start_spiot_session code path.

---

Troubleshooting

SSL / TLS error talking to the portal (common on macOS):

export SSL_CERT_FILE="$(python -m certifi)"
export REQUESTS_CA_BUNDLE="$SSL_CERT_FILE"

show video does nothing / errors: Make sure the vlc binary is installed and on your PATH (vlc --version). pip/pipx cannot install VLC.

No active connection: The daemon isn't running, or its pid died. Run mrg-iot connect again — stale connection.json state is cleared automatically.

A command hangs or times out: Re-run with --debug and inspect debug.log (and ~/.mrg-iot/daemon.log for daemon mode) for the underlying gRPC/SSH error.

Resources left behind after a crash: Use the cleanup sequence to remove the XDC, materialization, realization, and experiment individually.

---

Development

git clone https://gitlab.com/sphere-neu/mrg-iot.git
cd mrg-iot
python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"

Tests, type-checking, linting, formatting:

pytest tests/ -v                              # asyncio_mode=auto; no extra flags
pytest tests/test_daemon_protocol.py -v       # a single file
mypy mrg_iot
pylint mrg_iot
black mrg_iot tests && isort mrg_iot tests

The dev extras (black, pylint, isort, etc.) require Python 3.10+. On a 3.9 environment, install runtime deps only (pip install -e .).

---

Release process

CI (.gitlab-ci.yml) runs pytest on every push, builds an sdist + wheel on success, and publishes to PyPI on vX.Y.Z tags via OIDC trusted publishing.

The git tag must equal v$(cat mrg_version) — the publish job aborts otherwise. To cut a release:

# bump the version
echo "1.3.2" > mrg_version
git commit -am "increased version to 1.3.2"
git tag v1.3.2
git push && git push --tags

---

License

MIT © Sphere / NEU IoT Testbed contributors