otto

otto — Our Trusty Testing Orchestrator — is a framework for deploying products to remote hosts for testing and validation. It provides a CLI and a Python API for running commands on remote systems, transferring files, executing tests, and monitoring host metrics in real time.

Who is otto for?

Otto is a general-purpose tool for developers and testers who need to interact with one or more remote machines as part of their workflow — deploying builds, validating firmware, running integration tests, or collecting performance data.

Two ways to use otto

• CLI users — interact with otto through the otto run, otto test, otto monitor, and otto cov commands.
• API builders — import otto's Python packages to build higher-level automation on top of hosts, tests, and the monitor.

Installation

Otto requires Python 3.10 or later. Install the latest release from PyPI into a virtual environment:

python3 -m venv .venv
source .venv/bin/activate
pip install otto-sh

The distribution is named otto-sh; the CLI command it installs is otto. For development installs, building from a wheel, GitHub-release artifacts, and air-gapped installation, see docs/installation.md.

Key concepts

Hosts

A Host represents a machine otto can talk to. UnixHost connects over SSH or Telnet; EmbeddedHost (and its concrete ZephyrHost) drives a firmware/RTOS target over a serial console; LocalHost runs commands on the local machine with no network; DockerContainerHost targets a container. All extend a common BaseHost interface (run, exec, send/expect, and — on the networked hosts — put/get).

run executes a command on a host's persistent shell session (state like the working directory and environment variables are preserved between calls). exec runs each call independently of the persistent shell and of other concurrent exec calls, making it safe to fan out via asyncio.gather().

Labs

Hosts can be reached through intermediate hops — SSH jump hosts that otto tunnels through automatically. Hops can be chained for multi-hop paths (otto -> hop1 -> hop2 -> target). All file transfer protocols (SCP, SFTP, FTP, netcat) work through hops. Embedded hosts use their own console/tftp transfer backends instead (see the embedded-hosts guide). Set the hop field in a host's JSON definition or use --hop on the CLI.

A Lab is a JSON file that describes a set of hosts and their topology. Otto loads labs at startup (via --lab or the OTTO_LAB environment variable) and makes every host available to instructions, tests, and the monitor. Multiple labs can be merged by combining their names with + (--lab lab_a+lab_b).

[
    {
        "ip": "192.168.1.1",
        "ne": "router1",
        "osType": "unix",
        "term": "ssh",
        "creds": [{ "login": "admin", "password": "secret" }]
    },
    {
        "ip": "192.168.1.2",
        "ne": "switch1",
        "term": "telnet",
        "creds": [{ "login": "admin", "password": "secret" }]
    }
]

Projects and settings

Otto discovers your project through a .otto/settings.toml file at its project root, the directory you list in OTTOPATH (the vocabulary is in docs/overview.md). This file tells otto where to find your Python libraries, tests, run instructions, and lab data:

name = "my_project"
version = "1.0.0"

libs  = ["pylib"]
tests = ["tests"]
init  = ["my_instructions"]

[[lab.sources]]
backend = "json"
paths = ["../lab_data"]

Relative paths resolve against the project root at load time. The init list names Python modules that otto imports at startup — this is where you register your instructions and shared options. [[lab.sources]] declares where hosts come from, in order; several sources combine, with later ones overriding earlier ones per host record.

Instructions (otto run)

An instruction is an async function decorated with @instruction() that becomes a subcommand of otto run. Instructions have full access to the lab's hosts and can accept their own CLI options via Typer annotations:

import logging

from otto import all_hosts
from otto.instructions import instruction

logger = logging.getLogger("otto")


@instruction()
async def deploy(
    debug: Annotated[bool, typer.Option(help="Deploy debug products.")] = False,
):
    for host in all_hosts():
        await host.run(["echo deploying", "make install"])
    logger.info("Done")

otto -l my_lab run deploy --debug

Tests (otto test)

otto tests are pytest tests: Test-prefixed classes and test_-prefixed functions in your project's test directories. Fixtures, conftest.py, markers, parametrize and pytest plugins all work as pytest documents them. otto test runs them by name or marker expression. Its flags include those of every options class a project registers for the test verb, and a test reads their values with ctx.options(...):

import logging

from my_project_instructions import DeviceOptions  # the init module declares it

logger = logging.getLogger(__name__)


class TestDevice:
    async def test_device_reachable(self, ctx) -> None:
        logger.info(f"firmware={ctx.options(DeviceOptions).firmware}")
        assert True

otto -l my_lab test TestDevice --firmware 2.1
otto test --iterations 10 --threshold 95 TestDevice

Tests get pytest markers (timeout, retry, parametrize, integration), non-fatal assertions via the expect fixture, per-test artifact directories, and built-in monitoring.

An options class is declared in an init module (a module listed in your init setting), with the verbs whose flags it joins: @otto.options(verbs=["test"]) for flags only tests take, or @otto.options(verbs=["run", "test"]) for project-wide flags such as a device type or a lab environment, which every instruction and every test run then take.

Monitor (otto monitor)

The monitor collects live performance metrics (CPU, memory, disk, network) from one or more hosts and serves an interactive web dashboard:

otto -l my_lab monitor                     # all hosts, default 5 s interval
otto monitor host1,host2 --interval 2.0    # specific hosts, faster polling
otto monitor --db metrics.db               # persist data for later viewing
otto monitor --file metrics.db             # replay saved data

A test can also start a monitor of its own through the monitor fixture: await monitor.start(hosts=...), stopped for it when the test ends.

Coverage (otto cov)

Otto retrieves gcov code-coverage data from the systems under test and renders multi-tier HTML reports — e2e, unit, and manual coverage merged into a single per-line view:

otto -l my_lab test TestDevice --cov   # collect coverage during a test run
otto cov report                        # renders under this invocation's output dir (path printed)

This works for GCC- and clang-built products on Unix hosts (.gcda counters fetched over the network, cross-toolchains supported) — and for embedded RTOS targets, where otto pulls coverage over the serial console from an instrumented LLEXT extension. See docs/cli/cov/ and its per-build-type instrumenting subpages (GCC, clang, embedded).

Quick-start example

1. Set the environment — point otto at your project and lab:

   export OTTOPATH=/path/to/my_project
   otto --lab my_lab --list-hosts          # verify hosts are visible
   

2. Run an instruction:

   otto -l my_lab run deploy --debug
   

3. Run tests:

   otto -l my_lab test TestDevice --firmware 2.1
   

4. Monitor hosts:

   otto -l my_lab monitor
   

Documentation

Hosted documentation: otto-sh.readthedocs.io.

The same content lives under docs/ and can be built locally with make docs — the generated HTML is written to docs/_build/html/. Key entry points:

• docs/getting-started/index.md — installation and first steps
• docs/installation.md — air-gapped installs, team setup, offline docs
• docs/cli/ — one page per command, mirroring otto's own command tree
• docs/cli/host/ — the otto host verbs, who a command runs as, per-family support, capabilities and embedded hosts
• docs/cli/cov/ — coverage collection & reports (GCC, clang, embedded)
• docs/configuration/ — settings.toml, lab.json, host sources and options
• docs/cookbook/ — using otto from Python: library use, authoring, extending, and short recipes
• docs/api/ — full API reference for all otto packages
• docs/architecture/ — how otto is designed, one page per subsystem