UST Format Checker

Checks that xDOS firmware keeps its output format backward compatible and in step with the documentation, by running the firmware in simavr against a deterministic scenario and inspecting what it prints on the UART.

Plan and requirements: PLAN.md. Per-message rules: CHECKS.md. Where docs, parser and firmware disagree: DOC_DEVIATIONS.md.

The checker only knows format version 2.1. Every capture has to name its own format with $DATAFORMAT; without it the check fails. Older data is still readable by DOSPORTAL, but firmware is only released as 2.1.

Installing

pip install ust-format-checker

Needs simavr to run a firmware (apt install simavr libsimavr-dev libelf-dev gcc). Installed commands: xdos-check (a capture), xdos-check-firmware (an ELF, end to end), xdos-build (board + scenario -> runner directives).

In a device repository you normally do not install it by hand - the PlatformIO hook pins a version in xdos/checker.txt and builds its own venv from it.

Installed standalone, layer 3 is skipped and says so: packages.parsing lives in DOSPORTAL and is not a package of its own yet. Run from a DOSPORTAL checkout to get it.

Layout

• messages.yaml - the message schema: fields, types, ranges, which rules apply.
• components/ - what a component is, which runner directive drives it and which messages it makes the device produce.
• schema.py, checks.py, rules.py - schema loading and layer 1 checks.
• lines.py - a capture as bytes and lines ending at \n; nothing is stripped or decoded away before the checks see it.
• structure.py - rules about the file itself, checked on the bytes: allowed characters, line endings, line length.
• requirements.yaml, requirements.py - how each requirement of the format specification is verified. A check names the IDs it verifies where it creates the finding, a test with @pytest.mark.req(...); test_requirements_matrix.py keeps the three in step.
• matrix.py - prints the verification matrix as markdown (python -m ust_format_checker.matrix), for comparing with the documentation or keeping with a CI run.
• emitted.py - a message a component on the board produces must come out.
• parser_layer.py - layer 3: what packages.parsing reads out of the same log.
• stimulus_checks.py - the value the simulation injected has to be the value that comes out.
• findings.py, report.py, cli.py - findings, the rustc-style report and the xdos-check entry point.
• simulate.py - compiles the runner and runs it; no container in the loop.
• firmware.py - the whole chain in one command, plus the verdict cache.
• platformio.py - what the device-repository hook calls; holds every path convention.
• pytest_plugin.py - the capture fixture, so a device repository needs no conftest.
• regressions/ - real captures carrying format bugs the checker must keep catching. Only genuine captures belong here; a hand-written one shares its blind spots with the rule it guards, so that case goes in a unit test.
• Which component sits at which address or pin, and what the simulation injects, belong to the device repository as xdos/board.yaml and xdos/scenarios/*.yaml. A device with no scenario of its own gets one generated from its board.
• scenario.py, build_scenario.py - board + scenario -> runner directives + the stimulus to check against.
• simulator/runner.c - loads the firmware ELF into simavr, attaches component models, replays the directives, prints the UART.

Checking a captured log

cd backend
python -m packages.ust_format_checker <log>                        # schema + parser layer
python -m packages.ust_format_checker <log> --board <board.yaml>   # + messages the board must produce
python -m packages.ust_format_checker <log> --stimulus <json>      # + injected values (from xdos-build)
python -m packages.ust_format_checker <log> --no-parser            # schema only
python -m packages.ust_format_checker <log> --warnings-as-errors   # CI profile

Exit code 1 means the log failed. Layer 3 (packages.parsing) never produces an error on its own: the parser can be wrong just as easily as the firmware, so its findings are warnings that say so.

With --board, a message that a component on the board produces and the capture does not contain is an error. Without it the checker cannot tell "this device has no such sensor" from "the firmware stopped printing it", and has to treat both as the same piece of information.

Checking a firmware ELF

The whole chain in one command - this is what the PlatformIO hook in a device repository calls:

xdos-check-firmware firmware.elf --board xdos/board.yaml --scenario xdos/scenarios/basic.yaml \
    --cache .pio/build/<env>/xdos/last.json

--cache remembers the verdict against a fingerprint of the ELF, the board, the scenario and the checker itself: when none of them changed, the run is skipped but the finding is printed again, so it cannot quietly disappear after the first build. A new checker version or an edited board.yaml always runs the check again. A missing simulator is reported and exits 0, so a firmware build never fails because the checker could not run.

The simulation also reports any I2C address the firmware talked to that board.yaml does not know (UNMODELLED_DEVICE). Without it an incomplete board model makes the check silently test less than it appears to.

In a device repository

The hook and the path conventions live in platformio.py, so changing any of them is a version bump rather than a pull request against every device repository:

python -m ust_format_checker.platformio <project_dir> <build_dir> <elf>        # local, warns
python -m ust_format_checker.platformio --ci <project_dir> <build_dir> <elf>   # CI, blocks

--ci turns warnings into failures and drops the verdict cache. Under GitHub Actions the findings are also emitted as annotations, on stderr so they stay out of the report itself.

Tests of the device's own behaviour need nothing but a test file - the package registers as a pytest plugin and supplies the capture fixture:

def test_events_only_appear_inside_a_block(capture):
    assert len(capture.messages("$START")) == len(capture.messages("$STOP"))

Working on the checker

Linux, natively, against an ELF PlatformIO has already built. No container in the loop - the run takes under a second, so the edit-run cycle stays tight:

sudo apt install simavr libsimavr-dev libelf-dev gcc   # once
cd backend
A=~/AIRDOS03/fw/AIRDOS03_USTDFF

python -m packages.ust_format_checker.firmware $A/.pio/build/TFUNIPAYLOAD01_uart/firmware.elf \
    --board $A/xdos/board.yaml --scenario $A/xdos/scenarios/basic.yaml \
    --work-dir /tmp/xdos-dev --no-parser

--work-dir keeps what the run produced: capture.txt, the compiled run.scenario and run.stimulus.json. Reading capture.txt is usually the fastest way to understand a finding.

Most of the test suite needs none of this and runs in under two seconds. The -c /dev/null is there because the DOSPORTAL checkout carries a pytest.ini full of Django plugins this package does not use:

python -m pytest packages/ust_format_checker -q -c /dev/null

To see the checker the way a firmware developer does, run pio run in the device repository with XDOS_CHECKER_PATH pointing at this checkout - see that repository's xdos/README.md.

Board and scenario

A device is described by which components it carries:

# xdos/board.yaml
device: AIRDOS03B
mcu: atmega1284
f_cpu: 8000000
millis_symbol: timer0_millis     # the counter that paces a measurement block
seconds_symbol: rtc_seconds      # the device clock, moved forward along with it
parts:
  - {component: i2c_eeprom, addr: 0x5B, params: {offset: "0800", data: "0011...EEFF"}}
  - {component: sht31, addr: 0x45}
  - {component: spi_adc_pulse, pins: {conv: PB0, reset: PC2}}
  - {component: gnss, pins: {pps: PD4}}

A scenario says what to inject. Values may be rand(min, max) within the component's range; the seed comes from the scenario name, so runs stay reproducible:

name: basic
stop_blocks: 4
sensors:
  sht31:
    temp_c: rand(-40, 85)
    humidity: rand(0, 100)
events:
  at: 1.2
  channels: [0, 12, 12, 40, 63, 64, 100, 500, 1023]
gnss:
  fix_at: 1.0
  unix: 1789560000

Write a rand(min, max) on its own line, never inside a {...} flow mapping: the comma inside the call splits the mapping, so {temp_c: rand(-40, 85)} silently becomes temp_c: 'rand(-40' plus a key named 85). The scenario is rejected with an error that says so.

Skipping the idle wait between measurement blocks means moving millis_symbol forward. A device whose clock runs off something else - AIRDOS03 counts 1PPS pulses - would then have the two drift apart, and the time in the output could not be compared with anything. That is what seconds_symbol is for: both counters move together, so $STOP times stay meaningful and the run still finishes in under a second. Without it the time checks are skipped.

xdos-build turns the two into runner directives plus a stimulus file, and --stimulus then checks that every injected event, sensor reading and GNSS time comes back out of the firmware. A device with no scenario of its own gets one generated from its board.

Runner directives

The compiled intermediate the simulator reads, one directive per line, times in simulated seconds:

warp_u32 reschedules itself when period and count are given, which is how one directive covers a whole run: simavr has only 64 timer slots and the peripherals have to fit in too, so one warp per step would break the simulation.

Generated from YAML, so the only reason to read them is debugging a simulation.