UST Format Checker

Checks that xDOS firmware keeps its output format backward compatible by running the firmware in simavr against a deterministic scenario and inspecting what it prints on UART0.

Plan and requirements: PLAN.md. Per-message rules: CHECKS.md. Where docs, parser and firmware disagree: DOC_DEVIATIONS.md.

Status: phases F1-F3 done (checks, report, CLI, board and scenarios as data, PlatformIO hook), driven against AIRDOS03_USTDFF.

Installing

pip install "ust-format-checker @ git+https://github.com/UniversalScientificTechnologies/DOSPORTAL@<commit>#subdirectory=backend/packages/ust_format_checker"

Needs simavr to run a firmware (apt install simavr libsimavr-dev libelf-dev gcc). Installed commands: xdos-check (a capture), xdos-check-firmware (an ELF, end to end), xdos-build (board + scenario -> runner directives).

Installed standalone, layer 3 is skipped and says so: packages.parsing lives in DOSPORTAL and is not a package of its own yet. Run from a DOSPORTAL checkout to get it.

Layout

• messages.yaml - the message schema: fields, types, ranges, which rules apply.
• schema.py, checks.py, rules.py - schema loading and layer 1 checks.
• compat.py - layer 2: append-only comparison against a golden capture.
• parser_layer.py - layer 3: what packages.parsing reads out of the same log.
• stimulus_checks.py - the value the simulation injected has to be the value that comes out.
• regressions/ - real captures carrying format bugs the checker must keep catching. Only genuine captures belong here; a hand-written one shares its blind spots with the rule it guards, so that case goes in a unit test.
• report.py, cli.py - rustc-style report and the xdos-check entry point.
• components/ - what a component is and which runner directive drives it. Shared by every device.
• Which component sits at which address or pin, and what the simulation injects, belong to the device repository as xdos/board.yaml and xdos/scenarios/*.yaml. A device with no scenario of its own gets one generated from its board.
• scenario.py, build.py - board + scenario -> runner directives + the stimulus to check against.
• simulator/runner.c - loads the firmware ELF into simavr, attaches component models, replays the directives, prints UART0.
• docker/ - a reproducible toolchain (simavr, PlatformIO) for CI and for reproducing a finding on someone else's machine. Day-to-day development does not need it.

Checking a captured log

cd backend
python -m packages.ust_format_checker <log>                        # schema + parser layer
python -m packages.ust_format_checker <log> --golden <reference>   # + compatibility layer
python -m packages.ust_format_checker <log> --stimulus <json>      # + injected values (from xdos-build)
python -m packages.ust_format_checker <log> --no-parser            # schema only
python -m packages.ust_format_checker <log> --warnings-as-errors   # CI profile

Exit code 1 means the log failed. Layer 3 (packages.parsing) never produces an error on its own: the parser can be wrong just as easily as the firmware, so its findings are warnings that say so.

The reference for the compatibility layer is a plain captured log, so a format change shows up as a readable diff in the device repository. Recording one after a deliberate format change:

python -m packages.ust_format_checker <log> --golden xdos/golden/basic.txt --accept

Tests:

python -m pytest packages/ust_format_checker -q

Checking a firmware ELF

The whole chain in one command - this is what the PlatformIO hook in a device repository calls:

xdos-check-firmware firmware.elf --board xdos/board.yaml --scenario xdos/scenarios/basic.yaml \
    --golden xdos/golden/basic.txt --cache .pio/build/<env>/xdos/last.json

--cache skips the run when the ELF has not changed. A missing simulator is reported and exits 0, so a firmware build never fails because the checker could not run.

Working on the checker

Linux, natively, against an ELF PlatformIO has already built. No container in the loop - the run takes under a second, so the edit-run cycle stays tight:

sudo apt install simavr libsimavr-dev libelf-dev gcc   # once
cd backend
A=~/AIRDOS03/fw/AIRDOS03_USTDFF

python -m packages.ust_format_checker.firmware $A/.pio/build/TFUNIPAYLOAD01_uart/firmware.elf \
    --board $A/xdos/board.yaml --scenario $A/xdos/scenarios/basic.yaml \
    --golden $A/xdos/golden/basic.txt --work-dir /tmp/xdos-dev --no-parser

--work-dir keeps what the run produced: capture.txt, the compiled run.scenario and run.stimulus.json. Reading capture.txt is usually the fastest way to understand a finding.

Most of the test suite needs none of this and runs in under a second:

python -m pytest packages/ust_format_checker -q

To see the checker the way a firmware developer does, run pio run in the device repository with XDOS_CHECKER_PATH pointing at this checkout - see that repository's xdos/README.md.

Board and scenario

A device is described by which components it carries:

# xdos/board.yaml
device: AIRDOS03B
mcu: atmega1284
f_cpu: 8000000
millis_symbol: timer0_millis     # the counter that paces a measurement block
seconds_symbol: rtc_seconds      # the device clock, moved forward along with it
parts:
  - {component: i2c_eeprom, addr: 0x5B, params: {offset: "0800", data: "0011...EEFF"}}
  - {component: sht31, addr: 0x45}
  - {component: spi_adc_pulse, pins: {conv: PB0, reset: PC2}}
  - {component: gnss, pins: {pps: PD4}}

A scenario says what to inject. Values may be rand(min, max) within the component's range; the seed comes from the scenario name, so runs stay reproducible:

name: basic
stop_blocks: 2
sensors:
  sht31:
    temp_c: rand(-40, 85)
    humidity: rand(0, 100)
events:
  at: 1.2
  channels: [0, 12, 12, 40, 63, 64, 100, 500, 1023]
gnss:
  fix_at: 1.0
  unix: 1789560000

Write a rand(min, max) on its own line, never inside a {...} flow mapping: the comma inside the call splits the mapping, so {temp_c: rand(-40, 85)} silently becomes temp_c: 'rand(-40' plus a key named 85). The scenario is rejected with an error that says so.

Skipping the idle wait between measurement blocks means moving millis_symbol forward. A device whose clock runs off something else - AIRDOS03 counts 1PPS pulses - would then have the two drift apart, and the time in the output could not be compared with anything. That is what seconds_symbol is for: both counters move together, so $STOP times stay meaningful and the run still finishes in under a second. Without it the time checks are skipped.

xdos-build turns the two into runner directives plus a stimulus file, and --stimulus then checks that every injected event, sensor reading and GNSS time comes back out of the firmware. A device with no scenario of its own gets one generated from its board.

Runner directives

The compiled intermediate the simulator reads, one directive per line, times in simulated seconds:

Generated from YAML, so the only reason to read them is debugging a simulation.