pj-bridge 0.2.0

Bridge a delimiter-framed TCP binary stream into JSON over WebSocket for PlotJuggler.

pj-bridge

Bridge a delimiter-framed TCP binary stream into JSON over WebSocket for PlotJuggler.

Mode: PlotJuggler runs the WebSocket Server. The bridge connects as a WebSocket client and pushes JSON messages.

What’s in this repo

• derive_struct.py — Parse a C header (typedef struct { ... } Name;) and derive the Python struct format and expanded field labels.
• tcp_parser.py — Connect to the device, parse [DE AD BE EF][COUNT][PAYLOAD] × COUNT batches into NDJSON (one JSON per line).
• socket_client.py — Read NDJSON (stdin or file) and forward to PlotJuggler’s WebSocket Server.
• bridge.py — One-process solution: connect to device, parse, and forward to PlotJuggler (no shell pipes needed).

Requirements

• Python 3.12+

• Device emits batches framed like:

  [ 0xDE 0xAD 0xBE 0xEF ][ COUNT:1 byte ][ PAYLOAD ] * COUNT
  

• The payload is a packed C struct defined in a header file.

• PlotJuggler with the WebSocket Server plugin enabled (Protocol: JSON).

Install

python3.12 -m venv .venv
source .venv/bin/activate
pip install -e .

Start PlotJuggler’s WebSocket Server

• In PlotJuggler: Streaming → WebSocket Server
• Protocol: JSON
• Port: for example 9871
• Click Start

Start Bridge

Connect to device, parse batches, forward to PJ. Default TCP port is 5000; default WS URL is ws://127.0.0.1:9871. Timestamps are in milliseconds by default (--ts-scale 1e-3).

python3 bridge.py \
  --host 192.168.1.91 \
  --struct-header /path/to/telemetry.h \
  --struct-name MyStruct \
  --name-prefix "device_a."

Notes:

• Add --ws-url ws://<pj_host>:9871 if PlotJuggler runs elsewhere.
• If needed, guard against corrupted batches with --max-frames-per-batch N.
• To fall back to single [DELIM][PAYLOAD] (no COUNT), pass --no-counted-batch.

Two-process pipeline (for debugging)

0. Derive the struct (optional sanity check). Prints JSON describing the derived struct_fmt, fields, and record_size.

python3 derive_struct.py \
  --header /path/to/telemetry.h \
  --struct-name MyStruct \

1. Parse from device to NDJSON:

python3 tcp_parser.py \
 --host 192.168.1.91 \
 --struct-header /path/to/telemetry.h \
 --struct-name MyStruct \
 --name-prefix "device_a."

2. Forward NDJSON to PlotJuggler:

python3 tcp_parser.py --host 192.168.1.91 --struct-header /path/to/telemetry.h --struct-name MyStruct | python3 socket_client.py --ws-url ws://127.0.0.1:9871

Field naming

• All non-timestamp fields are emitted with the optional prefix:

  --name-prefix "device_a."
  

  Example JSON:

  {"t": 1727370023.415, "device_a.ax": 0.02, "device_a.ay": -0.01, "device_a.az": 9.81}
  

• Arrays like float gyro[3]; become device_a.gyro[0], device_a.gyro[1], device_a.gyro[2].

Timestamp (t)

• If --ts-field ts_ms is provided, t = ts_ms * --ts-scale (default 1e-3, ms → seconds).
• If no --ts-field is set, arrival time is used (time.time() in seconds).
• If your device time is relative (since boot) and you want wall-clock, you can add an epoch offset in code; ask if you want a ready-made flag for that.

Parsing log files

The project provides two standalone tools for working with telemetry logs:

• file-parser — converts on-device circular binary log files into NDJSON
• json-to-csv — converts NDJSON into CSV

You can generate JSON logs in one of two ways:

1. Live over TCP, using tcp-parser
2. Offline from stored binary log files, using file-parser

Both paths produce NDJSON (one JSON object per line), which can then be piped into json-to-csv for analysis in Excel, Pandas, or visualization tools.

1. Parsing binary circular log files

Devices store telemetry in fixed-size record slots.
file-parser reads these slots, applies the struct layout derived from your C header, and streams out JSON.

Example usage:

file-parser \
  --input ~/Downloads/Current.log.2 \
  --control-file ~/Downloads/Current.control.2 \
  --struct-header $PATH_TO_STRUCT/telemetry.h \
  --struct-name telemetry_t \
  | json-to-csv > telemetry_2.csv

Notes:

• --control-file provides metadata such as record size, checksum size, max-record count, and read/write indices.
• --struct-header and --struct-name define the payload layout for unpacking.
• Output is NDJSON streamed to stdout.

2. Converting NDJSON to CSV

json-to-csv converts streamed JSON objects into a well-formed CSV file.
All JSON lines must contain the same fields (the parsers ensure this).

Example from a live TCP stream:

tcp-parser ... | json-to-csv > live.csv

Example from offline logs:

file-parser ... | json-to-csv > logs.csv

Uninstall

• Deactivate the venv and remove the project directory, or run pip uninstall pj-bridge inside the venv (if installed as a package).

License

MIT