zelos-log-lines

Turns what a device or program prints into Zelos log rows and signals. Zelos extensions that read text, such as a serial console or a board's journal, share it.

Use

import time

import zelos_sdk
from zelos_log_lines.stream import LogStream

source = zelos_sdk.TraceSource("acme")
stream = LogStream(source, "board", notes="acme")

stream.feed(b"[00:00:12.340,000] <wrn> dcdc: rail=13.73V\n", time.time_ns())
stream.close()

That line becomes a warn row in board/log, named dcdc, placed by the board's own uptime, and the value rail becomes a signal in board/dcdc.

A LogStream has two ways in:

• feed(data, host_ns) takes bytes as they arrive from a port or a pipe. It splits them into lines with a terminal's rules: colors, \r, erased lines, and lines over 4 KB.
• write(text, host_ns, ...) takes a line that another framing produced, such as a journal entry's message, with what its source knows: default_level applies only when the line prints no level, while name, file, line and time_ns are facts that beat what the text says.

note(level, message, host_ns) writes the extension's own row, state() reports counters and signals, and close() writes what is still held. A reader whose connection closes calls end().

A record printed over several lines, such as a traceback or a message continued under its header, is one row placed at its first line. The stream holds the last record until the next one starts, or until the source has been quiet for 100 ms, and writes part of a line after 100 ms with no bytes; a timer of the stream's own does both, so a caller never flushes. The keep and on_line callbacks run under the stream's lock, so on that timer's thread too for a line written after a pause. Part of a line, cut or ended by a pause, joins nothing. A blank line neither ends a record nor joins it, as a terminal shows it.

Two hooks let an extension take part. keep sees each fed line first and returns it, a changed line, or None to leave it out, such as a shell prompt. on_line sees each line as soon as it is read, with what that line says on its own, before keep sees the next. An unexpected error, in the stream or a hook, costs at most its line, and a line keep fails on is written as it came. The first error is logged, then one a minute, and state().errors counts each.

What it reads

A level comes only from a line's header, never from a word in its sentence. A line in no known format is a plain info line, and its terminal color sets its level.

Printed values become signals, one event per module: key=value, Teleplot, the Arduino plotter, and a repeated label: value. Values come only from whole lines.

Time

A printed uptime places a live line on the host clock, through a mapping that follows drift and notices restarts (device_time=False places every line by its arrival). A wall-clock stamp never places a live row. A source that knows a row's time, such as the journal, passes time_ns.

Saved files

files.convert(path) writes a saved log file to a Zelos trace beside it, as Logs/<file>/log and its signals. files.replay(path, stream, stop) writes it into a live stream at the pace it was printed, its first line now. Plain text reads, and so does text compressed with gzip, bzip2, xz or zstd, in UTF-8 or in UTF-16 led by its mark, with lines that end as a terminal ends them. A conversion reads the same however long it takes: its records end at the next record, not after a pause.

A saved file's row goes where its line printed it:

• A stamp with a zone is placed as printed. A stamp with none is read in this computer's zone, or the Timing zone given.
• A stamp with no year takes the year of the file's last change, stepping back a year wherever the stamps wrap. A time with no date takes the date of the dated stamp before it, or else does the same with the date.
• A file that prints only its device's uptime ends at its last change, or starts at the Timing start given; after a restart, a run follows the one before.
• A line that prints no time follows the line before it by 1 µs.

Develop

just install   # uv sync --locked
just check     # ruff, pyright, pytest

Most formats are one regex with named groups, a row in a formats/ module. Syslog, logfmt, JSON and plain text are code. A row in native.py or managed.py names the characters its lines can start with, so a line meets only the rows that could read it. formats.read(text) returns the LogRecord for one line, with the format that read it.

tests/oracles/ holds the sample lines of lnav's format catalog and of Elastic Filebeat's module tests, with the level each gives, under their own licenses. No line may read above its level, and differ.json names each line that reads differently, with why.

To add a format, add its row, then tests/vectors/<format>.txt with at least 15 lines it must read and 10 it must not. Each line reads input ⇒ {expected}, and a --- block is one case whose lines share history. tests/zoo/ holds lines that real logging libraries printed, with what each program logged.