sodatools

A CLI toolkit for columnar text processing. Pipe-friendly, composable tools that operate on whitespace-delimited data from stdin.

pip install sodatools-core

Every tool is available both as a subcommand (soda <tool>) and as a direct sd<tool> entry point (e.g. sdalign, sdkut, sduniq). The sd prefix keeps the direct names out of the way of coreutils.

Tools

Examples

Tag temperature readings into states and convert to events:

soda sample-data weather | soda tag -H1 hot: 2gt25 cold: 2lt15 normal: 2ge15 2le25 \
  | soda events -H1 --gap 5m --print-header

state  start                stop                 duration  count
hot    2024-01-01T04:35:00  2024-01-01T10:55:00  6.3h      77
normal 2024-01-01T11:00:00  2024-01-01T18:00:00  7.0h      85
cold   2024-01-01T18:05:00  2024-01-02T04:30:00  10.4h     126
...

Select columns, compute deltas, and align:

soda sample-data weather | soda kut 1 2 3 -H1 | soda delta 2 3 -H1 | soda align -H1

timestamp            temp  humidity
-------------------  ----  --------
2024-01-01T00:05:00  0.03     -0.29
2024-01-01T00:10:00  0.12      0.51
2024-01-01T00:15:00  0.18     -0.87
...

Clean noisy sensor data — remove MAD outliers and short mode glitches, respecting time gaps:

soda sample-data noisy | soda filter 2:mad 3:run --gap 30s --drop | soda align

Sample every 10th row, compute statistics:

soda sample-data weather | soda sample every 10 -H1 | soda stats -H1

Convert hex register dumps to binary with aligned output:

echo -e "0xff\n0x0a\n0x80" | soda radix --from hex --to bin -b 8 --align

11111111
00001010
10000000

Filter lines where column 2 > 50, show only matching lines:

soda sample-data small | soda coltest 2gt50 -H1

Resample time series to 1-hour intervals, then plot temperature and humidity:

soda sample-data weather | soda sample interval 1h -H1 | soda plot -H1 2 c=red / 3 c=blue

Enforce 3 fields per line, pad short rows, smooth, and align:

soda sample-data small | soda nf 3 -H1 --pad '?' | soda smooth 3 -p 1 | soda align

Common flags

Most tools share these flags:

Plugins

Third-party packages can add new subcommands to soda by registering them under the sodatools.commands entry-point group. Core discovers plugins at startup; any failures are reported to stderr without crashing the CLI.

# your plugin's pyproject.toml
[project.entry-points."sodatools.commands"]
mytool = "sodatools_myplugin.cli:mytool"   # exposes `soda mytool`

[project.scripts]
sdmytool = "sodatools_myplugin.entrypoints:sdmytool"  # optional direct entry

The registered object must be a typer-compatible command function with the same shape as the built-ins (plain function with typer.Option / typer.Argument defaults). Plugin names that collide with built-ins are skipped with a warning.

Changelog

See CHANGELOG.md for release notes.

License

MIT