son 0.3.0

Tools to read and write .son files

son | sequential object notation

What is this?

son is a data format that builds on JSON and adds one feature inspired by YAML: concatenation of objects with ---. Optionally, the delimiter === can be used once per son file to delimit metadata.

Why son?

While JSON is perfect for storing structured data, it is inherently impossible to add new portions of data to a file without reading it first. YAML files on the other hand are self extensible by the --- delimiter, but the flexibility YAML offers makes the files inefficient to parse. They are thus unsuited to store significant amounts of data.

son fills the gap by allowing JSON objects to be concatenated with ---. It thus combines the speed and efficiency of JSON with the sequential extensibility of YAML, see example. It further adds to discern metadata from actual data by using ===.

son does not allow to overwrite data. In order to avoid accidental data loss, metada can only be written to fresh files, whereas data can only be appended to files.

Who needs this?

son originated from the need to store computational data that is produced portion by portion on a computer. The requirements were:

• Possible to be read by a human,
• possible to store arbitrary data structures including metadata,
• easy to write and parse by a computer,
• efficient to parse to allow files of up to GB size (takes forever to parse with YAML),
• sequential and incorruptible,
• resilient to data loss.

Example

This is a valid son string:

{
  "purpose": "store biography data",
  "version": 0.1
}
===
{
  "first name": "Hildegard",
  "second name": "Kneef",
  "age": 93
}
---
{
  "first name": "Wiglaf",
  "second name": "Droste",
  "age": 57
}

It will be parsed into the metadata object, and a list containing the data objects with

import son

metadata, data = son.load('file.son')

Changelog

v.0.3.0: support for reading compressed .bz2 and .gz files

v.0.2.5: progressbar is only shown when a terminal is attached (.isatty())

v.0.2.4: progressbar without external dependency

v.0.2.3: progress.bar prints to stdout instead of stderr

v.0.2.2: optionally be verbose and show progressbar with progress package (optional dependency, install with pip install son[progress])