son 0.3.3

Tools to read and write .son files

son | sequential object notation

son is a data format that builds on JSON and adds one feature inspired by YAML: concatenation of objects with ---. Optionally, the delimiter === can be used once per son file to delimit metadata.

---

Motivation

Why son?

While JSON is perfect for storing structured data, it is inherently impossible to add new portions of data to a file without reading it first. YAML files on the other hand are self extensible by the --- delimiter, but the flexibility YAML offers makes the files inefficient to parse. They are thus unsuited to store significant amounts of data.

son fills the gap by allowing JSON objects to be concatenated with ---. It thus combines the speed and efficiency of JSON with the sequential extensibility of YAML, see example. It further adds to discern metadata from actual data by using ===.

son does not allow to overwrite data. In order to avoid accidental data loss, metada can only be written to fresh files, whereas data can only be appended to files.

Who needs this?

son originated from the need to store computational data that is produced portion by portion on a computer. The requirements were:

• Possible to be read by a human,
• possible to store arbitrary data structures including metadata,
• easy to write and parse by a computer,
• efficient to parse to allow files of up to GB size (takes forever to parse with YAML),
• sequential and incorruptible,
• resilient to data loss.

son is targeted at users who would like to store their data in a format meeting these requirements.

---

Installation

son can be installed from pypi via

pip install son

---

Example

This is a valid son string:

{
  "purpose": "store biography data",
  "version": 0.1
}
===
{
  "first name": "Hildegard",
  "second name": "Kneef",
  "age": 93
}
---
{
  "first name": "Wiglaf",
  "second name": "Droste",
  "age": 57
}

It will be parsed into the metadata object, and a list containing the data objects with

>>> import son
>>> metadata, data = son.load('test.son')
>>> print(metadata)
{'purpose': 'store biography data', 'version': 0.1}
>>> print(data)
[{'first name': 'Hildegard', 'second name': 'Kneef', 'age': 93}, {'first name': 'Wiglaf', 'second name': 'Droste', 'age': 57}]

---

Changelog

v0.3.3: Add documentation via mkdocs and mkdocs-material

v0.3.2: fix for interactively working in ipython console

v0.3.1: inform before file is read, makes more sense when that takes some time

v0.3.0: support for reading compressed .bz2 and .gz files

v0.2.5: progressbar is only shown when a terminal is attached (.isatty())

v0.2.4: progressbar without external dependency

v0.2.3: progress.bar prints to stdout instead of stderr

v0.2.2: optionally be verbose and show progressbar with progress package (optional dependency, install with pip install son[progress])