omnist 0.2.17

One data model, many formats: read, validate, and write JSON, YAML, TOML, XML, and OML.

Omnist

Omnist ("omni-structure") is one canonical data model for JSON, YAML, TOML, XML, and its own native OML (Omnist Markup Language) — read any of them into a single tree, validate it against a schema, compare schema versions, and write it back out to any of the others.

from omnist import parse_schema, doc

s = parse_schema('''
    record Member { "name": string, "role": string }
    record Team   { "name": string, "members" [1,]: Member }
    root Team
''')

s.validate(doc({"name": "Platform",
                "members": [{"name": "Ann", "role": "dev"}]})).ok    # True

Why Omnist

If your service handles config or payloads in more than one format, you usually get a different library — and a different mental model — for each. Omnist gives you one model and one schema language over it, grounded in a small, self-contained formal model (inspired by Lee & Cheung, CIKM 2010):

• A Document is a tree — an ordered list of labeled edges. Arrays are just repeated labels, so the same Document represents JSON, YAML, TOML, XML (including its interleaved repeated elements), and OML — Omnist's own format, the only one with zero loss in either direction.
• A Schema is named record definitions (closed named fields, each with a cardinality), where every field's type is always exactly one fixed scalar (optionally nullable) or one Ref to a named record — referenced by name for reuse and recursion. Written as OSD (Omnist Schema Definition). Validate a Document, compare two schemas for backward-compatibility, infer a schema from examples, or extract the minimal subschema for a subset of labels (paper Algorithm 5).
• Closed by construction — records are closed, and scalar types are never composed into enums or unions. That is not a constraint bolted on top; it is what makes compatible_with, equivalent, normalize, and infer well-defined, decidable operations instead of best-effort heuristics. See why Omnist for the case in full.

The model is defined formally in docs/design/model.md; see the quickstart for the shortest possible example, or the user guide for the practical tour.

A 60-second tour

from omnist import Doc, parse_schema, infer, doc, read_json

# OML is omnist's own format -- see docs/formats/oml.md
Doc.from_oml('id: 1\ntags: "a"\ntags: "b"\n').to_oml()

# converting from other formats is just read one, write another
Doc.from_json('{"id": 1, "tags": ["a", "b"]}').to_yaml()

# describe a shape and check data against it; errors carry exact paths
s = parse_schema('record R { "id": integer, "tags" [0,]: string }\nroot R')
print(s.validate(doc({"id": "x", "tags": ["a"]})))
#   invalid:
#     at $.id: expected integer, got string ('x')

# learn a schema from examples
print(infer([doc({"id": 1, "tags": ["a"]})]).to_osd())
#   record Root {
#       "id": integer,
#       "tags": string,
#   }
#   root Root

# is a schema change backward-compatible? (operations are Schema methods)
v1 = parse_schema('record R { "host": string }\nroot R')
v2 = parse_schema('record R { "host": string, "port" [0,1]: integer }\nroot R')
v1.compatible_with(v2)        # True -- adding an optional field is safe

# schema-directed deserialization: upgrade leaves to match the schema
s2 = parse_schema('record R { "d": date }\nroot R')
read_json('{"d": "2024-01-01"}', schema=s2)   # [('d', datetime.date(2024, 1, 1))]

Run the full demo: python3 examples/canonical_model.py.

Installation

Requires Python 3.11+ (it uses the standard-library tomllib). The core and JSON support have no dependencies.

pip install omnist                      # core + JSON
pip install omnist[all]                 # + pyyaml, tomli_w, defusedxml

Installing also provides an omnist CLI command; see the CLI docs.

Or from a checkout:

git clone https://github.com/omnist-dev/omnist.git
cd omnist
python3 -m venv .venv && source .venv/bin/activate
pip install .                    # core + JSON
pip install pyyaml tomli_w defusedxml   # YAML / writing TOML / hardened XML

Documentation

Full index: docs/, also browsable as a site at omnist.dev.

• Quickstart — the shortest possible example: one OML snippet, one schema, validate(), infer().
• Why Omnist — the differentiation case: a falsifiable thesis, a verified capability matrix against JSON/YAML/TOML/XML, a worked compatible_with comparison against jsonschema, and the honest non-goals (including XML attribute/namespace dropping).
• User guide — the practical tour: documents, OML (the native format), OSD, the Python builder, validation, operations, other codecs, inference.
• OML — Omnist's own format, designed alongside the model so every Document round-trips with zero adjustments, and how it maps onto the Python Document.
• The Schema model & OSD — Omnist's other central feature: record definitions, cardinality, the Python builder, and the comparison/inference operations.
• API reference — every public name, with signatures.
• CLI — the omnist command-line tool.
• Schema-directed deserialization — what changes (and what doesn't) about a Document's Python types when a schema is, vs. isn't, passed to a reader.
• A real-life example — one order schema validated against documents in JSON, YAML, TOML, and XML, plus a compatibility check.
• Formats — how each format maps to the model and its caveats (OML · JSON · YAML · TOML · XML).
• Model spec — the formal Document and Schema models, self-contained and plain (no paper required).
• Glossary — one definition per term used across the docs and code, grouped by concept area.
• Testing — the test suite layout, coverage tooling and target, the fuzzing approach, and what CI runs.
• Repo layout — how the repo itself is organized: omnist/canonical/*.py module responsibilities, the docs page map, and the test file map.

Status

Omnist is alpha, built around a small, self-contained formalism; the public API may still change before a stable release. See the PyPI badge above (or CHANGELOG.md) for the current version.

Feedback and bug reports welcome: https://github.com/omnist-dev/omnist/issues. See SECURITY.md for the trust model if you parse untrusted input.

License

Apache-2.0 — see LICENSE and NOTICE.

Background

The model is inspired by Lee & Cheung, "XML Schema Computations: Schema Compatibility Testing and Subschema Extraction" (CIKM 2010), simplified for the JSON family of formats. You don't need the paper to use Omnist — the model spec is self-contained.