teptris-py — TOML for Python at libleptris speed

Mission: be the best TOML parser and builder ever made — in C, Ruby, Rust, and JS — measured, not claimed. The bar and the family ledger: BEST-EVER.md.

A native C-extension binding (no ctypes, no fallback) over libteptris — the TOML counterpart of leptris-py (XML) and yeptris-py (YAML). API shape mirrors tomllib/tomli (loads/load raising TOMLDecodeError) plus a dumps writer in the tomli_w spirit. Datetime mapping mirrors tomllib: aware/naive datetime, date, time.

Both directions are native: dumps walks the object tree in C through the builder API and emits via the shared engine emitter — 3.5×–14.2× faster than tomli_w on every benchmark shape.

dumps_json_natural(obj) emits the host JSON view instead of TOML (engine 0.3.0): real numbers, booleans, RFC 3339 datetime strings, non-finite floats as null — feedable straight into json.loads, and the engine's fastest emit mode. The engine implements the TOML 1.1 draft grammar (a strict superset of 1.0) with 100% toml-test conformance, and parses at ≥3× the best C/C++ competitor on every shape.

Performance (end-to-end, Python tier)

Best-of-12 runs on a GitHub ubuntu-latest runner (python 3.12), 2026-09-21, over the generated bench corpus (~700 KB per shape). The full per-shape table regenerates on every main push via the lang-tier lane.

End to end (parse + materialize into Python objects): 4.8-6x rtoml (the rust-backed incumbent), 19-42x the stdlib's tomllib, and hundreds of times tomlkit per shape. rtoml numbers include its own object construction; both libraries materialize fully.

Packaging

pip install teptris requires CPython >= 3.10 and resolves one of the version-specific wheels — cp310 through cp314 (+ free-threaded cp314t) across manylinux/musllinux x86_64 and aarch64, macOS arm64 and x86_64, Windows AMD64 and ARM64, plus a cp312 wheel for the emulated arches (armv7l, ppc64le, s390x). Every wheel builds against the full C API — no limited-API line — so borrowed UTF-8 reads and the datetime.h fast paths ship everywhere. The extension statically links libteptris, so each wheel is a single self-contained module (no shared-library chain). Linux wheels build the engine inside each container via cibuildwheel so musllinux links musl; the manylinux wheels carry the maximal tag set (glibc >= 2.17).

Every wheel also carries the engine C SOURCE at teptris/_engine (beside the compiled extension) — the recompile path for rebuilding against your own environment. The sdist vendors the same tree and compiles automatically at install.

Every other platform (FreeBSD, Solaris/illumos, any CPython on an exotic arch) installs the sdist, which vendors the engine and compiles it with the installing interpreter's compiler — a C compiler is required, and the build fails loudly without one.

Many-small batch (loads_batch)

Parse a list of TOML documents in ONE C call (teptris_parse_batch):

docs = teptris.loads_batch([toml_str_a, toml_str_b, toml_str_c])
# => [dict, dict, dict] — same datetime contract as loads()

The first failing document raises TOMLDecodeError carrying its line/column. Honest perf shape (benchmark/batch_tier.py): per-doc loads in a tight loop is faster on every measured shape (6.3x on 2000 tiny docs, 3.2x medium, 1.3x large — CPython's per-call overhead amortizes better than the batch's scratch + object pressure). Use loads_batch for the single API surface, one C crossing, and the per-doc error report — not for speed.

Lazy load (loads_lazy)

When most of the parsed tree is going to be ignored, loads_lazy returns a LazyNode wrapper that materializes host objects only along the paths actually accessed:

doc = teptris.loads_lazy(big_toml)
# one parse, no dicts/lists materialized yet
sku = doc["items"][0]["sku"].value()   # walks the path; scalars
                                       # materialize at .value()
rest = doc["items"]                    # returns a LazyNode for the array
for it in rest:
    print(it["sku"].value())
# flatten eagerly if you want the same dict-tree shape as loads()
flat = doc.to_dict()

LazyNode exposes kind() ('table' | 'array' | 'scalar'), value() (scalars only — errors on containers), len() (containers), iteration (tables yield (key, LazyNode) pairs; arrays yield LazyNodes), [] (table key or array index — None for absent), and to_dict() / to_list() for eager flattening. Datetime values materialize via the same datetime.datetime / date / time contract as loads(). The document and the borrowed-bytes input are kept alive by the wrapper until GC.

Same datetime contract as loads(). The eager path is the floor for "flatten everything" workloads; the lazy path is the win for "parse once, touch a small subset."

Descriptor (planned-key materialization)

teptris.Descriptor (the twin of teptris-ruby's Teptris::Descriptor) compiles a plan tree once, then materializes a whole document against it in one native pass — unplanned keys are never materialized:

desc = teptris.Descriptor.build({"children": [
    {"name": "name", "kind": "scalar"},
    {"name": "port", "kind": "scalar"},
    {"name": "hosts", "kind": "collection"},          # array of scalars
    {"name": "items", "kind": "nested", "plan": {     # recurses; spans
        "children": [{"name": "id", "kind": "scalar"}]},  # arrays of tables
    {"name": "everything_else", "kind": "raw"},       # untouched subtree
]})
doc = desc.walk(toml_string)
# {"name": "svc", "port": 8080, "hosts": [...], "items": [{"id": 1}, ...],
#  "everything_else": {...} or None}

Rows absent from the document read as None; keys not in the plan never appear. walk raises TOMLDecodeError with line/column like every other public path.

Typing

The package ships py.typed (PEP 561) with a _native.pyi stub for the C surface — mypy/pyright strict consumers type-check out of the box, and a mypy gate runs in CI. teptris.engine_version() reports the linked libteptris version (the C core, not the package version). teptris.loads_lazy_batch([...]) is the lazy twin of loads_batch.

Development

setup.py links the engine statically when it finds libteptris.a/teptris.lib under TEPTRIS_LIBDIR (what the wheels ship); otherwise it falls back to the shared library with an rpath to its build tree. Point TEPTRIS_INCLUDE/TEPTRIS_LIBDIR at a C checkout's build and run the suite:

cmake -B ../teptris/build -S ../teptris \
    -DCMAKE_BUILD_TYPE=Release -DTEPTRIS_BUILD_SHARED=OFF -DTEPTRIS_BUILD_STATIC=ON
cmake --build ../teptris/build
TEPTRIS_INCLUDE=../teptris/src/include TEPTRIS_LIBDIR=../teptris/build/src \
    python3 setup.py build_ext --inplace
PYTHONPATH=src python3 -m unittest discover -s tests -v

Publishing and version numbers are USER release decisions.