Skip to main content

CI PyPI Python License

e-serde

Universal structured-data loader: native Rust codecs that decode any popular config format into native Python objects — and, when you ask, into frozen msgspec models. Two verbs, loads/dumps, sync and async twins, one wheel. json-module semantics, strict typing, no surprises.

Part of the EDamLabs eager series: EDamLabs' own e- products (EDamLabs families).

Install

uv add e-serde          # or: pip install e-serde

Usage

import eserde
import msgspec
from eserde import Format
from pathlib import Path

Twelve functions — load/dump whole documents or stream them record by record, in sync and async — with json semantics and keyword-only options. Files are named by Path or plain string path and inferred from the suffix (.json .jsonc .yaml .yml .toml .ini .cfg .conf .csv .tsv); format= accepts a Format member or its name ("json"); pure JSON content sniffs itself, and everything else must declare its format.

Function Input Output
loads bytes | str | Path plain tree — or the model in type=
dumps any object bytes
load str | Path | BinaryIO like loads
dump object → file None
aloads · adumps · aload · adump same awaitables of the same
iloads · idumps records, streamed Iterator of objects / bytes chunks
ailoads · aidumps same AsyncIterator of the same

loads — decode

cfg = eserde.loads(b'{"host": "0.0.0.0", "port": 8080}', format=Format.JSON)
# {'host': '0.0.0.0', 'port': 8080}
class Server(msgspec.Struct, frozen=True):
    host: str
    port: int


srv = eserde.loads(src, format=Format.JSON, type=Server)  # Server(host='0.0.0.0', port=8080)
srv = eserde.loads(ini, format=Format.INI, type=Server, strict=False)  # "8080" → 8080
net = eserde.loads(src, type=Net, dec_hook=lambda t, v: t(v))  # custom fields inside type=
kwarg effect
format= a Format member or its name; inferred from a path, sniffed from pure JSON, required otherwise
type= validate into a Struct, dataclass, TypedDict, attrs or pydantic model; violations raise LoadError
strict=False msgspec coercion — the escape hatch INI needs
object_hook= rewrite every decoded mapping, innermost first (json semantics)
dec_hook= teach type= custom field types (requires type=)
registry= swap the default codec set
embed= inline source:-style references — .md as text, JSON/YAML/TOML/INI/CSV files as decoded trees, recursive, root-confined

dumps — encode

raw = eserde.dumps({"name": "demian", "n": 42}, format=Format.YAML)
# b'name: demian\n"n": 42\n'
from fractions import Fraction

eserde.dumps({"f": Fraction(1, 2)}, format=Format.JSON, encoders={Fraction: str})  # b'{"f":"1/2"}'
eserde.dumps({"f": Fraction(1, 2)}, format=Format.JSON, default=float)  # b'{"f":0.5}'

Every input is normalized through the Jsonable encoder first (datetime → ISO, Enum → value, bytes → base64), so each format sees the same tree.

kwarg effect
format= defaults to Format.JSON
encoders= exact-type hooks, ahead of every built-in; results are re-walked
default= json/orjson-style last resort for unknown types; none → EncoderError
registry= swap the default codec set

load / dump — files

cfg = eserde.load(Path("config.toml"))  # format from the .toml suffix

with open("app.json", "rb") as fh:
    data = eserde.load(fh, format=Format.JSON)  # an open handle needs the format

eserde.dump(cfg, Path("out.jsonc"))  # writes straight to disk → None

Same kwargs as loads / dumps.

async

aloads · adumps · aload · adump — same signatures; I/O and the GIL-free native parse run off the event loop. The Rust codecs release the GIL, so concurrent aloads parallelizes decode across cores.

async def main():
    srv = await eserde.aloads(Path("config.yaml"), type=Server)
    await eserde.adump(srv, Path("copy.json"))

compat — json drop-in

Frameworks duck-type the stdlib module (json_serialize=, renderers, formatters). eserde.compat speaks json.dumps/json.loads exactly, str output; behaviours e-serde cannot share faithfully delegate to the stdlib — slower, never a surprise.

from eserde import compat

compat.dumps({"a": 1}, ensure_ascii=False)  # '{"a":1}'   native compact utf-8
compat.dumps({"a": 1})  # byte-faithful to stdlib json
compat.loads('{"a": 1.5}', parse_float=Decimal)  # delegated to stdlib, never guessed

Errors are the LoaderError family: FormatError (no/unknown format), LoadError (decode or validation), DumpError / EncoderError (encode), CodecError (backend missing). The full guide lives at edamlabs.github.io/e-serde.

Formats and backends

Format Extension Spec Engine (pinned) Notable
JSON .json RFC 8259 msgspec.json 0.21 exact big ints, strict tokens
JSONC .jsonc Deno jsonc jsonc-parser 0.33 comments, trailing commas
YAML .yaml .yml YAML 1.2 core + merge keys saphyr 0.0.12 anchors, <<, bomb-guarded
TOML .toml TOML v1.1 toml-rs 1.1 inf/nan, no null, i64 ints
INI .ini .cfg .conf de-facto rust-ini 0.21 strings; merge on strict=False
CSV .csv RFC 4180 csv 1.4 list[dict], polars-style inference
TSV .tsv RFC 4180 (tab) csv 1.4 same codec, tab-delimited

The contract lives in code — eserde.STANDARDS — and test_standards.py executes every claim against the live codecs and the lockfiles. Bumping an engine or changing a format capability is a deliberate act, never silent drift. Full per-format pages (capabilities, limits, examples): docs → Formats.

Everything Rust is one extension module (eserde._native), compiled by maturin from crates/native. The only runtime dependency is msgspec.

Benchmarks

Median decode of a 100 KB config on CPython 3.14 (release build). Regenerate with make bench.

loads

Format e-serde nearest rival margin
JSON 0.17 ms orjson 0.17 ms ≈ tie — same decoder (msgspec)
YAML 2.20 ms pyyaml C 12–16 ms ≈6× — ruamel 66×
TOML 1.53 ms rtoml 2.3–2.5 ms 1.5× — tomlkit 42×
JSONC 0.79 ms pyjson5 0.39 ms the one format behind (×0.5)
INI 2.00 ms configparser 22 ms 11×
CSV 0.9–1.0 ms polars 1.3–1.7 ms ×1.3–1.8 — and polars never releases the GIL; stdlib csv is comparable and untyped

Because the Rust codecs release the GIL, aloads parallelizes decode: on 10 MB YAML/TOML the async fan-out is ~2× faster than serial sync (JSON stays flat — msgspec's C decoder holds the GIL). More charts in assets/benchmarks/: dumps · typed · async · memory.

Architecture

crates/native/          single Rust cdylib, one submodule per format
src/eserde/
  __init__.py           the one façade: import eserde; eserde.loads(...)
  infra/                contracts with zero internal deps: errors, formats, io, protocols
  backends/             one folder per engine: native/ (Rust), msgspec/ (C) + registry
  logic/                the facade functions: loads/dumps/load/dump/async + Jsonable encoder
tests/
  unit/eserde/          exact mirror of src
  benchmark/            rival matrix + report renderer
  resources/            canonical sample.* fixtures
docs/                   Zensical site

Only the package root has an __init__.py; every subpackage is a namespace folder. Import boundaries are enforced by tach: eserde → logic → backends → infra/_native.

Design rules: decoding is always Rust/C, validation is msgspec's or pydantic's — type= only routes the decoded tree. Round-trip losses are explicit: JSONC comments are dropped on dumps; TOML has no null; YAML non-scalar keys and multi-document streams are rejected.

Development

uv sync                  # installs the dev group, builds the extension in place
make test                # pytest
make check               # ruff + format + ty + tach + pytest (what CI runs)
make bench               # rival benchmark matrix → assets/benchmarks/*.png

Roadmap

Interop landed: eserde is a custom encoder for any framework that ducks-types json (eserde.compat), validates pydantic/attrs models through type=, accepts per-call default=/encoders=/dec_hook= hooks, inlines Markdown bodies and structured sub-documents (embed=), and streams records one at a time (iloads/idumps and async twins). Shipped alongside a comparative concurrency stress harness (make stress).

  • Pending consideration: YAML multi-document streaming (--- boundaries, reusing the event-stream scanner) — viable, but only pays for stream-shaped YAML corpora.
  • Rejected by design: TOML and INI streaming — their semantics (scattered tables, duplicate-section merge) require the global view; the contract says so loudly.
  • Next: broaden interop — msgspec/pydantic request-body and FastAPI response pipelines — and CI-gated regression against rival decoders under sustained load.

License

MIT — see LICENSE.

Metadata

Release files for e-serde 0.6.4

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for e-serde 0.6.4
File Size Uploaded Provenance Actions
e_serde-0.6.4.tar.gz 39.1 kB With provenance Details for e_serde-0.6.4.tar.gz

Built distributions (wheels)

Table of built distributions (wheels) for e-serde 0.6.4
File Actions
e_serde-0.6.4-cp314-cp314-win_amd64.whl CPython 3.14 CPython 3.14 Windows x86-64 Details for e_serde-0.6.4-cp314-cp314-win_amd64.whl
e_serde-0.6.4-cp314-cp314-manylinux_2_28_x86_64.whl CPython 3.14 CPython 3.14 Linux glibc 2.28+ x86-64 Details for e_serde-0.6.4-cp314-cp314-manylinux_2_28_x86_64.whl
e_serde-0.6.4-cp314-cp314-manylinux_2_28_aarch64.whl CPython 3.14 CPython 3.14 Linux glibc 2.28+ ARM64 Details for e_serde-0.6.4-cp314-cp314-manylinux_2_28_aarch64.whl
e_serde-0.6.4-cp314-cp314-macosx_11_0_arm64.whl CPython 3.14 CPython 3.14 macOS 11.0+ ARM64 Details for e_serde-0.6.4-cp314-cp314-macosx_11_0_arm64.whl
e_serde-0.6.4-cp314-cp314-macosx_10_12_x86_64.whl CPython 3.14 CPython 3.14 macOS 10.12+ x86-64 Details for e_serde-0.6.4-cp314-cp314-macosx_10_12_x86_64.whl

Total release size: 2.4 MB

Release files / e_serde-0.6.4.tar.gz

Download URL e_serde-0.6.4.tar.gz
Size 39.1 kB
Tags Source
SHA-256 checksum
How to use checksums
ff9bf31c69fcff6495d77fe022a4cba8111774d5f17169a6c7a268577604f99d
BLAKE2b-256 checksum
How to use checksums
d6b955eadcc0093f22c8bf5ab13cc22647650780847d45214371841d6ed5b0d5
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 3, 2026.

Transparency log

Release files / e_serde-0.6.4-cp314-cp314-win_amd64.whl

Download URL e_serde-0.6.4-cp314-cp314-win_amd64.whl
Size 424.6 kB
Tags CPython 3.14 Windows x86-64
SHA-256 checksum
How to use checksums
4a4aa8fe2ab254f4cbd7482a0d45119e735817ba4c5d6f80bbc1a12fcbb03c23
BLAKE2b-256 checksum
How to use checksums
741aff8bb8bb9e2b16b38da480db6ff0ecc92651adaefea282ed6b7f3a3c24ef
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 3, 2026.

Transparency log

Release files / e_serde-0.6.4-cp314-cp314-manylinux_2_28_x86_64.whl

Download URL e_serde-0.6.4-cp314-cp314-manylinux_2_28_x86_64.whl
Size 516.6 kB
Tags CPython 3.14 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
fec9f9fe85129a8bd2995aeb1b59f93596d574623bb5d34092692ded7cdb6dcf
BLAKE2b-256 checksum
How to use checksums
e9c3ce717de3c05c4dc6c0e5e515f9a135dc3373c3604d5ace3f054b0dd15c3a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 3, 2026.

Transparency log

Release files / e_serde-0.6.4-cp314-cp314-manylinux_2_28_aarch64.whl

Download URL e_serde-0.6.4-cp314-cp314-manylinux_2_28_aarch64.whl
Size 497.7 kB
Tags CPython 3.14 Linux glibc 2.28+ ARM64
SHA-256 checksum
How to use checksums
88b5b98466fe9a2e69793c2e8147ce7adb82b0531832b85ec9f8271d5b45625c
BLAKE2b-256 checksum
How to use checksums
eccee7d90b02e4b3b24b04489f35f94580a475ec299f63e77f65ce31106aeb65
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 3, 2026.

Transparency log

Release files / e_serde-0.6.4-cp314-cp314-macosx_11_0_arm64.whl

Download URL e_serde-0.6.4-cp314-cp314-macosx_11_0_arm64.whl
Size 467.6 kB
Tags CPython 3.14 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
69b7db23527c1e6545f5e19295f8e5db49c2d0fc65bbb898fd83bbc3cfe02e25
BLAKE2b-256 checksum
How to use checksums
2a45ce52573477caacf3b4b6c7ca026050137cdf8be0d547af6851a8f01ccae1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 3, 2026.

Transparency log

Release files / e_serde-0.6.4-cp314-cp314-macosx_10_12_x86_64.whl

Download URL e_serde-0.6.4-cp314-cp314-macosx_10_12_x86_64.whl
Size 496.4 kB
Tags CPython 3.14 macOS 10.12+ x86-64
SHA-256 checksum
How to use checksums
c8daf2a37013a0c15e41d612de91ae767d200fdfe7fd5b7ff66a19239dad9660
BLAKE2b-256 checksum
How to use checksums
399fad9499d439786471cdc7d9a1983f5dc11ffe83cec4cfd09cdbaf955323a7
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 3, 2026.

Transparency log

Security for e-serde 0.6.4

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

All 6 files in this release have been uploaded with provenance metadata.

Select from 6 files

Attestation for e_serde-0.6.4.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 3, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page