Skip to main content

CI PyPI Python License

e-serde

Universal structured-data loader: native Rust codecs that decode any popular config format into native Python objects — and, when you ask, into frozen msgspec models. Two verbs, loads/dumps, sync and async twins, one wheel. json-module semantics, strict typing, no surprises.

Part of the EDamLabs eager series: EDamLabs' own e- products (EDamLabs families).

Install

uv add e-serde          # or: pip install e-serde

Usage

import eserde
import msgspec
from eserde import Format
from pathlib import Path

Twelve functions — load/dump whole documents or stream them record by record, in sync and async — with json semantics and keyword-only options. Files are named by Path or plain string path and inferred from the suffix (.json .jsonc .yaml .yml .toml .ini .cfg .conf .csv .tsv); format= accepts a Format member or its name ("json"); pure JSON content sniffs itself, and everything else must declare its format.

Function Input Output
loads bytes | str | Path plain tree — or the model in type=
dumps any object bytes
load str | Path | BinaryIO like loads
dump object → file None
aloads · adumps · aload · adump same awaitables of the same
iloads · idumps records, streamed Iterator of objects / bytes chunks
ailoads · aidumps same AsyncIterator of the same

loads — decode

cfg = eserde.loads(b'{"host": "0.0.0.0", "port": 8080}', format=Format.JSON)
# {'host': '0.0.0.0', 'port': 8080}
class Server(msgspec.Struct, frozen=True):
    host: str
    port: int


srv = eserde.loads(src, format=Format.JSON, type=Server)  # Server(host='0.0.0.0', port=8080)
srv = eserde.loads(ini, format=Format.INI, type=Server, strict=False)  # "8080" → 8080
net = eserde.loads(src, type=Net, dec_hook=lambda t, v: t(v))  # custom fields inside type=
kwarg effect
format= a Format member or its name; inferred from a path, sniffed from pure JSON, required otherwise
type= validate into a Struct, dataclass, TypedDict, attrs or pydantic model; violations raise LoadError
strict=False msgspec coercion — the escape hatch INI needs
object_hook= rewrite every decoded mapping, innermost first (json semantics)
dec_hook= teach type= custom field types (requires type=)
registry= swap the default codec set
embed= inline source:-style references — .md as text, JSON/YAML/TOML/INI/CSV files as decoded trees, recursive, root-confined

dumps — encode

raw = eserde.dumps({"name": "demian", "n": 42}, format=Format.YAML)
# b'name: demian\n"n": 42\n'
from fractions import Fraction

eserde.dumps({"f": Fraction(1, 2)}, format=Format.JSON, encoders={Fraction: str})  # b'{"f":"1/2"}'
eserde.dumps({"f": Fraction(1, 2)}, format=Format.JSON, default=float)  # b'{"f":0.5}'

Every input is normalized through the Jsonable encoder first (datetime → ISO, Enum → value, bytes → base64), so each format sees the same tree.

kwarg effect
format= defaults to Format.JSON
encoders= exact-type hooks, ahead of every built-in; results are re-walked
default= json/orjson-style last resort for unknown types; none → EncoderError
registry= swap the default codec set

load / dump — files

cfg = eserde.load(Path("config.toml"))  # format from the .toml suffix

with open("app.json", "rb") as fh:
    data = eserde.load(fh, format=Format.JSON)  # an open handle needs the format

eserde.dump(cfg, Path("out.jsonc"))  # writes straight to disk → None

Same kwargs as loads / dumps.

async

aloads · adumps · aload · adump — same signatures; I/O and the GIL-free native parse run off the event loop. The Rust codecs release the GIL, so concurrent aloads parallelizes decode across cores.

async def main():
    srv = await eserde.aloads(Path("config.yaml"), type=Server)
    await eserde.adump(srv, Path("copy.json"))

compat — json drop-in

Frameworks duck-type the stdlib module (json_serialize=, renderers, formatters). eserde.compat speaks json.dumps/json.loads exactly, str output; behaviours e-serde cannot share faithfully delegate to the stdlib — slower, never a surprise.

from eserde import compat

compat.dumps({"a": 1}, ensure_ascii=False)  # '{"a":1}'   native compact utf-8
compat.dumps({"a": 1})  # byte-faithful to stdlib json
compat.loads('{"a": 1.5}', parse_float=Decimal)  # delegated to stdlib, never guessed

Errors are the LoaderError family: FormatError (no/unknown format), LoadError (decode or validation), DumpError / EncoderError (encode), CodecError (backend missing). The full guide lives at edamlabs.github.io/e-serde.

Formats and backends

Format Extension Spec Engine (pinned) Notable
JSON .json RFC 8259 msgspec.json 0.21 exact big ints, strict tokens
JSONC .jsonc Deno jsonc jsonc-parser 0.33 comments, trailing commas
YAML .yaml .yml YAML 1.2 core + merge keys saphyr 0.0.12 anchors, <<, bomb-guarded
TOML .toml TOML v1.1 toml-rs 1.1 inf/nan, no null, i64 ints
INI .ini .cfg .conf de-facto rust-ini 0.21 strings; merge on strict=False
CSV .csv RFC 4180 csv 1.4 list[dict], polars-style inference
TSV .tsv RFC 4180 (tab) csv 1.4 same codec, tab-delimited

The contract lives in code — eserde.STANDARDS — and test_standards.py executes every claim against the live codecs and the lockfiles. Bumping an engine or changing a format capability is a deliberate act, never silent drift. Full per-format pages (capabilities, limits, examples): docs → Formats.

Everything Rust is one extension module (eserde._native), compiled by maturin from crates/native. The only runtime dependency is msgspec.

Benchmarks

Median decode of a 100 KB config on CPython 3.14 (release build). Regenerate with make bench.

loads

Format e-serde nearest rival margin
JSON 0.17 ms orjson 0.17 ms ≈ tie — same decoder (msgspec)
YAML 2.20 ms pyyaml C 12–16 ms ≈6× — ruamel 66×
TOML 1.53 ms rtoml 2.3–2.5 ms 1.5× — tomlkit 42×
JSONC 0.79 ms pyjson5 0.39 ms the one format behind (×0.5)
INI 2.00 ms configparser 22 ms 11×
CSV 0.9–1.0 ms polars 1.3–1.7 ms ×1.3–1.8 — and polars never releases the GIL; stdlib csv is comparable and untyped

Because the Rust codecs release the GIL, aloads parallelizes decode: on 10 MB YAML/TOML the async fan-out is ~2× faster than serial sync (JSON stays flat — msgspec's C decoder holds the GIL). More charts in assets/benchmarks/: dumps · typed · async · memory.

Architecture

crates/native/          single Rust cdylib, one submodule per format
src/eserde/
  __init__.py           the one façade: import eserde; eserde.loads(...)
  infra/                contracts with zero internal deps: errors, formats, io, protocols
  backends/             one folder per engine: native/ (Rust), msgspec/ (C) + registry
  logic/                the facade functions: loads/dumps/load/dump/async + Jsonable encoder
tests/
  unit/eserde/          exact mirror of src
  benchmark/            rival matrix + report renderer
  resources/            canonical sample.* fixtures
docs/                   Zensical site

Only the package root has an __init__.py; every subpackage is a namespace folder. Import boundaries are enforced by tach: eserde → logic → backends → infra/_native.

Design rules: decoding is always Rust/C, validation is msgspec's or pydantic's — type= only routes the decoded tree. Round-trip losses are explicit: JSONC comments are dropped on dumps; TOML has no null; YAML non-scalar keys and multi-document streams are rejected.

Development

uv sync                  # installs the dev group, builds the extension in place
make test                # pytest
make check               # ruff + format + ty + tach + pytest (what CI runs)
make bench               # rival benchmark matrix → assets/benchmarks/*.png

Roadmap

Interop landed: eserde is a custom encoder for any framework that ducks-types json (eserde.compat), validates pydantic/attrs models through type=, accepts per-call default=/encoders=/dec_hook= hooks, inlines Markdown bodies and structured sub-documents (embed=), and streams records one at a time (iloads/idumps and async twins). Shipped alongside a comparative concurrency stress harness (make stress).

  • Pending consideration: YAML multi-document streaming (--- boundaries, reusing the event-stream scanner) — viable, but only pays for stream-shaped YAML corpora.
  • Rejected by design: TOML and INI streaming — their semantics (scattered tables, duplicate-section merge) require the global view; the contract says so loudly.
  • Next: broaden interop — msgspec/pydantic request-body and FastAPI response pipelines — and CI-gated regression against rival decoders under sustained load.

License

MIT — see LICENSE.

Metadata

Release files for e-serde 0.6.3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for e-serde 0.6.3
File Size Uploaded Provenance Actions
e_serde-0.6.3.tar.gz 39.0 kB With provenance Details for e_serde-0.6.3.tar.gz

Built distributions (wheels)

Table of built distributions (wheels) for e-serde 0.6.3
File Actions
e_serde-0.6.3-cp314-cp314-win_amd64.whl CPython 3.14 CPython 3.14 Windows x86-64 Details for e_serde-0.6.3-cp314-cp314-win_amd64.whl
e_serde-0.6.3-cp314-cp314-manylinux_2_28_x86_64.whl CPython 3.14 CPython 3.14 Linux glibc 2.28+ x86-64 Details for e_serde-0.6.3-cp314-cp314-manylinux_2_28_x86_64.whl
e_serde-0.6.3-cp314-cp314-manylinux_2_28_aarch64.whl CPython 3.14 CPython 3.14 Linux glibc 2.28+ ARM64 Details for e_serde-0.6.3-cp314-cp314-manylinux_2_28_aarch64.whl
e_serde-0.6.3-cp314-cp314-macosx_11_0_arm64.whl CPython 3.14 CPython 3.14 macOS 11.0+ ARM64 Details for e_serde-0.6.3-cp314-cp314-macosx_11_0_arm64.whl
e_serde-0.6.3-cp314-cp314-macosx_10_12_x86_64.whl CPython 3.14 CPython 3.14 macOS 10.12+ x86-64 Details for e_serde-0.6.3-cp314-cp314-macosx_10_12_x86_64.whl

Total release size: 2.4 MB

Release files / e_serde-0.6.3.tar.gz

Download URL e_serde-0.6.3.tar.gz
Size 39.0 kB
Tags Source
SHA-256 checksum
How to use checksums
5ac7996446c83ab8a7237f2b93c1ed53f20fe5cc966929fefda2408463833cef
BLAKE2b-256 checksum
How to use checksums
e229b61048a3b92fc7bb7aa536aadacbdad8ee5457c2af0649eac93e65ab5dca
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 3, 2026.

Transparency log

Release files / e_serde-0.6.3-cp314-cp314-win_amd64.whl

Download URL e_serde-0.6.3-cp314-cp314-win_amd64.whl
Size 424.5 kB
Tags CPython 3.14 Windows x86-64
SHA-256 checksum
How to use checksums
3850f5768bcd0085bab490e432f929e9d116bf8645aef0ea40b766ac1e3410ee
BLAKE2b-256 checksum
How to use checksums
0662ac8d2f867fe29a3a56390a112d47fca52ad8378620afc963dad41edd9c91
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 3, 2026.

Transparency log

Release files / e_serde-0.6.3-cp314-cp314-manylinux_2_28_x86_64.whl

Download URL e_serde-0.6.3-cp314-cp314-manylinux_2_28_x86_64.whl
Size 516.5 kB
Tags CPython 3.14 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
70cfbb9b33286020ca5054af302b1cff1889fc28bd89390c114bdf74d29b951d
BLAKE2b-256 checksum
How to use checksums
d432db513357235fd1637592e83e2d6c5aeed57fc62b6c85b8ab79fdc269136f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 3, 2026.

Transparency log

Release files / e_serde-0.6.3-cp314-cp314-manylinux_2_28_aarch64.whl

Download URL e_serde-0.6.3-cp314-cp314-manylinux_2_28_aarch64.whl
Size 497.6 kB
Tags CPython 3.14 Linux glibc 2.28+ ARM64
SHA-256 checksum
How to use checksums
589fb3adf86727d910486a08cbe73cb797f8d02c72cffebb63d6df5374933699
BLAKE2b-256 checksum
How to use checksums
df3e24869b83665dc5ca9c4d871caa2df98db7d543df3f37be286dac7b326e32
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 3, 2026.

Transparency log

Release files / e_serde-0.6.3-cp314-cp314-macosx_11_0_arm64.whl

Download URL e_serde-0.6.3-cp314-cp314-macosx_11_0_arm64.whl
Size 467.5 kB
Tags CPython 3.14 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
2644f652407cdd59a273fd253d0f99d5dff9743cb51013f874fadc6fe03b002b
BLAKE2b-256 checksum
How to use checksums
cbc36ac4c5974e81278242ee1ed946335abf52bf3b41e262932f5aee5f707311
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 3, 2026.

Transparency log

Release files / e_serde-0.6.3-cp314-cp314-macosx_10_12_x86_64.whl

Download URL e_serde-0.6.3-cp314-cp314-macosx_10_12_x86_64.whl
Size 496.4 kB
Tags CPython 3.14 macOS 10.12+ x86-64
SHA-256 checksum
How to use checksums
85764570637b348751fc5dfdc7309c1bb7be0a76a05655d902bd1778ff5ed580
BLAKE2b-256 checksum
How to use checksums
85c149822a9b8f4b2774a96c95f123e018b3bc98af45ccc6a3ab27383e725f2a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 3, 2026.

Transparency log

Security for e-serde 0.6.3

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

All 6 files in this release have been uploaded with provenance metadata.

Select from 6 files

Attestation for e_serde-0.6.3.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 3, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page