pulse-marshmallow 0.1.0

Rust/PyO3 accelerator for marshmallow 4.x: native Schema.dump/load with transparent fallback to marshmallow.

pulse-marshmallow

A Rust/PyO3 accelerator for marshmallow 4.x — a native Schema.dump/Schema.load that is iso-functional with marshmallow, with transparent fallback to marshmallow for anything outside its fast path. marshmallow is structurally "pydantic v1 in pure Python" (per-field interpreted dispatch) and has no native core; this is a separate package that depends on marshmallow and delegates back to it — so it never diverges, by construction.

pip install pulse-marshmallow

# was:  from marshmallow import Schema, fields
from pulse_marshmallow import Schema, fields

class UserSchema(Schema):
    id   = fields.Int(required=True)
    name = fields.Str(validate=validate.Length(max=64))
    tags = fields.List(fields.Str())
    addr = fields.Nested(AddressSchema)

s = UserSchema()           # compiled once
s.dump(obj)                # native serialization
s.load(payload)            # native validation + deserialization; .errors tree identical to marshmallow

Only the import changes. dump/dumps/load/loads/validate, ValidationError, the .errors tree, hooks, fields and validators all work unchanged.

Performance

marshmallow's dump/load is interpreted per-field dispatch (field×object), spending ~88 % of its time in Python machinery (and ~0 % in re on the common path). pulse-marshmallow compiles the schema once into a Rust plan and iterates natively.

Drift-immune A/B (median per call), pre-built schema, realistic schema (7 fields incl. a nested dict and a list), CPython 3.11, Apple Silicon:

path	marshmallow	pulse (native)	speedup
load (validation + error tree)	~13 µs	~1.5 µs	~×8-9
dump (serialization)	~5 µs	~0.8 µs	~×4-6

load carries the bigger win (validation dispatch + error accumulation); dump is leaner. Validators are executed via Python (the And(*validators) object), so message parity is exact at no measurable cost (the cost is the dispatch, not the comparison).

What's covered natively (and what falls back)

Native fields: Str, Int (+as_string/strict), Float (+as_string/allow_nan), Bool, Raw, Nested, List, Dict (keys/values), and Email/Url (coercion identical to String, regex run via Python). Options: data_key, attribute, required, allow_none, load_default (const), load_only/dump_only, many, unknown (RAISE/EXCLUDE/INCLUDE). Validators (Range/Length/OneOf/ custom/Regexp) run via Python → exact messages, no restriction.

Everything else is transparently delegated to marshmallow: dump hooks (@pre_dump/@post_dump) disable native dump; load hooks (@pre_load/@post_load/@validates/@validates_schema) disable native load; C-bound/computed fields (DateTime/UUID/Decimal/Enum/Method/Function), dump_default, callable load_default, dotted attribute, Nested only/exclude, custom Bool truthy/falsy, explicit partial/unknown, custom dict_class/get_attribute. Note: native acceleration targets marshmallow 4.x; on 3.x it transparently falls back (correct, no speedup).

A key point on coverage: @post_load (the idiomatic "load → build an object") is load-only, so a schema that uses it still gets native dump. Across a sample of real OSS repos, only ~12 % of schemas carry a load hook — hooks are not the limiting factor; the field whitelist is.

Iso-functionality

Proven by a typed differential oracle comparing dump AND load (the result, or the full ValidationError.messages tree) against marshmallow, on a curated corpus plus adversarial fuzzing of random schemas and payloads — including the bool/coercion subtleties, the nested/list/dict error trees, unknown policies, and the @post_load-still-allows-native-dump lever. Error messages and truthy/ falsy sets are read from the field, so custom messages work. The pure-marshmallow fallback path is verified iso too (PULSE_FORCE_FALLBACK=1).

Wheels

abi3 wheels (Python ≥ 3.11) for Linux (x86_64/aarch64, manylinux + musllinux), macOS (Apple Silicon), and Windows; sdist elsewhere (builds the Rust core via maturin).

License

MIT (same as marshmallow).