MSCS — Safe Serialization for Python

v2.5.0 | Changelog | PyPI

Status: Beta — API is stable but the format may evolve. Not yet battle-tested in large-scale production.

A secure, fast, binary serialization library. Drop-in replacement for pickle that does not execute arbitrary code during deserialization of unregistered classes.

Built for AI/ML workflows — native support for NumPy arrays and PyTorch tensors with zero-copy performance.

Why not pickle?

# pickle: arbitrary code execution on load
data = pickle.loads(untrusted_bytes)  # can run os.system("rm -rf /")

# mscs: only reconstructs explicitly registered classes
data = mscs.loads(untrusted_bytes)    # MSCSecurityError if class not registered

Comparison with Alternatives

* mscs executes __setstate__ on registered classes. See Security Model for details.

When to use safetensors instead: If you only need to serialize tensors and arrays (model weights, embeddings), safetensors is the industry standard — it's written in Rust, truly zero-code-execution, and widely adopted. Use mscs when you need to serialize mixed Python objects (configs, custom classes, nested structures) alongside tensors.

Install

pip install mscs              # core (no dependencies)
pip install mscs[numpy]       # + numpy support
pip install mscs[torch]       # + numpy + PyTorch tensor support
pip install mscs[all]         # everything

Quick Start

import mscs

# Primitives, collections, nested structures — just works
data = {"model": "v5.2", "lr": 0.001, "layers": [64, 128, 256]}
encoded = mscs.dumps(data)
decoded = mscs.loads(encoded)

# NumPy arrays
import numpy as np
arr = np.random.randn(100, 100).astype(np.float32)
encoded = mscs.dumps(arr)

# PyTorch tensors — no .numpy() conversion needed
import torch
weights = torch.randn(256, 256)
encoded = mscs.dumps(weights)  # safe, no pickle involved

# Full model checkpoints
checkpoint = {
    "epoch": 100,
    "model_state": {k: v for k, v in model.state_dict().items()},
    "optimizer_lr": 0.0003,
}
mscs.dump(checkpoint, open("checkpoint.mscs", "wb"))
restored = mscs.load(open("checkpoint.mscs", "rb"))

Custom Classes

import mscs
from dataclasses import dataclass

@mscs.register
@dataclass
class Config:
    state_size: int = 256
    lr: float = 0.001

config = Config(512, 0.0003)
data = mscs.dumps(config)
restored = mscs.loads(data)  # Config(state_size=512, lr=0.0003)

# Unregistered classes raise MSCSecurityError in strict mode
mscs.loads(data_with_unknown_class)  # MSCSecurityError

# Or get a dict fallback in non-strict mode
mscs.loads(data_with_unknown_class, strict=False)  # {'__class__': '...', '__state__': {...}}

Backward Compatibility with Renamed Classes

mscs.register_alias("my_module.OldConfig", Config)

Register All Classes in a Module

import my_models
mscs.register_module(my_models)

Compression & Integrity

# zlib compression
with open("data.mscs.z", "wb") as f:
    mscs.dump_compressed(large_obj, f)

with open("data.mscs.z", "rb") as f:
    obj = mscs.load_compressed(f)

# CRC32 integrity check (detects accidental corruption, NOT tamper-proof)
data = mscs.dumps(obj, with_crc=True)
mscs.loads(data)  # verifies CRC, raises MSCDecodeError if corrupted

# HMAC-SHA256 authentication (cryptographic, tamper-proof)
key = b'your-secret-key-here'
data = mscs.dumps(obj, hmac_key=key)
mscs.loads(data, hmac_key=key)          # verifies HMAC, raises MSCSecurityError if tampered
mscs.loads(data)                         # MSCSecurityError: no key provided for signed payload
mscs.loads(unsigned_data, hmac_key=key)  # MSCSecurityError: anti-downgrade protection

API Reference

Core

Registry

Utilities

Supported Types

Performance

Benchmarked on a single machine (results may vary by hardware and payload):

Payload: state_dict with 4 tensors (~57K parameters, dominated by contiguous float32 buffers):

mscs is fast for tensor-heavy payloads because it writes raw buffers with minimal framing overhead. For small, nested Python structures (dicts, strings, configs), the speedup is smaller. Always benchmark with your actual data.

Run python tests/benchmark.py to reproduce on your machine.

Security Model

mscs provides a defense-in-depth approach, but it is not a sandbox. Understand the boundaries:

What mscs prevents

1. No dynamic imports: Class names in the binary stream are only used as registry lookup keys — never passed to importlib
2. Explicit registry: Custom classes must be registered before deserialization; unregistered classes raise MSCSecurityError
3. NumPy dtype whitelist: Blocks object, void, and structured dtypes that could execute code
4. Total-size and depth limits: loads()/load() reject any blob larger than max_size (default MAX_SIZE=512MB) before decoding — this bounds the sum of all fields, not just each field individually (every decoded string/collection lives inside the input). The limit is on the whole encoded blob (data + framing), so the largest round-trippable single field is slightly below max_size; pass a larger max_size to load trusted data at that edge. load() reads at most max_size+1 bytes, so an oversized file is never materialized. Other per-field caps: MAX_COLLECTION=10M elements per collection, MAX_STRING=100MB per string/bytes, MAX_INT_BYTES=8192, MAX_DEPTH=256 nesting. Peak memory is a multiple of max_size (Python object overhead is ~6× for tiny primitives, plus transient copies during decompression), so set max_size conservatively for untrusted input — e.g. loads(data, max_size=50*1024*1024)
5. Anti zip-bomb: load_compressed bounds both the compressed read and the decompressed output to max_size (the compressed read additionally capped by MAX_COMPRESSED) and decompresses incrementally, aborting the moment the output crosses the limit — a bomb cannot exhaust memory before the size check, and lowering max_size lowers peak memory on both sides
6. Compressed-container integrity: load_compressed verifies the zlib stream ended cleanly (checksum validated), rejects trailing bytes and concatenated streams, and checks the decompressed length against the declared header — a truncated, forged, or padded container fails closed with MSCDecodeError instead of yielding partial data
7. Path null byte rejection: Paths containing null bytes are rejected
8. CRC32 corruption detection: Optional checksum to detect accidental data corruption (not cryptographic — an attacker can forge CRC32)
9. HMAC-SHA256 authentication: Optional cryptographic signature to detect intentional tampering. Anti-downgrade protection rejects any payload lacking a valid HMAC when a key is supplied — including the legacy v1 format — so an attacker cannot strip or version-downgrade the signature.
10. Trailing bytes rejection: Payloads with unexpected bytes after the serialized object are rejected
11. Integer size limit: Ints larger than MAX_INT_BYTES (8192 bytes, ~19,700 digits) are rejected to prevent CPU exhaustion attacks
12. Forged forward-ref rejection: Payloads referencing a container still under construction from an unpatchable position (dict key, set/frozenset member, self-referential root tuple) fail closed with MSCDecodeError — and loads() verifies no unresolved placeholder survives decoding. The encoder can never produce these shapes

What mscs does NOT prevent

1. __setstate__ execution: If you register a class that implements __setstate__, that method will execute during deserialization. Only register classes you trust.
2. Path traversal: Deserialized Path objects may contain ../ sequences. The consumer must validate paths before using them for file I/O.
3. Malicious registered classes: The security boundary is the registry. If you register a class with dangerous behavior in __init__, __setstate__, or property setters, mscs cannot protect you.
4. Bounded-but-not-tiny memory: the size limit bounds the input, but peak memory is a multiple of it — decoding expands compact bytes into Python objects (~6× for tiny primitives), and decompression holds transient copies. Budget for a few times max_size, and lower max_size for untrusted input.

Configuring limits

Pass max_size / max_depth per call — these are the supported, thread-safe knobs:

obj = mscs.loads(untrusted, max_size=50 * 1024 * 1024, max_depth=64)
mscs.dumps(obj, max_depth=64)

Note: rebinding the module constant (mscs.MAX_DEPTH = 1) has no effect — it is a re-exported name, not the global the encoder/decoder read. Use the parameters above.

Rule of thumb: mscs is safe for deserializing untrusted data as long as your registry only contains trusted classes and you set max_size to fit your memory budget.

Binary Format

┌──────────┬─────────┬───────┬──────────┬────────────────┬──────────────┬──────────────┐
│ Magic(4) │ Ver.(1) │ Fl(1) │ Tag(1)   │ Payload(var)   │ CRC32(4)?    │ HMAC(32)?    │
│ "MSCS"   │ 0x02    │ bits  │ type tag │ type-dependent  │ if flag 0x01 │ if flag 0x02 │
└──────────┴─────────┴───────┴──────────┴────────────────┴──────────────┴──────────────┘

Header (6 bytes fixed):

• Bytes 0-3: Magic MSCS (0x4D534353)
• Byte 4: Format version (currently 0x02)
• Byte 5: Flags (bit 0 = CRC32 appended, bit 1 = HMAC-SHA256 appended)

CRC32 and HMAC are mutually exclusive (HMAC is strictly superior).

Payload: Recursive type-length-value encoding. Each value starts with a 1-byte type tag:

ndarray meta: "{dtype}|{shape}" where shape is "dim0xdim1x..." (e.g., "float32|100x100").

Tensor meta: "{dtype}|{shape}|{requires_grad}" (e.g., "float32|256x256|0").

Reference tracking: Mutable containers (list, dict, set, etc.), strings, bytes, and arrays are assigned incrementing IDs. Tag 0x15 refers back to a previously seen object by ID, enabling circular reference support.

License

MIT