ga-serializer

Robust Python object serialization with lazy backends, optional compression, and a self-describing binary format.

ga-serializer uses dill by default to support a broad range of Python objects. If dill is unavailable, it emits a warning and falls back to the standard-library pickle module.

Installation

pip install ga-serializer

Install all optional compression backends with:

pip install "ga-serializer[compression]"

The distribution is named ga-serializer; the Python package is imported as serializer.

Quick start

import serializer

data = {"items": [1, 2, 3], "enabled": True}

payload = serializer.dumps(data, compression="gzip", level=7)
restored = serializer.loads(payload)

serializer.dump(data, "state.srl", compression="lzma")
restored_from_file = serializer.load("state.srl")

Serialized envelopes record the backend and compression codec, so loads() and load() do not require those parameters when reading data created by this library.

Binary streams

dump() and load() accept filesystem paths or open binary streams. User-provided streams are never closed.

from io import BytesIO

import serializer

stream = BytesIO()
serializer.dump([1, 2, 3], stream, compression="zlib")

stream.seek(0)
assert serializer.load(stream) == [1, 2, 3]

Fluent configuration

Serializer is an immutable, reusable configuration object. Configuration methods return a new instance, while dump() returns the current instance to support chained writes.

from serializer import Serializer

codec = Serializer().using("gzip").at_level(9).with_backend("auto").atomic()

codec.dump({"id": 1}, "one.srl").dump({"id": 2}, "two.srl")
assert codec.load("one.srl") == {"id": 1}

Call .strict() to disable dependency fallbacks. The functional API provides the equivalent fallback=False argument.

Compression

The following codecs are always available:

• gzip, bz2, lzma, zlib, and zip
• None or "none" for no compression
• "auto" to prefer Zstandard and otherwise use gzip

Optional codecs are imported only when requested:

When an explicitly requested optional compressor is unavailable during serialization, the default behavior emits DependencyWarning and writes an uncompressed envelope. Deserialization never pretends that compressed data is uncompressed: a missing decoder raises MissingDependencyError.

Legacy payloads

Headerless pickle or dill payloads remain supported when their original settings are provided explicitly:

value = serializer.loads(
    legacy_payload,
    compression="gzip",
    backend="pickle",
)

Security

Benchmarking

Use the public API to benchmark every codec and compression level on any serializable Python object. The result contains median end-to-end times and final serialized sizes in bytes, and renders both matrices directly as Markdown pivot tables:

import serializer

result = serializer.benchmark(
    my_object,
    repeats=3,
    codecs=("gzip", "zstd", "lz4"),  # None benchmarks every codec
    levels=range(10),
)

print(result.to_markdown())

gzip_time = result.value("gzip", 5)
gzip_size = result.size("gzip", 5)

result.values contains timing data and result.sizes contains total serialized envelope sizes. Unavailable codecs are displayed as x; unexpected failures are displayed as ERR and listed in result.errors.

The included script applies this API to a DataFrame containing 10 integer columns and 100,000 rows by default:

python -m pip install -e ".[benchmark,compression]"
python benchmarks/benchmark_codecs.py

Progress is written to stderr and the final pivot table to stdout as Markdown. This makes it possible to save a clean report with:

python benchmarks/benchmark_codecs.py > benchmark-results.md

To run the benchmark and replace the results section below automatically:

python benchmarks/benchmark_codecs.py --update-readme

Use --help to change the dataset size, repetitions, backend, codecs, levels, or README path. Missing dependencies are never replaced by uncompressed fallback data.

Latest DataFrame benchmark

Dataset: 100,000 rows x 10 columns | Backend: pickle | Repetitions: 3

Time

Serialized size

Development

python -m pip install -e ".[dev]"
ruff check .
mypy
pytest
python -m build
twine check dist/*

Releasing

Update __version__ in src/serializer/_version.py, then run the Create release workflow from GitHub Actions. By default, it creates the tag v<version>, generates the GitHub Release notes, and starts the build and PyPI publication workflow.

The workflow accepts an optional tag input when a different tag is required. Package metadata reads the same _version.py value, so code and distribution versions cannot drift.

Released under the MIT License.