Comserde
Comserde (compact serialization and deserialization) is a Python library for serializing data structures into compact binary data.
Installation
$ pip install comserde
Usage
Serializing dataclasses
Decorating a dataclass with @serializable will mark it as serializable and deserializable. Type annotations are used to infer the format of each field.
from comserde import serializable
# Add a decorator to make the dataclass serializable
# The order of decorators matters, the reverse won't work.
@serializable
@dataclass
class User:
age: Optional[int]
name: str
To produce and consume bytes from a serializable object, call dumps() and loads(). A major difference compared to other serialization algorithms (such as json) is that you must provide the type of object you want to decode, User in this example.
from comserde import dumps, loads
user = User(age=34, name="John Doe")
data = dumps(user)
user = loads(data, User)
More complex types can be defined using most of the typical type hints, such as generics and union types.
from typing import NewType
class User:
id: UserId
friends: list[Admin | User]
images: dict[ImageId, tuple[str, Optional[Image]]]
profile_image: bytes
To prevent a specific field from being serialized, assign it with Comserde's field() instead of dataclasses.field() and set serialize=False. Other field attributes can be set as usual.
from comserde import field
class User:
age: Optional[int]
name: str
_cache: Optional[int] = field(default=None, serialize=False)
Serializing regular classes
Regular classes can also be serialized, although the types of each attribute must be explicitly provided, which is why dataclasses are preferred.
from comserde import serializable
# Define serialization fields explicitly
@serializable({
'age': Optional[int],
'name': str
})
class User:
def __init__(self, age: Optional[int], name: str):
self.age = age
self.name = name
# Not serialized
self._cache: Optional[int] = None
Customizing deserialization
If present, the __init_deserialize__ method is called upon deserialization of an object. This is similar to the __setstate__ method called when unpickling.
@serializable({
'age': Optional[int],
'name': str
})
class User:
def __init_deserialize__(self, *, age: Optional[int], name: str):
self.age = age
self.name = name
Customizing serialization formats
The full serialization and deserialization process can be customized by adding the __serialize__ and __deserialize__ methods.
from typing import IO
class User:
def __serialize__(self, file: IO[bytes]):
file.write(...)
@classmethod
def __deserialize__(cls, file: IO[bytes]):
return cls(file.read(...))
Specifying an explicit format
For every field in a class, a format can be explicitly selected by using the typing.Annotated type introduced by PEP 593. The use of this type doesn't affect the behavior of type checkers while providing additional information to Comserde.
from comserde import SerializationFormat, serializable
@serializable
@dataclass
class Vec2d:
x: Annotated[float, SerializationFormat('f64')]
y: Annotated[float, SerializationFormat('f64')]
The same syntax can be used on on regular classes, however the format can also be specified directly.
@serializable({
'x': 'f64',
'y': 'f64'
})
class Vec2d:
...
Serializing any object
Objects that are not marked with @serializable and that do not implement __serializable__() are pickled when serialized, and a warning is emitted if the the pickle format is not explicit. Keep in mind that pickling a large number of individual objects can significantly increase the output size due to the overhead of each pickled object.
Handling exceptions
Any error caused by corrupt data raises a DeserializationError which can be caught if the data is not trusted.
try:
user = loads(data, User)
except DeserializationError:
pass
Reference
Primitive formats
The following serialization formats are supported:
- Booleans and scalars
boolu8,u16,u32,u64i8,i16,i32,i64f32,f64v8,v16,v32,v64– unsigned variable-length quantity (VLQ) encoding of at least 1, 2, 4 and 8 bytesw8,w16,w32,w64– signed VLQ encoding
- Bytes and strings
bytes– length-prefixed bytesnt-bytes– null-terminated bytesutf-8,utf-16
- Others
jsonobject– stores the full qualified name of the object's class and uses the class as the serialization format; useful when the class is not known in advance, e.g. because it might be a subclasspicklevoid
Composite formats
deque[T]NewType('X', T)PurePathand all of its subclassesboolbytearray,bytescomplex– as twof64dict[K, V]EllipsisType,NoneType,Literal[T],tuple[()]– no data producedfloat– asf64frozensetint– asw8list[T]Literal[T, S, etc.]set[T]strT | Stuple[T, ...]tuple[T, S, etc.]
Metadata
Release files for comserde 0.2.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| comserde-0.2.0.tar.gz | 11.2 kB | Without provenance | Details for comserde-0.2.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| comserde-0.2.0-py3-none-any.whl | Python 3 | none | any | Details for comserde-0.2.0-py3-none-any.whl |
Total release size: 22.2 kB
Release files / comserde-0.2.0.tar.gz
| Download URL | comserde-0.2.0.tar.gz |
|---|---|
| Size | 11.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
40b8538e6498a8c24f4f55312f99d3fa3ce93fd68a09aea68b3ae429ddcf657b
|
|
BLAKE2b-256 checksum How to use checksums |
630365eb9adb15a4dd5d287b676ddfc4a72536f265d37f03a5c1b13153343a53
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.10.6
|
Release files / comserde-0.2.0-py3-none-any.whl
| Download URL | comserde-0.2.0-py3-none-any.whl |
|---|---|
| Size | 11.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
fd4bfbaf4c0af246cdf1e2fc4eee6964dcf8b91aa8d2eaba3c85e48c9bf090a3
|
|
BLAKE2b-256 checksum How to use checksums |
ee3a33274e38dea3f962626a5940ce9c4672c12431639a2d6fee63f6369bedd8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.10.6
|
Security for comserde 0.2.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.