Skip to main content

Python3 serializer with memoryview support

Project description

https://travis-ci.org/vodik/cerial.svg?branch=master

Fast, simple, and more pythonic binary serializer library.

Cerial combines Python 3 features, Cython, and memory views to create a fast and easy library for packing and unpacking binary structures.

Removes a lot of the boilerplate that sometimes ends up surrounding struct when used for serialization.

Example

Here’s a quick example of how to use cerial to handle a UDP packet. First we define the structure:

>>> UDP = Struct('UDP',
...    ('sport', 'uint16'),
...    ('dport', 'uint16'),
...    ('len', 'uint16'),
...    ('csum', 'uint16'),
...    endianness='!'
...)

Unlike the builtin struct library, cerial needs the fields to be bound to a name. This is because we expose the binary structure with dictionary interface instead of as a tuple of values.

Now that we have a structure defined, we can use to parse data:

>>> data = bytearray(b'\x08R\x02\x02\x00w\xa9k')
>>> hdr = UDP.parse(data)
>>> hdr
<UDP b'\x08R\x02\x02\x00w\xa9k'>

We can now inspect the binary structure:

>>> hdr['csum']
43371
>>> dict(hdr)
{'csum': 43371, 'len': 119, 'sport': 2130, 'dport': 514}
>>> bytes(hdr)
b'\x08R\x02\x02\x00w\xa9k'

And we can also update individual fields, which will directly write to the underlying memory view, no rendering step needed.

>>> hdr['csum'] = 0xffff
>>> data
bytearray(b'\x08R\x02\x02\x00w\xff\xff')
>>> bytes(hdr)
b'\x08R\x02\x02\x00w\xff\xff'

TODO

  • Feature parity with the various primitives the struct library supports
  • First class support for bit fields
  • Improve performance
  • Support readonly memoryviews (e.g. support bytes). An upstream Cython issue.

Performance

Currently slightly slower than the struct module, but as far as I can tell, significantly faster than other similar libraries like pypacker or scrapy at parsing packets.

Project details


Release history Release notifications

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
cerial-0.0.1.tar.gz (471.9 kB) Copy SHA256 hash SHA256 Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page