Skip to main content
Join the official 2019 Python Developers SurveyStart the survey!

Super fast data serialization for Python 3

Project description

This is a data serialization library. It’s a lot like JSON but it uses a new syntax called “typed netstrings” that Zed has proposed for use in the Mongrel2 webserver. It’s designed to be simpler and easier to implement than JSON, with a happy consequence of also being faster in many cases.

An ordinary netstring is a blob of data prefixed with its length and postfixed with a sanity-checking comma. The string “hello world” encodes like this:

11:hello world,

Typed netstrings add other datatypes by replacing the comma with a type tag. Here’s the integer 12345 encoded as a tnetstring:

5:12345#

And here’s the list [12345,True,0] which mixes integers and bools:

19:5:12345#4:true!1:0#]

Simple enough? This module gives you the following functions:

dump:dump an object as a tnetstring to a file
dumps:dump an object as a tnetstring to a string
load:load a tnetstring-encoded object from a file
loads:load a tnetstring-encoded object from a string
pop:pop a tnetstring-encoded object from the front of a string

Note that since parsing a tnetstring requires reading all the data into memory at once, there’s no efficiency gain from using the file-based versions of these functions. They’re only here so you can use load() to read precisely one item from a file or socket without consuming any extra data.

The tnetstrings specification explicitly states that strings are binary blobs and forbids the use of unicode at the protocol level. As a convenience to python programmers, this library lets you specify an application-level encoding to translate python’s unicode strings to and from binary blobs:

>>> print repr(tnetstring.loads("2:\xce\xb1,"))
'\xce\xb1'
>>>
>>> print repr(tnetstring.loads("2:\xce\xb1,", "utf8"))
u'\u03b1'
Copyright:
  1. 2012-2013 by Ryan Kelly <ryan@rfk.id.au>.
Copyright:
  1. 2014 by Carlo Pires <carlopires@gmail.com>.
License:

MIT, see LICENCE for more details.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for tnetstring3, version 0.3.1
Filename, size File type Python version Upload date Hashes
Filename, size tnetstring3-0.3.1.tar.gz (15.0 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page