ruamel.ext.msgpack 0.1.0

thin wrapper around msgpack to deal with naive datetime and ruamel defined extension types

ruamel.ext.msgpack is a thin wrapper around msgpack, that deals with naive datetime instances and ruamel defined extension types (date).

naive datetime.datetime

If you try to pack a naive datetime.datetime instance, you’ll get an error:

import datetime
from msgpack import packb

try:
    packb(datetime.datetime(2011, 10, 2), datetime=True)
except Exception as e:
    print('exception:', type(e), e)

which will print:

exception: <class 'ValueError'> can not serialize 'datetime.datetime' object where tzinfo=None

using pack/packb from ruamel.ext.msgpack will prevent this from happening, without having to go over the data structure and replacing all naive datetime.datetime instances. It will provide both an argument to default that handles the naive datetimestamp and some Ext types, set use_bin_type=True and datetime=True.

import sys
import datetime
from ruamel.ext.msgpack import packb, unpackb

res = packb(datetime.datetime(2011, 10, 2, 17, 15, 1))
print(unpackb(res))

which will print:

2011-10-02 17:15:01+00:00

as you can see from the output this will make the instance timezone aware.

use_bin_type=False

The pack and packb routines set use_bin_type=False. So the UTF-8 “bytestrings” get dumped as “fixstr”, “str 8/16/32”, and those will load as str back into Python (the function hex() returns a string of hexadecimal values from the bytes array passed into it):

from ruamel.ext.msgpack import packb, unpackb, hex

res = packb('こんにちは世界'.encode('utf-8'))
print(hex(res), len(res))
print(unpackb(res))

\xb5 indicates “fixstr” of length 21

\xb5\xe3\x81\x93\xe3\x82\x93\xe3\x81\xab\xe3\x81\xa1\xe3\x81\xaf\xe4\xb8\x96\xe7\x95\x8c 22
こんにちは世界

If you need byte arrays, an alternate version of pack/unpack can be constructed, that still handle naive datetime objects, and the other types:

from functools import partial
from ruamel.ext.msgpack import hex, unpackb, msgpack_default
import msgpack

pack = partial(msgpack.pack, default=msgpack_default, use_bin_type=True, datetime=True)
packb = partial(msgpack.packb, default=msgpack_default, use_bin_type=True, datetime=True)

res = packb('こんにちは世界'.encode('utf-8'))
print(hex(res), len(res))
print(unpackb(res))

\xc4\x15 indicates “bin 8” of length 21

extension types

The following extension types are provided by ruamel.ext.msgpack. Each has associated attribute on msgpack_default with the type number. This number can be changed, but the same numbers should be used for packing and unpacking. If a number is set to None the associated type will not be packed or unpacked. The type used for naive datetime.datetime, -1, cannot be changed.

datetime.date

Python’s datetime.date instances are packed in a two bytes stucture for dates in the range 2000-01-01 and 2126-12-31.

\xc4\x15\xe3\x81\x93\xe3\x82\x93\xe3\x81\xab\xe3\x81\xa1\xe3\x81\xaf\xe4\xb8\x96\xe7\x95\x8c 23
b'\xe3\x81\x93\xe3\x82\x93\xe3\x81\xab\xe3\x81\xa1\xe3\x81\xaf\xe4\xb8\x96\xe7\x95\x8c'

import datetime
from ruamel.ext.msgpack import packb, unpackb, hex, msgpack_default

res = packb(datetime.date(2011, 10, 2))
print('hex:', hex(res), len(res))
print(unpackb(res))
print(f'{msgpack_default.date=}')

msgpack_default.date = 42
res = packb(datetime.date(2011, 10, 2))
print('hex:', hex(res), len(res))
print(unpackb(res))

try:
    msgpack_default.date = None
    res = packb(datetime.date(2011, 10, 2))
except Exception as e:
    print('exception:', type(e), e)

which will print:

hex: \xd5\x11\x17\x82 4
2011-10-02
msgpack_default.date=17
hex: \xd5\x2a\x17\x82 4
2011-10-02
exception: <class 'ValueError'> year out of range 2000-2126