A python library for fast serialization and deserialization of complex Python objects into JSON.
Project description
serializejson
Authors |
|
PyPI |
|
Documentation |
|
Sources |
|
Issues |
|
Noncommercial license |
|
Commercial license |
serializejson is a python library for fast serialization and deserialization of python objects in JSON designed as a safe, interoperable and human-readable drop-in replacement for the Python pickle package. Complex python object hierarchies are serializable, deserializable or updatable in once, allowing to save or restore a complete application state in few lines of code. The library is build upon python-rapidjson, pybase64 and blosc for optional zstandard compression.
Some of the main features:
supports Python 3.7 (maybe lower) or greater.
serializes arbitrary python objects into a dictionary by adding __class__ ,and eventually __init__ and __state__ keys.
calls the same objects methods as pickle. Therefore almost all pickable objects are serializable with serializejson without any modification.
for not already pickable object, you will allways be able to serialize it by adding methodes to the object or creating plugins for serializejson.
generally 2x slower than pickle for dumping and 3x slower than pickle for loading (on your benchmark) except for big arrays (optimisation will soon be done).
serializes and deserializes bytes and bytearray very quickly in base64 thanks to pybase64 and lossless blosc compression.
serialized objects take generally less space than when serialized with pickle: for binary data, the 30% increase due to base64 encoding is in general largely compensated using the lossless blosc compression.
serialized objects are human-readable. Unlike pickled data, your data will never become unreadable if your code evolves: you will always be able to modify your datas with a text editor.
serialized objects are text and therefore versionable and comparable with versionning and comparaison tools.
can safely load untrusted / unauthenticated sources if authorized_classes list parameter is set carefully with strictly necessary objects (unlike pickle).
can update existing objects recursively instead of override them (serializejson can be used to save and restore in place a complete application state).
filters attribute starting with “_” by default (unlike pickle).
numpy arrays can be serialized as lists with automatic conversion in both ways or in a conservative way.
supports circular references and serializes only once duplicated objects (WARNING :not yet if the object is a list or dictionary).
tries to call attribute setters and properties setters when loading if set_attributes = True.
accepts json with comment (// and /* */).
can automatically recognize objects in json from keys names and recreate them, without the need of __class__ key, if passed in recognized_classes.
serializejson is easly interoperable outside of the Python ecosystem with this recognition of objects from keys names or with __class__ translation between python and other language classes.
dump and load support string path.
can iteratively encode (with append) and decode (with iterator) a list in json, which helps saving memory space during the process of serialization and deserialization.
Installation
Last offical release
pip install serializejson
Developpement version unreleased
pip install git+https://github.com/SmartAudioTools/serializejson.git
Examples
Serialization with fonctions API
import serializejson
#serialize in string
object1 = set([1,2])
dumped1 = serializejson.dumps(object1)
loaded1 = serializejson.loads(dumped1)
print(dumped1)
>{
> "__class__": "set",
> "__init__": [1,2]
>}
#serialize in file
object2 = set([3,4])
serializejson.dump(object2,"dumped2.json")
loaded2 = serializejson.load("dumped2.json")
Serialization with classes based API.
import serializejson
encoder = serializejson.Encoder()
decoder = serializejson.Decoder()
# serialize in string
object1 = set([1,2])
dumped1 = encoder.dumps(object1)
loaded1 = decoder.loads(dumped1)
print(dumped1)
# serialize in file
object2 = set([3,4])
encoder.dump(object2,"dumped2.json")
loaded2 = decoder.load("dumped2.json")
Update existing object
import serializejson
object1 = set([1,2])
object2 = set([3,4])
dumped1 = serializejson.dumps(object1)
print(f"id {id(object2)} : {object2}")
serializejson.loads(dumped1,obj = object2, updatables_classes = [set])
print(f"id {id(object2)} : {object2}")
Iterative serialization and deserialization
import serializejson
encoder = serializejson.Encoder("my_list.json",indent = None)
for elt in range(3):
encoder.append(elt)
print(open("my_list.json").read())
for elt in serializejson.Decoder("my_list.json"):
print(elt)
>[0,1,2]
>0
>1
>2
More examples and complete documentation here
License
Copyright 2020 Baptiste de La Gorce
For noncommercial use or limited free-trial period commercial use, this project is licensed under the Prosperity Public License 3.0.0.
For non limited commercial use, this project is licensed under the Patron License 1.0.0. To acquire a license please contact me, or just sponsor me on GitHub under the appropriate tier ! This funding model helps me making my work sustainable and compensates me for the work it took to write this crate!
Third-party contributions are licensed under Apache License, Version 2.0 and belong to their respective authors.
History
Version 0.1.0
- Date:
2020-11-28
change description for pipy
add license for pipy
enable load of tuple, time.struct_time, Counter, OrderedDict and defaultdict
Version 0.0.4
- Date:
2020-11-24
API changed
add plugins support
add bytes, bytearray and numpy.array compression with blosc zstd
fix itertive append and decode (not fully tested).
fix dump of numpy types without conversion to python types(not yet numpy.float64)
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for serializejson-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 43f54cfb0155f18475a343f2d3c60f83b5f1ef04aa301b8ab682ca47b06827ce |
|
MD5 | 53ca023764af4f92b48b43bb1aadfbc9 |
|
BLAKE2b-256 | 2450a951f2816b1d427ddf47a18bf070c7af2e50cd366bc4c0334096e0af6da9 |