A secure python pickle replacement
Project description
## larch.pickle - A secure python pickle replacement
This module can be used as transparent replacement for pickle. It does not suffer the
vulnerability of original pickle see: http://michael-rushanan.blogspot.de/2012/10/why-python-pickle-is-insecure.html.
Difference to python pickle:
- no memo attribute
- no clear_memo()
- no persistent_id interface
- byte string are always assumed to be utf-8 encoded
- `Pickler` has an additional `with_refs` parameter. Setting `with_refs`
to `false`, the pickler will ignore object references, this results in
an extra speed boost.
### Installation
larch-pickle needs the boost library for compilation. If boost
is not in the standard include path install it with:
`python build_ext -I /path/to/boost install`
### Speed compared to some other pickler packages
#### dump Dictionaries (10 loops)
| Package | Seconds | Size |
|:-------------|----------:|---------:|
| marshal | 0.454694 | 15975952 |
| larch-pickle | 0.586087 | 9714411 |
| ujson | 1.94068 | 13141354 |
| msgpack | 1.95001 | 9843459 |
| json | 2.1489 | 14365311 |
| cPickle | 4.29243 | 12781866 |
#### load Dictionaries (10 loops)
| Package | Seconds |
|:-------------|----------:|
| msgpack | 1.43802 |
| larch-pickle | 1.53487 |
| marshal | 1.56561 |
| cPickle | 2.44472 |
| ujson | 2.48116 |
| json | 5.78576 |
#### dump Objects (10 loops)
| Package | Seconds | Size |
|:-------------|----------:|---------:|
| larch-pickle | 0.968418 | 10094415 |
| ujson | 2.56079 | 13331354 |
| cPickle | 4.8822 | 12961909 |
#### load Objects (10 loops)
| Package | Seconds |
|:-------------|----------:|
| larch-pickle | 1.6902 |
| ujson | 2.51544 |
| cPickle | 2.58821 |
#### dump Strings (10 loops)
| Package | Seconds | Size |
|:-------------|----------:|---------:|
| marshal | 0.582926 | 32826787 |
| msgpack | 0.980826 | 29464951 |
| json | 1.18866 | 31966498 |
| ujson | 1.51086 | 30987525 |
| larch-pickle | 1.70253 | 17054823 |
| cPickle | 4.21949 | 19871780 |
#### load Strings (10 loops)
| Package | Seconds |
|:-------------|----------:|
| larch-pickle | 0.431427 |
| marshal | 0.449067 |
| msgpack | 0.549114 |
| cPickle | 1.70127 |
| ujson | 1.88499 |
| json | 3.17667 |
#### dump Lists (10 loops)
| Package | Seconds | Size |
|:-------------|----------:|---------:|
| marshal | 0.879272 | 42703907 |
| larch-pickle | 3.11243 | 21005671 |
| json | 3.27417 | 35917346 |
| ujson | 4.34822 | 34938373 |
| msgpack | 4.44871 | 31440375 |
| cPickle | 15.2853 | 33700258 |
#### load Lists (10 loops)
| Package | Seconds |
|:-------------|----------:|
| larch-pickle | 2.12795 |
| marshal | 2.14723 |
| msgpack | 2.41231 |
| ujson | 4.27773 |
| json | 5.29066 |
| cPickle | 8.1543 |
This module can be used as transparent replacement for pickle. It does not suffer the
vulnerability of original pickle see: http://michael-rushanan.blogspot.de/2012/10/why-python-pickle-is-insecure.html.
Difference to python pickle:
- no memo attribute
- no clear_memo()
- no persistent_id interface
- byte string are always assumed to be utf-8 encoded
- `Pickler` has an additional `with_refs` parameter. Setting `with_refs`
to `false`, the pickler will ignore object references, this results in
an extra speed boost.
### Installation
larch-pickle needs the boost library for compilation. If boost
is not in the standard include path install it with:
`python build_ext -I /path/to/boost install`
### Speed compared to some other pickler packages
#### dump Dictionaries (10 loops)
| Package | Seconds | Size |
|:-------------|----------:|---------:|
| marshal | 0.454694 | 15975952 |
| larch-pickle | 0.586087 | 9714411 |
| ujson | 1.94068 | 13141354 |
| msgpack | 1.95001 | 9843459 |
| json | 2.1489 | 14365311 |
| cPickle | 4.29243 | 12781866 |
#### load Dictionaries (10 loops)
| Package | Seconds |
|:-------------|----------:|
| msgpack | 1.43802 |
| larch-pickle | 1.53487 |
| marshal | 1.56561 |
| cPickle | 2.44472 |
| ujson | 2.48116 |
| json | 5.78576 |
#### dump Objects (10 loops)
| Package | Seconds | Size |
|:-------------|----------:|---------:|
| larch-pickle | 0.968418 | 10094415 |
| ujson | 2.56079 | 13331354 |
| cPickle | 4.8822 | 12961909 |
#### load Objects (10 loops)
| Package | Seconds |
|:-------------|----------:|
| larch-pickle | 1.6902 |
| ujson | 2.51544 |
| cPickle | 2.58821 |
#### dump Strings (10 loops)
| Package | Seconds | Size |
|:-------------|----------:|---------:|
| marshal | 0.582926 | 32826787 |
| msgpack | 0.980826 | 29464951 |
| json | 1.18866 | 31966498 |
| ujson | 1.51086 | 30987525 |
| larch-pickle | 1.70253 | 17054823 |
| cPickle | 4.21949 | 19871780 |
#### load Strings (10 loops)
| Package | Seconds |
|:-------------|----------:|
| larch-pickle | 0.431427 |
| marshal | 0.449067 |
| msgpack | 0.549114 |
| cPickle | 1.70127 |
| ujson | 1.88499 |
| json | 3.17667 |
#### dump Lists (10 loops)
| Package | Seconds | Size |
|:-------------|----------:|---------:|
| marshal | 0.879272 | 42703907 |
| larch-pickle | 3.11243 | 21005671 |
| json | 3.27417 | 35917346 |
| ujson | 4.34822 | 34938373 |
| msgpack | 4.44871 | 31440375 |
| cPickle | 15.2853 | 33700258 |
#### load Lists (10 loops)
| Package | Seconds |
|:-------------|----------:|
| larch-pickle | 2.12795 |
| marshal | 2.14723 |
| msgpack | 2.41231 |
| ujson | 4.27773 |
| json | 5.29066 |
| cPickle | 8.1543 |
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
larch-pickle-1.0.1.tar.gz
(3.9 MB
view hashes)
Built Distributions
larch-pickle-1.0.1.win32-py3.3.exe
(261.8 kB
view hashes)
larch-pickle-1.0.1.win32-py2.7.exe
(260.8 kB
view hashes)
Close
Hashes for larch-pickle-1.0.1.win32-py3.3.exe
Algorithm | Hash digest | |
---|---|---|
SHA256 | bb6757eef5d923e18a743dffeacf99f62bebbf49f12822f19988f00e2b5b5d22 |
|
MD5 | 4bb77055ed1051fdf90a87acd93550d4 |
|
BLAKE2b-256 | 9ab850c0ae1385e439a1991c338e929aa15e5ce77d1cc477867250f2e8649df6 |
Close
Hashes for larch-pickle-1.0.1.win32-py2.7.exe
Algorithm | Hash digest | |
---|---|---|
SHA256 | 66b87c48e5897595e23c2df8c6c0f3cb991cc9060f900bb82b5da77dac795f5b |
|
MD5 | a376763b4f0dd8db74b4bd3dcbb10b58 |
|
BLAKE2b-256 | 4aa00f7144c8b62b77614e28881aa8042f7d5e3d58a0b5d1fc1a2ecf4f172483 |