Automatic slicer and unslicer of pickle files.
Project description
pickleslicer: Automatic slicer and unslicer of pickle files
The pickleslicer
package provides a convenient way to create and load pickle files
in scenarios where storing many small files is preferable to storing a single large file.
The pickleslicer
package wraps Python's built-in pickle
package and has a very similar usage pattern.
Installation
The package distribution is hosted on PyPI and can be installed via pip
:
pip install pickleslicer
Creating sliced pickle files
To create sliced pickle files, call pickleslicer.dump()
:
import pickleslicer
obj = {} # obj can be any Python object
pickleslicer.dump(obj, "myfilename.pickle")
This will create sliced pickle files named myfilename.pickle.1
, myfilename.pickle.2
, etc., with each file having a maximum size of 10 MB.
To customize the maximum file size, specify the max_size
parameter:
pickleslicer.dump(obj, "myfilename.pickle", max_size=50*1024*1024) # 50 MB
Loading sliced pickle files
To load sliced pickle files that have previously been created, call pickleslicer.load()
:
import pickleslicer
obj = pickleslicer.load("myfilename.pickle")
This will load sliced pickle files named myfilename.pickle.1
, myfilename.pickle.2
, etc. The unpickled Python object is returned.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file pickleslicer-0.1.0.tar.gz
.
File metadata
- Download URL: pickleslicer-0.1.0.tar.gz
- Upload date:
- Size: 2.6 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/50.3.2 requests-toolbelt/0.9.1 tqdm/4.49.0 CPython/3.7.3
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | b8685a060771184738db9d412a0e3c26babd2097d8ba034ab1636b59e31cc865 |
|
MD5 | 87ee861426ffa354db69c248e8dff821 |
|
BLAKE2b-256 | f31c9fd68bb7a686dbc7dc7c6bf0e7c8cd7f29ebabe8048027d64effee2ad00e |
File details
Details for the file pickleslicer-0.1.0-py3-none-any.whl
.
File metadata
- Download URL: pickleslicer-0.1.0-py3-none-any.whl
- Upload date:
- Size: 3.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/50.3.2 requests-toolbelt/0.9.1 tqdm/4.49.0 CPython/3.7.3
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2bf384c0b1024af2055cf2b3bcf85c177ab6ea6a9e49bae91453f29a44104fb2 |
|
MD5 | 4f4905b765770a4e6d8823b334ca28fa |
|
BLAKE2b-256 | e6639200527db6dab320e24c0c0a5ed79bf3ac36173b6d63c51db2293906f11e |