Skip to main content

BinPickle - efficient binary pickled data

PyPI version Test and Build codecov

This package uses the new Pickle Protocol 5 in Python 3.8 (or its pickle5 backport) to efficiently serialize large objects, particularly from scientific Python packages, to an on-disk format. This format is designed to support two use cases:

  1. Serializing data-intensive statistical models in a memory-mappable format so multiple processes can share the same (read-only) model memory.
  2. Serializing data-intensive statistical models with good compression for long-term storage and cross-machine transportation.

BinPickle does this by using Pickle 5's out-of-band buffer serialization support to write buffers uncompressed and page-aligned for memory mapping (use case 1) or with per-buffer efficient compression with libraries like Blosc (use case 2).

Format Stability

We do not yet guarantee the stability of the BinPickle format. We will avoid gratuitous changes, but BinPickle 1.0 will be the first with a stability guarantee.

Acknowledgements

This material is based upon work supported by the National Science Foundation under Grant No. IIS 17-51278. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. This page has not been approved by Boise State University and does not reflect official university positions.

Metadata

Release files for binpickle 0.3.4

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for binpickle 0.3.4
File Size Uploaded Provenance Actions
binpickle-0.3.4.tar.gz 22.4 kB Without provenance Details for binpickle-0.3.4.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for binpickle 0.3.4
File Interpreter ABI Platform Actions
binpickle-0.3.4-py3-none-any.whl Python 3 none any Details for binpickle-0.3.4-py3-none-any.whl

Total release size: 36.3 kB

Release files / binpickle-0.3.4.tar.gz

Download URL binpickle-0.3.4.tar.gz
Size 22.4 kB
Tags Source
SHA-256 checksum
How to use checksums
13a35953d7f3470d571a97a904cbd42875b9a2a69a7181bf9a6e034d415426ee
BLAKE2b-256 checksum
How to use checksums
99ce73eae6a7d4b08054cb6506e6b8609eb0d5f34fc356cc4eab1546377ccd86
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via python-requests/2.25.1

Release files / binpickle-0.3.4-py3-none-any.whl

Download URL binpickle-0.3.4-py3-none-any.whl
Size 13.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
85d77b3288d529d43bbebf77e543ebaf3703e42f74e9267f918ec2be5e91c506
BLAKE2b-256 checksum
How to use checksums
2e0df94ed7391a5be911355e6c857c1a72e19045517f7e3e765e2d1d22e33ade
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via python-requests/2.25.1

Security for binpickle 0.3.4

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page