cloudpickle

cloudpickle makes it possible to serialize Python constructs not supported by the default pickle module from the Python standard library.

cloudpickle is especially useful for cluster computing where Python code is shipped over the network to execute on remote hosts, possibly close to the data.

Among other things, cloudpickle supports pickling for lambda functions along with functions and classes defined interactively in the __main__ module (for instance in a script, a shell or a Jupyter notebook).

Cloudpickle can only be used to send objects between the exact same version of Python.

Using cloudpickle for long-term object storage is not supported and strongly discouraged.

Security notice: one should only load pickle data from trusted sources as otherwise pickle.load can lead to arbitrary code execution resulting in a critical security vulnerability.

Installation

The latest release of cloudpickle is available from pypi:

pip install cloudpickle

Examples

Pickling a lambda expression:

>>> import cloudpickle
>>> squared = lambda x: x ** 2
>>> pickled_lambda = cloudpickle.dumps(squared)

>>> import pickle
>>> new_squared = pickle.loads(pickled_lambda)
>>> new_squared(2)
4

Pickling a function interactively defined in a Python shell session (in the __main__ module):

>>> CONSTANT = 42
>>> def my_function(data: int) -> int:
...     return data + CONSTANT
...
>>> pickled_function = cloudpickle.dumps(my_function)
>>> depickled_function = pickle.loads(pickled_function)
>>> depickled_function
<function __main__.my_function(data:int) -> int>
>>> depickled_function(43)
85

Running the tests

With tox, to test run the tests for all the supported versions of Python and PyPy:
```
pip install tox
tox
```
or alternatively for a specific environment:
```
tox -e py37
```
With py.test to only run the tests for your current version of Python:
```
pip install -r dev-requirements.txt
PYTHONPATH='.:tests' py.test
```

History

cloudpickle was initially developed by picloud.com and shipped as part of the client SDK.

A copy of cloudpickle.py was included as part of PySpark, the Python interface to Apache Spark. Davies Liu, Josh Rosen, Thom Neale and other Apache Spark developers improved it significantly, most notably to add support for PyPy and Python 3.

The aim of the cloudpickle project is to make that work available to a wider audience outside of the Spark ecosystem and to make it easier to improve it further notably with the help of a dedicated non-regression test suite.

Metadata

Release files for cloudpickle 1.6.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for cloudpickle 1.6.0
File	Size	Uploaded
cloudpickle-1.6.0.tar.gz	52.7 kB	Aug 25, 2020	Details

Built distribution (wheel)

Table of built distributions (wheels) for cloudpickle 1.6.0
File	Interpreter	ABI	Platform	Reset
cloudpickle-1.6.0-py3-none-any.whl 23.0 kB Aug 25, 2020	Python 3	none	any	Details

Total release size: 75.8 kB

Release files / cloudpickle-1.6.0.tar.gz

Download URL	cloudpickle-1.6.0.tar.gz
Size	52.7 kB
Tags	Source
SHA-256 checksum How to use checksums	`9bc994f9e9447593bd0a45371f0e7ac7333710fcf64a4eb9834bf149f4ef2f32`
BLAKE2b-256 checksum How to use checksums	`3b67d1ab1b875e3cae4fc96d2372c92e590f220109738e26470b2a68ddf2baec`
Upload date	Aug 25, 2020
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/46.4.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.7.3`

Release files / cloudpickle-1.6.0-py3-none-any.whl

Download URL	cloudpickle-1.6.0-py3-none-any.whl
Size	23.0 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`3a32d0eb0bc6f4d0c57fbc4f3e3780f7a81e6fee0fa935072884d58ae8e1cc7c`
BLAKE2b-256 checksum How to use checksums	`e7e3898487e5dbeb612054cf2e0c188463acb358167fef749c53c8bb8918cea1`
Upload date	Aug 25, 2020
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/46.4.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.7.3`