# `keras_pickle_wrapper`
This small library exposes a KerasPickleWrapper class that allows keras models to be pickled, and even used across multiprocessing (or with a library like [job\_stream](https://github.com/wwoods/job_stream)).
It won't be necessary if https://github.com/fchollet/keras/issues/789 is ever properly resolved.
## Installation
`pip install keras-pickle-wrapper`
## Usage
```python
import keras
import pickle
import numpy as np
from keras_pickle_wrapper import KerasPickleWrapper
ins = keras.layers.Input((2,))
x = ins
x = keras.layers.Dense(3)(x)
x = keras.layers.Dense(1)(x)
m = keras.models.Model(inputs=ins, outputs=x)
m.compile(loss='mse', optimizer='sgd')
# Wrap a compiled model
mw = KerasPickleWrapper(m)
# Calling the object returns the wrapped Keras model
mw().fit([[0,0], [0,1], [1,0], [1,1]], [[0], [1], [1], [0]])
# Pickle / unpickle the wrapper
data = pickle.dumps(mw)
mw2 = pickle.loads(data)
output_1 = mw().predict(np.asarray([[0, 0]]))
output_2 = mw2().predict(np.asarray([[0, 0]]))
# You can unload the object from memory as well
mw.unload()
# The object will remain unloaded until requested again
output_3 = mw().predict(np.asarray([[0, 0]]))
print("All outputs:")
print(output_1)
print(output_2)
print(output_3)
# If using tensorflow and pickling / unpickling a lot, be sure to clear the
# session:
keras.backend.clear_session()
# When using a custom layer class, be sure to register it so the pickler works
class MyLayer(keras.layers.Layer):
pass # ...
KerasPickleWrapper.register(MyLayer)
```
If your model takes 1GB of RAM, the default approach should require 2GB additional RAM to encode, as it dumps to shared memory by default. To disable this, set `KerasPickleWrapper.NO_SHM = True`. Temporary files will then be written to the standard temporary directory. Using `KerasPickleWrapper.unload(clear_session=True)` prior to pickling combined with `NO_SHM` should eliminate excess memory consumption, but clears the session (session clearing only applies to Tensorflow at the moment).
## Changelog
* 2018-7-17 - Fix for Python 2 in setup.py (FileNotFoundError not existing in Python 2.7), up to V1.0.5.
* 2017-9-19 - Fix for Python 2, up to V1.0.3.
* 2017-8-17 - Renamed to keras-pickle-wrapper because PyPI. V1.0.2.
* 2017-8-17 - Packaged up KerasWrapper for PyPI distribution.
This small library exposes a KerasPickleWrapper class that allows keras models to be pickled, and even used across multiprocessing (or with a library like [job\_stream](https://github.com/wwoods/job_stream)).
It won't be necessary if https://github.com/fchollet/keras/issues/789 is ever properly resolved.
## Installation
`pip install keras-pickle-wrapper`
## Usage
```python
import keras
import pickle
import numpy as np
from keras_pickle_wrapper import KerasPickleWrapper
ins = keras.layers.Input((2,))
x = ins
x = keras.layers.Dense(3)(x)
x = keras.layers.Dense(1)(x)
m = keras.models.Model(inputs=ins, outputs=x)
m.compile(loss='mse', optimizer='sgd')
# Wrap a compiled model
mw = KerasPickleWrapper(m)
# Calling the object returns the wrapped Keras model
mw().fit([[0,0], [0,1], [1,0], [1,1]], [[0], [1], [1], [0]])
# Pickle / unpickle the wrapper
data = pickle.dumps(mw)
mw2 = pickle.loads(data)
output_1 = mw().predict(np.asarray([[0, 0]]))
output_2 = mw2().predict(np.asarray([[0, 0]]))
# You can unload the object from memory as well
mw.unload()
# The object will remain unloaded until requested again
output_3 = mw().predict(np.asarray([[0, 0]]))
print("All outputs:")
print(output_1)
print(output_2)
print(output_3)
# If using tensorflow and pickling / unpickling a lot, be sure to clear the
# session:
keras.backend.clear_session()
# When using a custom layer class, be sure to register it so the pickler works
class MyLayer(keras.layers.Layer):
pass # ...
KerasPickleWrapper.register(MyLayer)
```
If your model takes 1GB of RAM, the default approach should require 2GB additional RAM to encode, as it dumps to shared memory by default. To disable this, set `KerasPickleWrapper.NO_SHM = True`. Temporary files will then be written to the standard temporary directory. Using `KerasPickleWrapper.unload(clear_session=True)` prior to pickling combined with `NO_SHM` should eliminate excess memory consumption, but clears the session (session clearing only applies to Tensorflow at the moment).
## Changelog
* 2018-7-17 - Fix for Python 2 in setup.py (FileNotFoundError not existing in Python 2.7), up to V1.0.5.
* 2017-9-19 - Fix for Python 2, up to V1.0.3.
* 2017-8-17 - Renamed to keras-pickle-wrapper because PyPI. V1.0.2.
* 2017-8-17 - Packaged up KerasWrapper for PyPI distribution.
Release files for keras-pickle-wrapper 1.0.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| keras-pickle-wrapper-1.0.5.tar.gz | 3.5 kB | Details |
Release files / keras-pickle-wrapper-1.0.5.tar.gz
| Download URL | keras-pickle-wrapper-1.0.5.tar.gz |
|---|---|
| Size | 3.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
edf53a71bcf5f14f3653cfa9986a9ad638f9758340c699dd1cc993b7d6927cb0
|
|
BLAKE2b-256 checksum How to use checksums |
57913fad8592aaf17614257fff9bd2206f7d0563bf4b1493fbbf48d17cedb9e1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |