Some utilities for tensorflow serving
Project description
serving-utils
Some python utilities when using tensorflow-serving.
Installation
Prepare an environment with python version >= 3.6
From PYPI: 1. Manually install tensorflow CPU or GPU version. 2. pip install serving-utils
From Github repository: 1. git clone git@github.com:Yoctol/serving-utils.git 2. Manually install tensorflow CPU or GPU version. 3. make install
Usage
Saver and Loader ```python import tensorflow as tf
from serving_utils.saver import Saver from serving_utils.loader import Loader
saver = Saver( session=tf.Session(graph=your_graph), output_dir=’/path/to/serving’, signature_def_map={ ‘predict’: tf.saved_model.signature_def_utils.predict_signature_def( inputs={‘input’: tf.Tensor…}, outputs={‘output’: tf.Tensor…}, ) }, freeze=True, # (default: True) Frozen graph will be saved if True. ) saver.save()
loader = Loader( path=’/path/to/serving’, # version=1, # if not specified, use the latest version ) new_sess = tf.Session()
loader.load(new_sess) # load the saved model into new session ```
Client ```python from serving_utils import Client, PredictInput
client = Client(host=”localhost”, port=8500, n_trys=3) client.predict( [PredictInput(name=’input’, value=np.ones(1, 10))], output_names=[‘output’], model_signature_name=’predict’, )
or async
await client.async_predict(…) ```
Freeze graph ```python from serving_utils.freeze_graph import freeze_graph, create_session_from_graphdef
frozen_graph_def = freeze_graph(session, output_op_names) new_session = create_session_from_graphdef(frozen_graph_def)
## Test Run the following commands:
make lint make test
## Dev
make install-dev
### Protos
python -m grpc_tools.protoc -I. –python_out=. –python_grpc_out=. –grpc_python_out=. serving_utils/protos/*.proto ```
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for serving_utils-0.9.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 87449e9796b43a6c561f0b2ef6b0333ca2f3a18639ee554720acea83f22ef801 |
|
MD5 | 7753fa4b4cf5952769736dc8a39f1239 |
|
BLAKE2b-256 | 34ee45eaf78594b100ee583e6a07437dcdc2d1e9d0e43747b5172eb0f3209be4 |