Skip to main content

Py-Smartreply

Python bindings for Google's Smart Reply Tensorflow Lite model. This is the much cleaner and easily deployable version of Smartreply which I built earlier.

Features:

  1. Python package which can be used end-to-end without any dependencies.
  2. Uses CMake build-system, to build C++ codebase (Earlier Repository used bazel)
  3. Custom Tensorflow Lite op-resolver, which binds C++ operators to the Directed-Acyclic-Graph at Compile-Time.
  4. A Web-server written in Flask, which serves the model for inference over a REST API.
  5. Kubernetes deployment configuration.

Why another repository?

The earlier version used bazel build-system, which led to many build issues. Bazel has poor interoperability with Python build-system. It was literally impossible to build and deploy a python package on the fly, in an automated, hassle-free manner. The newer version of Smartreply uses CMake, which makes it easy to integrate with python build-system, now the build is automated and does not require any intervention (or manual steps in between), this allows us to write CI/CD pipelines with GitHub actions and automate the entire code-test-package-deploy cycle. The earlier version was also dependant on a function called RegisterSelectedOps, this was a dynamically generating function at compile-time (as a header file) and Google somehow automated it with Bazel, which was not possible with CMake, thus the newer version removed its dependency and impelementes it's own op-resolver function, which hardcodes the operators rather than generating a function and linking it at compile-time dynamically.

How to build:

To build this package, you may require CMake and skbuild which are easily installable. Once you have installed these packages, run:

python3 setup.py bdist_wheel

Using package from PyPI:

You can install this package via pip, as follows:

pip install py_smartreply

Usage

The example below shows the usage:

from smartreply import SmartReplyRuntime

#create SmartReplyRuntime() object, you can specify your own model path as an argument, or default provided 
#model will be used, ex : SmartReplyRuntime('mymodel.tflite')
rt = SmartReplyRuntime()

#Prediction on a single string
single = rt.predict("hi")
print(single)

#Prediction on multiple strings, runs in a loop
multi = rt.predictMultiple(["hello", "i am happy", "great"])
print(multi)

#Prediction on multiple strings, exploits batching capability of tflite
multiBatch = rt.predictMultiple(["hello", "i am happy", "see me tomorrow"], batch = True)
print(multiBatch)

Using the REST API

Start the server normally, as follows:

python3 web-server/app.py

This will start the server on PORT 8000, then you can use curl to test:

Singe Input Request:

curl -d '{"input" : "Hello"}' -H "Content-Type:application/json" http://localhost:8000/api/inference

Output:

{"result":{"Hi, how are you doing?":1.0899782180786133,"How are you sir?":1.4489225149154663,"I do not understand":1.1177111864089966,"No pictures":0.4019201695919037,"Sending now":0.4459223747253418,"So how did it go?":1.0521267652511597},"success":true}

Multiple Inputs Request:

curl -d '{"input" : ["Hello", "hi", "I am happy"]}' -H "Content-Type:application/json" http://localhost:8000/api/inference

Using the Dockerfile

The Dockerfile can be found at web-server/Dockerfile. You can build and run as follows:

  1. Build:
cd web-server
docker build . -t smartreply:latest
  1. Run: docker run --rm -ti -p 8000:8000 smartreply:latest

Deploy on K8s

The deplpyment file is located at web-server/kubernetes/deploy.yaml. If you have a K8s cluster and kubectl installed, you can try:

kubectl create -f web-server/kubernetes/deploy.yaml

Running tests:

The project uses pytest for testing the functionalities, tests can be found at tests/test_smartreply.py. You can run the test suite as follows:

pytest tests/

Contributing:

Contributions of any sort are always welcome. You can raise issues, suggest features or make PRs directly, if you find something can be better or need to add any feature that can help others.

Metadata

Release files for py-smartreply 0.0.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distributions (wheels)

Table of built distributions (wheels) for py-smartreply 0.0.1
File Interpreter ABI Platform Actions
py_smartreply-0.0.1-cp37-cp37m-manylinux2010_x86_64.whl CPython 3.7 CPython 3.7 pymalloc Linux glibc 2.12+ x86-64 Details for py_smartreply-0.0.1-cp37-cp37m-manylinux2010_x86_64.whl
py_smartreply-0.0.1-cp36-cp36m-manylinux2010_x86_64.whl CPython 3.6 CPython 3.6 pymalloc Linux glibc 2.12+ x86-64 Details for py_smartreply-0.0.1-cp36-cp36m-manylinux2010_x86_64.whl

Total release size: 15.9 MB

Release files / py_smartreply-0.0.1-cp37-cp37m-manylinux2010_x86_64.whl

Download URL py_smartreply-0.0.1-cp37-cp37m-manylinux2010_x86_64.whl
Size 7.9 MB
Tags CPython 3.7 CPython 3.7 pymalloc Linux glibc 2.12+ x86-64
SHA-256 checksum
How to use checksums
7bd4ea6ac8dd6f2d71c078245aac3e987c4aab3a9749bab29eef94a0402f5491
BLAKE2b-256 checksum
How to use checksums
9050ad59228bfcd141567d364d25f9fdbf5e3fd126d703c75690c41347c5e4bd
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/49.2.1 requests-toolbelt/0.9.1 tqdm/4.58.0 CPython/3.8.7

Release files / py_smartreply-0.0.1-cp36-cp36m-manylinux2010_x86_64.whl

Download URL py_smartreply-0.0.1-cp36-cp36m-manylinux2010_x86_64.whl
Size 7.9 MB
Tags CPython 3.6 CPython 3.6 pymalloc Linux glibc 2.12+ x86-64
SHA-256 checksum
How to use checksums
17ee1a36e0fb21515856c266b86906a44e88df5655d26c1185fb01e410cb9f2a
BLAKE2b-256 checksum
How to use checksums
047a009d8ad36d3390de758468b40523be888731b8aa85e1f45216936d8fe9bf
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/49.2.1 requests-toolbelt/0.9.1 tqdm/4.58.0 CPython/3.8.7

Security for py-smartreply 0.0.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.0.1 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page