Py-Smartreply
Python bindings for Google's Smart Reply Tensorflow Lite model. This is the much cleaner and easily deployable version of Smartreply which I built earlier.
Features:
- Python package which can be used end-to-end without any dependencies.
- Uses CMake build-system, to build C++ codebase (Earlier Repository used bazel)
- Custom Tensorflow Lite op-resolver, which binds C++ operators to the Directed-Acyclic-Graph at Compile-Time.
- A Web-server written in Flask, which serves the model for inference over a REST API.
- Kubernetes deployment configuration.
Why another repository?
The earlier version used bazel build-system, which led to many build issues. Bazel has poor interoperability with Python build-system. It was literally impossible to build and deploy a python package on the fly, in an automated, hassle-free manner. The newer version of Smartreply uses CMake, which makes it easy to integrate with python build-system, now the build is automated and does not require any intervention (or manual steps in between), this allows us to write CI/CD pipelines with GitHub actions and automate the entire code-test-package-deploy cycle. The earlier version was also dependant on a function called RegisterSelectedOps, this was a dynamically generating function at compile-time (as a header file) and Google somehow automated it with Bazel, which was not possible with CMake, thus the newer version removed its dependency and impelementes it's own op-resolver function, which hardcodes the operators rather than generating a function and linking it at compile-time dynamically.
How to build:
To build this package, you may require CMake and skbuild which are easily installable. Once you have installed these packages, run:
python3 setup.py bdist_wheel
Using package from PyPI:
You can install this package via pip, as follows:
pip install py_smartreply
Usage
The example below shows the usage:
from smartreply import SmartReplyRuntime
#create SmartReplyRuntime() object, you can specify your own model path as an argument, or default provided
#model will be used, ex : SmartReplyRuntime('mymodel.tflite')
rt = SmartReplyRuntime()
#Prediction on a single string
single = rt.predict("hi")
print(single)
#Prediction on multiple strings, runs in a loop
multi = rt.predictMultiple(["hello", "i am happy", "great"])
print(multi)
#Prediction on multiple strings, exploits batching capability of tflite
multiBatch = rt.predictMultiple(["hello", "i am happy", "see me tomorrow"], batch = True)
print(multiBatch)
Using the REST API
Start the server normally, as follows:
python3 web-server/app.py
This will start the server on PORT 8000, then you can use curl to test:
Singe Input Request:
curl -d '{"input" : "Hello"}' -H "Content-Type:application/json" http://localhost:8000/api/inference
Output:
{"result":{"Hi, how are you doing?":1.0899782180786133,"How are you sir?":1.4489225149154663,"I do not understand":1.1177111864089966,"No pictures":0.4019201695919037,"Sending now":0.4459223747253418,"So how did it go?":1.0521267652511597},"success":true}
Multiple Inputs Request:
curl -d '{"input" : ["Hello", "hi", "I am happy"]}' -H "Content-Type:application/json" http://localhost:8000/api/inference
Using the Dockerfile
The Dockerfile can be found at web-server/Dockerfile. You can build and run as follows:
- Build:
cd web-server
docker build . -t smartreply:latest
- Run: docker run --rm -ti -p 8000:8000 smartreply:latest
Deploy on K8s
The deplpyment file is located at web-server/kubernetes/deploy.yaml.
If you have a K8s cluster and kubectl installed, you can try:
kubectl create -f web-server/kubernetes/deploy.yaml
Running tests:
The project uses pytest for testing the functionalities, tests can be found at tests/test_smartreply.py.
You can run the test suite as follows:
pytest tests/
Contributing:
Contributions of any sort are always welcome. You can raise issues, suggest features or make PRs directly, if you find something can be better or need to add any feature that can help others.
Metadata
Release files for py-smartreply 0.0.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distributions (wheels)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| py_smartreply-0.0.1-cp37-cp37m-manylinux2010_x86_64.whl | CPython 3.7 | CPython 3.7 pymalloc | Linux glibc 2.12+ x86-64 | Details for py_smartreply-0.0.1-cp37-cp37m-manylinux2010_x86_64.whl |
| py_smartreply-0.0.1-cp36-cp36m-manylinux2010_x86_64.whl | CPython 3.6 | CPython 3.6 pymalloc | Linux glibc 2.12+ x86-64 | Details for py_smartreply-0.0.1-cp36-cp36m-manylinux2010_x86_64.whl |
Total release size: 15.9 MB
Release files / py_smartreply-0.0.1-cp37-cp37m-manylinux2010_x86_64.whl
| Download URL | py_smartreply-0.0.1-cp37-cp37m-manylinux2010_x86_64.whl |
|---|---|
| Size | 7.9 MB |
| Tags | CPython 3.7 CPython 3.7 pymalloc Linux glibc 2.12+ x86-64 |
|
SHA-256 checksum How to use checksums |
7bd4ea6ac8dd6f2d71c078245aac3e987c4aab3a9749bab29eef94a0402f5491
|
|
BLAKE2b-256 checksum How to use checksums |
9050ad59228bfcd141567d364d25f9fdbf5e3fd126d703c75690c41347c5e4bd
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/49.2.1 requests-toolbelt/0.9.1 tqdm/4.58.0 CPython/3.8.7
|
Release files / py_smartreply-0.0.1-cp36-cp36m-manylinux2010_x86_64.whl
| Download URL | py_smartreply-0.0.1-cp36-cp36m-manylinux2010_x86_64.whl |
|---|---|
| Size | 7.9 MB |
| Tags | CPython 3.6 CPython 3.6 pymalloc Linux glibc 2.12+ x86-64 |
|
SHA-256 checksum How to use checksums |
17ee1a36e0fb21515856c266b86906a44e88df5655d26c1185fb01e410cb9f2a
|
|
BLAKE2b-256 checksum How to use checksums |
047a009d8ad36d3390de758468b40523be888731b8aa85e1f45216936d8fe9bf
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/49.2.1 requests-toolbelt/0.9.1 tqdm/4.58.0 CPython/3.8.7
|
Security for py-smartreply 0.0.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.