Struct2Tensor
Introduction
struct2tensor is a library for parsing structured data inside of tensorflow. In particular, it makes it easy to manipulate structured data, e.g., slicing, flattening, copying substructures, and so on, as part of a TensorFlow model graph. The notebook in 'examples/prensor_playground.ipynb' provides a few examples of struct2tensor in action and an introduction to the main concepts. You can run the notebook in your browser through Google's colab environment, or download the file to run it in your own Jupyter environment.
There are two main use cases of this repo:
- To create a PIP package. The PIP package contains plug-ins (OpKernels) to an existing tensorflow installation.
- To staticlly link with tensorflow-serving.
As these processes are independent, one can follow either set of directions below.
Use a pre-built Linux PIP package.
From a virtual environment, run:
pip install struct2tensor
Nightly Packages
Struct2Tensor also hosts nightly packages at https://pypi-nightly.tensorflow.org on Google Cloud. To install the latest nightly package, please use the following command:
pip install --extra-index-url https://pypi-nightly.tensorflow.org/simple struct2tensor
This will install the nightly packages for the major dependencies of struct2tensor such as TensorFlow Metadata (TFMD).
Creating a PIP package.
The struct2tensor PIP package is useful for creating models. It works with tensorflow 2.x.
In order to unify the process, we recommend compiling struct2tensor inside a docker container.
Downloading the Code
Go to your home directory.
Download the source code.
git clone https://github.com/google/struct2tensor.git
cd ~/struct2tensor
Use docker-compose
Install docker-compose.
Use it to build a pip wheel for Python 3.8 with tensorflow version 2:
docker-compose build --build-arg PYTHON_VERSION=3.8 manylinux2014
docker-compose run -e TF_VERSION=RELEASED_TF_2 manylinux2014
This will create a manylinux package in the ~/struct2tensor/dist directory.
Creating a static library
In order to construct a static library for tensorflow-serving, we run:
bazel build -c opt struct2tensor:struct2tensor_kernels_and_ops
This can also be linked into another library.
TensorFlow Serving docker image
struct2tensor needs a couple of custom TensorFlow ops to function. If you train
a model with struct2tensor and wants to serve it with TensorFlow Serving, the
TensorFlow Serving binary needs to link with those custom ops. We have a
pre-built docker image that contains such a binary. The Dockerfile is
available at tools/tf_serving_docker/Dockerfile. The image is available at
gcr.io/tfx-oss-public/s2t_tf_serving.
Please see the Dockerfile for details. But in brief, the image exposes port
8500 as the gRPC endpoint and port 8501 as the REST endpoint. You can set
two environment variables MODEL_BASE_PATH and MODEL_NAME to point it to
your model (either mount it to the container, or put your model on GCS).
It will look for a saved model at
${MODEL_BASE_PATH}/${MODEL_NAME}/${VERSION_NUMBER}, where VERSION_NUMBER
is an integer.
Compatibility
| struct2tensor | tensorflow |
|---|---|
| 0.52.0 | 2.21.0 |
| 0.48.0 | 2.17.0 |
| 0.47.0 | 2.16.2 |
| 0.46.0 | 2.15.0 |
| 0.45.0 | 2.13.0 |
| 0.44.0 | 2.12.0 |
| 0.43.0 | 2.11.0 |
| 0.42.0 | 2.10.0 |
| 0.41.0 | 2.9.0 |
| 0.40.0 | 2.9.0 |
| 0.39.0 | 2.8.0 |
| 0.38.0 | 2.8.0 |
| 0.37.0 | 2.7.0 |
| 0.36.0 | 2.7.0 |
| 0.35.0 | 2.6.0 |
| 0.34.0 | 2.6.0 |
| 0.33.0 | 2.5.0 |
| 0.32.0 | 2.5.0 |
| 0.31.0 | 2.5.0 |
| 0.30.0 | 2.4.0 |
| 0.29.0 | 2.4.0 |
| 0.28.0 | 2.4.0 |
| 0.27.0 | 2.4.0 |
| 0.26.0 | 2.3.0 |
| 0.25.0 | 2.3.0 |
| 0.24.0 | 2.3.0 |
| 0.23.0 | 2.3.0 |
| 0.22.0 | 2.2.0 |
| 0.21.1 | 2.1.0 |
| 0.21.0 | 2.1.0 |
| 0.0.1.dev* | 1.15 |
2 maintainers
Metadata
2 maintainers
Release files for struct2tensor 0.52.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distributions (wheels)
| File | Reset | |||
|---|---|---|---|---|
| struct2tensor-0.52.0-cp313-cp313-manylinux_2_39_x86_64.whl | CPython 3.13 | CPython 3.13 | Linux glibc 2.39+ x86-64 | Details |
| struct2tensor-0.52.0-cp313-cp313-macosx_11_0_arm64.whl | CPython 3.13 | CPython 3.13 | macOS 11.0+ ARM64 | Details |
| struct2tensor-0.52.0-cp312-cp312-manylinux_2_39_x86_64.whl | CPython 3.12 | CPython 3.12 | Linux glibc 2.39+ x86-64 | Details |
| struct2tensor-0.52.0-cp312-cp312-macosx_11_0_arm64.whl | CPython 3.12 | CPython 3.12 | macOS 11.0+ ARM64 | Details |
| struct2tensor-0.52.0-cp311-cp311-manylinux_2_39_x86_64.whl | CPython 3.11 | CPython 3.11 | Linux glibc 2.39+ x86-64 | Details |
| struct2tensor-0.52.0-cp311-cp311-macosx_11_0_arm64.whl | CPython 3.11 | CPython 3.11 | macOS 11.0+ ARM64 | Details |
| struct2tensor-0.52.0-cp310-cp310-manylinux_2_39_x86_64.whl | CPython 3.10 | CPython 3.10 | Linux glibc 2.39+ x86-64 | Details |
| struct2tensor-0.52.0-cp310-cp310-macosx_11_0_arm64.whl | CPython 3.10 | CPython 3.10 | macOS 11.0+ ARM64 | Details |
Total release size: 32.6 MB
Release files / struct2tensor-0.52.0-cp313-cp313-manylinux_2_39_x86_64.whl
| Download URL | struct2tensor-0.52.0-cp313-cp313-manylinux_2_39_x86_64.whl |
|---|---|
| Size | 4.7 MB |
| Tags | CPython 3.13 Linux glibc 2.39+ x86-64 |
|
SHA-256 checksum How to use checksums |
dd95ea97b99aede02c96aedee6f6c8ab67d9e7b760b5a8ff65b83028e122b634
|
|
BLAKE2b-256 checksum How to use checksums |
7a1096b990f61d0d5a41e87c959b51dcc0d46de43b2b0d5b4e3535f195476e88
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.12
|
Release files / struct2tensor-0.52.0-cp313-cp313-macosx_11_0_arm64.whl
| Download URL | struct2tensor-0.52.0-cp313-cp313-macosx_11_0_arm64.whl |
|---|---|
| Size | 3.5 MB |
| Tags | CPython 3.13 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
6aaa327aa9b29eea5f364142134c3c390c912ccb3680d8d57da661d9fbaef7ab
|
|
BLAKE2b-256 checksum How to use checksums |
da01cfc8b24bb849c0ff38c21a9b0eec1be2cfa4e9ccdfac637fe9c73749b620
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.12
|
Release files / struct2tensor-0.52.0-cp312-cp312-manylinux_2_39_x86_64.whl
| Download URL | struct2tensor-0.52.0-cp312-cp312-manylinux_2_39_x86_64.whl |
|---|---|
| Size | 4.7 MB |
| Tags | CPython 3.12 Linux glibc 2.39+ x86-64 |
|
SHA-256 checksum How to use checksums |
ad939d19810c2a912b453e97b91476cc0d02280af18864307ce5ce698ecafb25
|
|
BLAKE2b-256 checksum How to use checksums |
a410d8d62e35b470cea9da473189c7a2984d74678caf3127db0de06cb22c53a1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.12
|
Release files / struct2tensor-0.52.0-cp312-cp312-macosx_11_0_arm64.whl
| Download URL | struct2tensor-0.52.0-cp312-cp312-macosx_11_0_arm64.whl |
|---|---|
| Size | 3.5 MB |
| Tags | CPython 3.12 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
8976187e298f0f867d4d67a3cd5e6e8e4c3f693fda9c24079b0dfed57603b18c
|
|
BLAKE2b-256 checksum How to use checksums |
a73d95052eba97a3f2464b7233d0d985e0a87e26f139af36f43dbc85cb51620c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.12
|
Release files / struct2tensor-0.52.0-cp311-cp311-manylinux_2_39_x86_64.whl
| Download URL | struct2tensor-0.52.0-cp311-cp311-manylinux_2_39_x86_64.whl |
|---|---|
| Size | 4.7 MB |
| Tags | CPython 3.11 Linux glibc 2.39+ x86-64 |
|
SHA-256 checksum How to use checksums |
9a582e739139129d3791fd4ec8540c2b466638d6e9f0cb0fed9f05f3509c611a
|
|
BLAKE2b-256 checksum How to use checksums |
c9b75c1a7abc1bdd5ec90f4861ca0a9db1dc39c08b98be696a52c92b7deec566
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.12
|
Release files / struct2tensor-0.52.0-cp311-cp311-macosx_11_0_arm64.whl
| Download URL | struct2tensor-0.52.0-cp311-cp311-macosx_11_0_arm64.whl |
|---|---|
| Size | 3.5 MB |
| Tags | CPython 3.11 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
a1a6dea5aa7123193df3f432f21a45681763a65ba53b52cef0105f9c53d85bbc
|
|
BLAKE2b-256 checksum How to use checksums |
deef6f181a491b2c6c9420b5b155a676f78a151cd5e3972507528bf9561bab40
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.12
|
Release files / struct2tensor-0.52.0-cp310-cp310-manylinux_2_39_x86_64.whl
| Download URL | struct2tensor-0.52.0-cp310-cp310-manylinux_2_39_x86_64.whl |
|---|---|
| Size | 4.7 MB |
| Tags | CPython 3.10 Linux glibc 2.39+ x86-64 |
|
SHA-256 checksum How to use checksums |
00e3392166ea2034bfcfcb35fa1e3c46ef1d31ea075020839ae30cbf5a807b99
|
|
BLAKE2b-256 checksum How to use checksums |
f3a6327c80e5b31edbf7788eb27c189240ac481b6eaa8c67a6fbbf4d97a5c938
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.12
|
Release files / struct2tensor-0.52.0-cp310-cp310-macosx_11_0_arm64.whl
| Download URL | struct2tensor-0.52.0-cp310-cp310-macosx_11_0_arm64.whl |
|---|---|
| Size | 3.5 MB |
| Tags | CPython 3.10 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
9f0d611afe6c524a5451ffdc173f8302a7ccd5f64b099c891d13df72ed1a0015
|
|
BLAKE2b-256 checksum How to use checksums |
26814dcc0c7066e596d66525daeaf769b1ccb125af7d58bd0b1473b77f70d8c3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.12
|
