Skip to main content

A library for data preprocessing with TensorFlow

Project description

TensorFlow Transform

Python PyPI Documentation

TensorFlow Transform is a library for preprocessing data with TensorFlow. tf.Transform is useful for data that requires a full-pass, such as:

  • Normalize an input value by mean and standard deviation.
  • Convert strings to integers by generating a vocabulary over all input values.
  • Convert floats to integers by assigning them to buckets based on the observed data distribution.

TensorFlow has built-in support for manipulations on a single example or a batch of examples. tf.Transform extends these capabilities to support full-passes over the example data.

The output of tf.Transform is exported as a TensorFlow graph to use for training and serving. Using the same graph for both training and serving can prevent skew since the same transformations are applied in both stages.

For an introduction to tf.Transform, see the tf.Transform section of the TFX Dev Summit talk on TFX (link).

Caution: tf.Transform may be backwards incompatible before version 1.0.

Installation

The tensorflow-transform PyPI package is the recommended way to install tf.Transform:

pip install tensorflow-transform

Dependencies

tf.Transform requires TensorFlow but does not depend on the tensorflow PyPI package. See the TensorFlow install guides for instructions.

Apache Beam is required to run distributed analysis. By default, Apache Beam runs in local mode but can also run in distributed mode using Google Cloud Dataflow. tf.Transform is designed to be extensible for other Apache Beam runners.

Compatible versions

The following table is the tf.Transform package versions that are compatible with each other. This is determined by our testing framework, but other untested combinations may also work.

tensorflow-transform tensorflow apache-beam[gcp]
GitHub master nightly (1.x/2.x) 2.16.0
0.15.0 1.15 / 2.0 2.16.0
0.14.0 1.14 2.14.0
0.13.0 1.13 2.11.0
0.12.0 1.12 2.10.0
0.11.0 1.11 2.8.0
0.9.0 1.9 2.6.0
0.8.0 1.8 2.5.0
0.6.0 1.6 2.4.0
0.5.0 1.5 2.3.0
0.4.0 1.4 2.2.0
0.3.1 1.3 2.1.1
0.3.0 1.3 2.1.1
0.1.10 1.0 2.0.0

Questions

Please direct any questions about working with tf.Transform to Stack Overflow using the tensorflow-transform tag.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for tensorflow-transform, version 0.15.0
Filename, size File type Python version Upload date Hashes
Filename, size tensorflow-transform-0.15.0.tar.gz (222.9 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page