Skip to main content

TensorFlow project scaffolding

Project description

# TF Stage

A fast and canonical project setup for TensorFlow models. The most difficult part of getting started with TensorFlow isn't deep learning, it's putting together hundreds of API calls into a cohesive model.

```
$ tfstage --help
usage: tfstage [-h] name

TensorFlow project scaffolding

positional arguments:
name Project name
install_dependencies Install pip dependencies

optional arguments:
-h, --help show this help message and exit
```

## Usage

1. Install `tfstage`:

```
pip install tfstage
```

2. Create a new empty project directory

```
$ mkdir my_project/
$ cd my_project/
```

3. Run `tfstage my_project`:

```
$ tfstage my_project
Project created: ./my_project
```

4. This stubs out an entire TensorFlow project, completely runnable using a simple XOR dataset and model. For example:

```
$ python -m my_project.main --job-dir logs/

...

INFO:tensorflow:Saving checkpoints for 1 into logs/model.ckpt.
INFO:tensorflow:loss = 1.20236, step = 1
INFO:tensorflow:Starting evaluation at 2017-07-13-18:22:20
INFO:tensorflow:Restoring parameters from logs/model.ckpt-1

...
```

## Workflow

When starting a new project we run `tfstage`, run the code to verify everything works, then search and replace the `TODO` comments in the code which mark important changes.

## Environment

High-level description of a new project:

- main.py: defines command-line arguments and sets up [`learn_runner`](https://goo.gl/I6TwxA)
- experiment.py: defines a [`tf.contrib.learn.Experiment`](https://goo.gl/nMvwLx) for training
- inputs.py: defines the input pipeline for training and evaluation
- model.py: defines the model, loss, and training optimization
- augment.py: defines any data augmentation or feature engineering
- serve.py: defines placeholders for [TensorFlow Serving](https://goo.gl/bM3jpA) and [Google Cloud ML Engine predictions](https://goo.gl/yTBv2e).

In addition, several common files are created including:

- README.md
- requirements.txt for local _development_
- setup.py for local and GCE _deployment_
- .gitignore

### Local Deployment

```
PROJECT_NAME=my_project
MODULE_NAME="${PROJECT_NAME}.main"
PACKAGE_PATH="${PROJECT_NAME}/"
JOB_DIR=logs/

gcloud ml-engine local train \
--module-name $MODULE_NAME \
--package-path $PACKAGE_PATH \
--job-dir $JOB_DIR \
-- \
[args]
```

### Cloud Deployment

```
MODULE_NAME="${PROJECT_NAME}.main"
PACKAGE_PATH="${PROJECT_NAME}/"
JOB_NAME="${PROJECT_NAME}_1"
JOB_DIR="gs://${PROJECT_NAME}/${JOB_NAME}"
REGION=us-east1

gcloud ml-engine jobs submit training $JOB_NAME \
--job-dir $JOB_DIR \
--module-name $MODULE_NAME \
--package-path $PACKAGE_PATH \
--region $REGION \
-- \
[args]
```


Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

tfstage-0.1.7-py2.py3-none-any.whl (14.5 kB view details)

Uploaded Python 2Python 3

File details

Details for the file tfstage-0.1.7-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for tfstage-0.1.7-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 3379406898821bf3ac24ae78eea762e884b807425dc0e0c91bab4033b4e5fad7
MD5 f4f08a6d771d77e2ed8343e857bda64a
BLAKE2b-256 243dd1525a58b94c45c6bc1c73f6f485a99d49ed6d273d09789d4f8a0ac3c49d

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page