Skip to main content

Set Tensor values during training in Tensorflow.

Project description

Validation Curve

Validation Curve

Change the hyper-parameters of your Tensorflow training session on the fly. The package allows you to schedule events that change the values of arbitrary Tensors with a simple command.

Requirements

  • Python >= 3
  • tensorflow >= 1.0

Set Up

Clone the repository with either

git clone https://github.com/ondrejba/interactive-tensorflow.git interactive_tensorflow

or

git submodule add https://github.com/ondrejba/interactive-tensorflow.git interactive_tensorflow

if you want to clone to an existing git repository.

We change the name of the repository to interactive_tensorflow because Python does not like the dash symbol when importing modules.

Usage

Interactive Tensorflow DEMO

Check MNIST_demo.ipynb for a demostration of the usage of Interactive Tensorflow in a simple training script.

Server

Import Interactive Tensorflow server.

import path.to.interactive_tensorflow.server as server

Create Tensors for your hyper-parameters.

learning_rate = tf.get_variable("learning_rate", initializer=tf.constant(0.1, dtype=tf.float32))
dropout_prob = tf.get_variable("dropout_prob", initializer=tf.constant(0.9, dtype=tf.float32))

Create and start a Session Server.

# "session" is a Tensorflow session
s, thread = server.run_server([learning_rate, dropout_prob], session)

Periodically check for events.

# "step" is the global step of your training procedure
s.check_events(step)

Stop the server.

s.shutdown()
thread.join(timeout=10)

Client

Get status.

python client.py -s

Add an event (this event sets the learning rate to 0.01 at iteration 10000).

python client.py -a -n learning_rate:0 -i 10000 --value 0.01

Remove an event (with index 0 in this case).

python clien.py -r -e 0

Events

Interactive Tensorflow schedules hyper-parameter changes based on events. An event contains the following information:

  • iteration: when to execute the event
  • Tensor name: which Tensor to change
  • value: value to set the Tensor to

The reason for the use of events is that you might want to schedule hyper-parameter changes in the future (e.g. lower learning rate to 10e-3 at 800k iteration). If two events targeting the same Tensor are scheduled at the same iteration, the one that was scheduled later is going to be executed.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for tfset, version 1.0
Filename, size File type Python version Upload date Hashes
Filename, size tfset-1.0.tar.gz (5.6 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page