tensorflow meta-framework
Project description
IceFlow
=======
_ice floe, nowhere to go_
A lightweight meta-framework for training neural networks with [TensorFlow](https://www.tensorflow.org/).
Installation
------------
pip install iceflow
### Dependencies
- `tensorflow>=1.3.0`
- `dm-sonnet>=1.11`
Quick start
-----------
1. Define [Sonnet modules](https://deepmind.github.io/sonnet/) in `models.py`:
import tensorflow as tf
import sonnet as snt
class MLP(snt.AbstractModule):
def __init__(self, hidden_size, output_size, nonlinearity=tf.tanh):
super(MLP, self).__init__()
self._hidden_size = hidden_size
self._output_size = output_size
self._nonlinearity = nonlinearity
def _build(self, inputs):
lin_x_to_h = snt.Linear(output_size=self._hidden_size, name="x_to_h")
lin_h_to_o = snt.Linear(output_size=self._output_size, name="h_to_o")
return lin_h_to_o(self._nonlinearity(lin_x_to_h(inputs)))
2. Define [Datasets](https://www.tensorflow.org/programmers_guide/datasets)
in `datasets.py`:
from tensorflow.contrib.data import Dataset
from tensorflow.examples.tutorials.mnist import input_data
def mnist():
# load mnist data
mnist = input_data.read_data_sets('MNIST_data', one_hot=True)
# make Datasets
train_dataset = Dataset.from_tensor_slices(
(mnist.train._images, mnist.train._labels))
test_dataset = Dataset.from_tensor_slices(
(mnist.test._images, mnist.test._labels))
return train_dataset, test_dataset
3. Describe what you want to do in `test1.cfg`:
[DEFAULT]
model_dir=test1
model=MLP
hidden_size=50
output_size=10
4. Train your model, evaluating every 1000 steps:
$ iceflow train test1.cfg mnist --eval_period 1000
5. Evaluate your model:
$ iceflow eval test1.cfg mnist
{'global_step': 10000, 'loss': 0.13652229, 'accuracy': 0.96079999}
6. Visualize your learning in TensorBoard:
$ tensorboard --logdir=test1
Navigate to <http://localhost:6006> to see the metrics:
![](images/tensorboard.png)
7. Add some new data to `datasets.py`
import numpy as np
def random_image():
return None, Dataset.from_tensors(
np.random.random((784,)).astype(np.float32))
def random_images():
return None, Dataset.from_tensor_slices(
np.random.random((32, 784,)).astype(np.float32))
and make predictions on it
$ iceflow predict test1.cfg random_image
[5]
$ iceflow predict test1.cfg random_images
[5, 5, 5, 5, 3, 5, 5, 5, 5, 5, 3, 5, 5, 5, 5, 5, 3, 5, 5, 5, 5, 5, 3, 3, 5, 5, 5, 5, 5, 5, 3, 5]
Config format reference
-----------------------
The format of the `iceflow` config file is roughly
[DEFAULT]
model_dir=test1
model=MLP
hyperparam_1=50
hyperparam_2=10
[more_hiddens]
model_dir=test2
hyperparam_1=100
To train the model defined in the `[DEFAULT]` section, run
$ iceflow train <config_file> <dataset>
To train the `[more_hiddens]` variant model, which inherits all hyperparameters
from the `[DEFAULT]` section but overrides `model_dir` (to avoid conflicting
with the `[DEFAULT]` model) and `hyperparam_1`, run
$ iceflow train <config_file> <dataset> --config_section more_hiddens
`model` must refer to a Sonnet module defined in `models.py`.
Every key besides `model_dir` and `model` is taken to be a hyperparameter which
will be passed as a kwarg to the constructor of the Sonnet module.
Design philosophy
-----------------
Our typical workload involves training lots of models (usually with complex or
experimental architecture) with different sets of hyperparameters on different
datasets.
Previously, we had been using a hand-built meta-framework around TensorFlow to
organize training, evaluation, and inference.
As of TensorFlow 1.3, the [Dataset API](https://www.tensorflow.org/programmers_guide/datasets),
[Estimator API](https://www.tensorflow.org/programmers_guide/estimators), and
[DeepMind's Sonnet library](https://deepmind.github.io/sonnet/) have arisen as
mature alternatives to our hand-crafted solutions.
IceFlow aims to provide the small bit of code needed to get these three APIs to
work together seamlessly - without sacrificing flexibility - and provide an
efficient "command line and config file"-based interface to the basic train,
eval, predict cycle.
Caveats and future directions
-----------------------------
- Currently, the only supported type of problem is a softmax classification
problem with one-hot labels. We plan to extend this.
- Currently, the only possible output you can obtain from `iceflow predict` is
tensors being printed to the command line. We plan to extend this to allow
specification of an arbitrary Python function that takes the prediction
results (arrays) as input.
- Currently, the optimizer used for training is hard-coded. We plan to expose
this as a parameter either in the config or on the command line. We also plan
to extend this to support learning rate decay and related use cases.
- Currently, there is no easy way to use IceFlow to inject a properly-restored
Estimator into arbitrary Python code. We plan to add this capability.
- Currently, the batch size and shuffle buffer size are not exposed. We plan to
expose this soon.
- Currently, performing validation every so often during training is very
awkward. We are awaiting the return of [`ValidationMonitor`](https://www.tensorflow.org/get_started/monitors#configuring_a_validationmonitor_for_streaming_evaluation)
from its banishment in the desert of deprecation (and following
[this GitHub issue](https://github.com/tensorflow/tensorflow/issues/7669)).
=======
_ice floe, nowhere to go_
A lightweight meta-framework for training neural networks with [TensorFlow](https://www.tensorflow.org/).
Installation
------------
pip install iceflow
### Dependencies
- `tensorflow>=1.3.0`
- `dm-sonnet>=1.11`
Quick start
-----------
1. Define [Sonnet modules](https://deepmind.github.io/sonnet/) in `models.py`:
import tensorflow as tf
import sonnet as snt
class MLP(snt.AbstractModule):
def __init__(self, hidden_size, output_size, nonlinearity=tf.tanh):
super(MLP, self).__init__()
self._hidden_size = hidden_size
self._output_size = output_size
self._nonlinearity = nonlinearity
def _build(self, inputs):
lin_x_to_h = snt.Linear(output_size=self._hidden_size, name="x_to_h")
lin_h_to_o = snt.Linear(output_size=self._output_size, name="h_to_o")
return lin_h_to_o(self._nonlinearity(lin_x_to_h(inputs)))
2. Define [Datasets](https://www.tensorflow.org/programmers_guide/datasets)
in `datasets.py`:
from tensorflow.contrib.data import Dataset
from tensorflow.examples.tutorials.mnist import input_data
def mnist():
# load mnist data
mnist = input_data.read_data_sets('MNIST_data', one_hot=True)
# make Datasets
train_dataset = Dataset.from_tensor_slices(
(mnist.train._images, mnist.train._labels))
test_dataset = Dataset.from_tensor_slices(
(mnist.test._images, mnist.test._labels))
return train_dataset, test_dataset
3. Describe what you want to do in `test1.cfg`:
[DEFAULT]
model_dir=test1
model=MLP
hidden_size=50
output_size=10
4. Train your model, evaluating every 1000 steps:
$ iceflow train test1.cfg mnist --eval_period 1000
5. Evaluate your model:
$ iceflow eval test1.cfg mnist
{'global_step': 10000, 'loss': 0.13652229, 'accuracy': 0.96079999}
6. Visualize your learning in TensorBoard:
$ tensorboard --logdir=test1
Navigate to <http://localhost:6006> to see the metrics:
![](images/tensorboard.png)
7. Add some new data to `datasets.py`
import numpy as np
def random_image():
return None, Dataset.from_tensors(
np.random.random((784,)).astype(np.float32))
def random_images():
return None, Dataset.from_tensor_slices(
np.random.random((32, 784,)).astype(np.float32))
and make predictions on it
$ iceflow predict test1.cfg random_image
[5]
$ iceflow predict test1.cfg random_images
[5, 5, 5, 5, 3, 5, 5, 5, 5, 5, 3, 5, 5, 5, 5, 5, 3, 5, 5, 5, 5, 5, 3, 3, 5, 5, 5, 5, 5, 5, 3, 5]
Config format reference
-----------------------
The format of the `iceflow` config file is roughly
[DEFAULT]
model_dir=test1
model=MLP
hyperparam_1=50
hyperparam_2=10
[more_hiddens]
model_dir=test2
hyperparam_1=100
To train the model defined in the `[DEFAULT]` section, run
$ iceflow train <config_file> <dataset>
To train the `[more_hiddens]` variant model, which inherits all hyperparameters
from the `[DEFAULT]` section but overrides `model_dir` (to avoid conflicting
with the `[DEFAULT]` model) and `hyperparam_1`, run
$ iceflow train <config_file> <dataset> --config_section more_hiddens
`model` must refer to a Sonnet module defined in `models.py`.
Every key besides `model_dir` and `model` is taken to be a hyperparameter which
will be passed as a kwarg to the constructor of the Sonnet module.
Design philosophy
-----------------
Our typical workload involves training lots of models (usually with complex or
experimental architecture) with different sets of hyperparameters on different
datasets.
Previously, we had been using a hand-built meta-framework around TensorFlow to
organize training, evaluation, and inference.
As of TensorFlow 1.3, the [Dataset API](https://www.tensorflow.org/programmers_guide/datasets),
[Estimator API](https://www.tensorflow.org/programmers_guide/estimators), and
[DeepMind's Sonnet library](https://deepmind.github.io/sonnet/) have arisen as
mature alternatives to our hand-crafted solutions.
IceFlow aims to provide the small bit of code needed to get these three APIs to
work together seamlessly - without sacrificing flexibility - and provide an
efficient "command line and config file"-based interface to the basic train,
eval, predict cycle.
Caveats and future directions
-----------------------------
- Currently, the only supported type of problem is a softmax classification
problem with one-hot labels. We plan to extend this.
- Currently, the only possible output you can obtain from `iceflow predict` is
tensors being printed to the command line. We plan to extend this to allow
specification of an arbitrary Python function that takes the prediction
results (arrays) as input.
- Currently, the optimizer used for training is hard-coded. We plan to expose
this as a parameter either in the config or on the command line. We also plan
to extend this to support learning rate decay and related use cases.
- Currently, there is no easy way to use IceFlow to inject a properly-restored
Estimator into arbitrary Python code. We plan to add this capability.
- Currently, the batch size and shuffle buffer size are not exposed. We plan to
expose this soon.
- Currently, performing validation every so often during training is very
awkward. We are awaiting the return of [`ValidationMonitor`](https://www.tensorflow.org/get_started/monitors#configuring_a_validationmonitor_for_streaming_evaluation)
from its banishment in the desert of deprecation (and following
[this GitHub issue](https://github.com/tensorflow/tensorflow/issues/7669)).
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
iceflow-0.0.1a2.tar.gz
(21.6 kB
view hashes)
Built Distribution
Close
Hashes for iceflow-0.0.1a2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5c59d3fc8630e58acc5509eb9993d834fcfe390300a22b7e2cfc389fc4fc0c3f |
|
MD5 | a18495a85160759dff5221f08a4398e7 |
|
BLAKE2b-256 | b52236dea2f26cf8947da954044b0b0ac4675c1bb2fdd33f37d5c4e4077c927e |