Skip to main content

A library of neural nets in theano

Project description

This package contains implementations of several common neural network structures, using Theano for optimization, symbolic differentiation, and transparent GPU computations. Some things it does:

  • Provides several common neural network models:
    • Feedforward Classifier, Autoencoder, Regression
    • Recurrent Classifier, Autoencoder, Regression, Prediction
    • Easy to specify models with any number of layers
  • Allows for many different types of regularization:
    • L1 and L2 weight decay
    • L1 and L2 hidden activation penalties (e.g., sparse autoencoders)
    • Dropout/gaussian noise on inputs (e.g., denoising autoencoders)
    • Dropout/gaussian noise on hidden units
    • Implement custom regularization with a bit of Python code
  • Implements several optimization algorithms:
    • SGD variants: NAG, Rprop, RmsProp
    • Many algorithms in scipy.optimize.minimize
    • Hessian-Free (not currently compatible with Python3)
    • Greedy layerwise pre-training
  • Compatible with Python2 and Python3
  • Configure many parameters from the command-line
  • Relatively easy to extend

At present there are no RBMs, convolutions, or maxout in theanets.

Installation

Install the latest published code using pip:

pip install theanets

Or download the current source and run it from there:

git clone http://github.com/lmjohns3/theano-nets
cd theano-nets
python setup.py develop

Getting started

There are a few example scripts in the examples directory. You can run these from the command-line:

python examples/mnist-autoencoder.py

This example trains an autoencoder with a single hidden layer to reconstruct images of handwritten digits from the MNIST dataset.

Command-line configuration

The theanets package comes built-in with several network models and optimization algorithms available. Many of the available options can be configured from the command-line. To get help on the command-line options, run an example with the --help flag:

python examples/mnist-autoencoder.py

There are many arguments, but some of the notable ones are:

-n or --layers N1 N2 N3 N4

Builds a network with N1 inputs, two hidden layers with N2 and N3 units, and N4 outputs. (Note that this argument is held constant in most of the examples, since it needs to correspond to the shape of the data being processed.)

::
-g or –hidden-activation logistic|relu|linear|…

Use the given activation function for hidden layer units. (Output layer units have a linear activation function by default, but an alternative can be given using the --output-activation flag.) Several activation functions can be pipelined together using the plus symbol.

::
-O or –optimize sgd|hf|sgd hf|layerwise hf|…

Use the given optimization algorithm(s) to train network parameters. Several training algorithms can be used in sequence by separating their names with spaces on the command line.

Using the library

Probably the easiest way to start with the library is to copy one of the examples and modify it to perform your tasks. The usual workflow involves instantiating theanets.Experiment with a subclass of theanets.Network, and then calling train() to learn a good set of parameters for your data:

exp = theanets.Experiment(theanets.Classifier)
exp.train(my_dataset[:1000], my_dataset[1000:])

You can save() the trained model to a pickle, or use the trained network directly to predict() the outputs on a new dataset:

print(exp.network.predict(new_dataset))
exp.save('network-pickle.pkl.gz')

Documentation and support

The package documentation lives at readthedocs. The documentation is relatively sparse, so please file bugs if you find that there’s a particularly hard-to-understand area in the code.

For project announcements and discussion, subscribe to the project mailing list.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for theanets, version 0.4.0
Filename, size File type Python version Upload date Hashes
Filename, size theanets-0.4.0-py2-none-any.whl (38.4 kB) File type Wheel Python version 2.7 Upload date Hashes View hashes
Filename, size theanets-0.4.0-py3-none-any.whl (38.4 kB) File type Wheel Python version 3.4 Upload date Hashes View hashes
Filename, size theanets-0.4.0.tar.gz (28.8 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page