npdl 0.4.0.1

Deep Learning Library based on pure Numpy

NumpyDL: Numpy Deep Learning Library

Descriptions

NumpyDL is:

1. Based on Pure Numpy/Python

2. For DL Education

3. And for My Homework

Features

Its main features are:

1. Pure in Numpy

2. Native to Python

3. Automatic differentiations are basically supported

4. Commonly used models are provided: MLP, RNNs, LSTMs and CNNs

5. API like Keras library

6. Examples for several AI tasks

7. Application for a toy chatbot

8. Mobile friendly documents

Documentation

Available online documents:

1. latest docs

2. development docs

3. stable docs

Available offline PDF:

1. latest PDF

Installation

Install NumpyDL using pip:

$> pip install npdl

Install from source code:

$> python setup.py install

Examples

NumpyDL provides several examples of AI tasks:

• sentence classification

  • LSTM in examples/lstm_sentence_classification.py

  • CNN in examples/cnn_sentence_classification.py

• mnist handwritten recognition

  • MLP in examples/mlp-mnist.py

  • MLP in examples/mlp-digits.py

  • CNN in examples/cnn-minist.py

• language modeling

  • RNN in examples/rnn-character-lm.py

  • LSTM in examples/lstm-character-lm.py

One concrete code example in examples/mlp-digits.py:

import numpy as np
from sklearn.datasets import load_digits
import npdl

# prepare
npdl.utils.random.set_seed(1234)

# data
digits = load_digits()
X_train = digits.data
X_train /= np.max(X_train)
Y_train = digits.target
n_classes = np.unique(Y_train).size

# model
model = npdl.model.Model()
model.add(npdl.layers.Dense(n_out=500, n_in=64, activation=npdl.activation.ReLU()))
model.add(npdl.layers.Dense(n_out=n_classes, activation=npdl.activation.Softmax()))
model.compile(loss=npdl.objectives.SCCE(), optimizer=npdl.optimizers.SGD(lr=0.005))

# train
model.fit(X_train, npdl.utils.data.one_hot(Y_train), max_iter=150, validation_split=0.1)

Applications

NumpyDL provides one toy application:

• Chatbot

  • seq2seq in applications/chatbot/model.py

And its final result:

Supports

NumpyDL supports following deep learning techniques:

• Layers

  • Linear

  • Dense

  • Softmax

  • Dropout

  • Convolution

  • Embedding

  • BatchNormal

  • MeanPooling

  • MaxPooling

  • SimpleRNN

  • GRU

  • LSTM

  • Flatten

  • DimShuffle

• Optimizers

  • SGD

  • Momentum

  • NesterovMomentum

  • Adagrad

  • RMSprop

  • Adadelta

  • Adam

  • Adamax

• Objectives

  • MeanSquaredError

  • HellingerDistance

  • BinaryCrossEntropy

  • SoftmaxCategoricalCrossEntropy

• Initializations

  • Zero

  • One

  • Uniform

  • Normal

  • LecunUniform

  • GlorotUniform

  • GlorotNormal

  • HeNormal

  • HeUniform

  • Orthogonal

• Activations

  • Sigmoid

  • Tanh

  • ReLU

  • Linear

  • Softmax

  • Elliot

  • SymmetricElliot

  • SoftPlus

  • SoftSign

Changelog

0.4.0 (2017.-06-18)

Version 0.4.0.

• Embedding backward

• Momentum

• NesterovMomentum

• Adagrad

• RMSprop

• Adadelta

• Adam

• Adamax

0.3.0 (2017-06-15)

Version 0.3.0.

• Add chatbot application.

• Add more examples.

• Support LSTM.

• Support GRU.

0.2.5 (2017-05-30)

Version 0.2.5.

Add almost all test.

0.2 (2017-05-10)

Second release.

Support Layers:

• Batch Normalization Layer

• Embedding Layer

• MeanPooling Layer

• Flatten Layer

Support Activations:

• SymmetricElliot

• LReLU

• SoftPlus

• SoftSign

Support Initializations:

• HeNormal

• HeUniform

• Orthogonal

Add more tutorials.

Add more API comments.

0.1 (2017-04-11)

First release.

Support layers:

• Dense (perceptron) Layer

• Softmax Layer

• Dropout Layer

• Convolution Layer

• MaxPooling Layer

• SimpleRNN Layer

Support Activations:

• Sigmoid

• Tanh

• ReLU

• Softmax

• Elliot

Support Initializations:

• Uniform

• Normal

• LecunUniform

• GlorotUniform

• GlorotNormal

Support Objectives:

• MeanSquaredError

• HellingerDistance

• BinaryCrossEntropy

• SoftmaxCategoricalCrossEntropy

Support Optimizers:

• SGD