torchy-nn 0.2.3.3

NumPy based neural network package with PyTorch-like API

Overview

Torchy is a neural network framework implemented only using NumPy and based on PyTorch API but with manual backpropogation calculations. The main idea was to build a neural network from scratch for educational purposes.

Installation

pip install torchy-nn

Getting started

I suggest you to take a look at currently implemented stuff to be familiar with current possibilities for building neural network models with Torchy. Also I've created package structure in case if you stuck where to get specific layers.

Example usage

First we can define our model using Torchy with its PyTorch-like API

from torchy.sequential import Sequential # Same as nn.Sequential
import torchy.layer as layer

# Define 2-layer wtth 100 neurons hidden layer.
model = Sequential(
  layer.Linear(n_input=10, n_output=100),
  layer.BatchNorm1d(n_output=100),
  layer.ReLU(),
  layer.Linear(n_input=100, n_output=2)
)

Next step is to create instances of optimizer and criterion for loss function and scheduler for fun

import torchy.loss as loss
import torchy.optim as optim
import torchy.scheduler as sched

optimizer = optim.SGD(model.params(), lr=1e-3)
criterion = loss.CrossEntropyLoss()
scheduler = sched.StepLR(optimizer, step_size=10)

I won't cover whole training process like loops and stuff, just show you main differences while training

...
predictions = model(X)  # Nothing changed
        
loss, grad = criterion(predictions, y)  # Now return tuple of (loss, grad) instead of only loss 
        
optimizer.zero_grad() 
model.backward(grad)  # Call backward on model object and pass gradient from loss as argument
optimizer.step()

Demonstration

The demo notebook showcases what Torchy currently can do.

Roadmap

There is still a lot of work to be done, but here are the main points that will be completed soon

• Docstring every entity & add type hinting
• Add evaluation & inference for model

Resources

The opportunity to create such a project was given to me thanks to these people

• PyTorch
• CS231n - 2016
• Deep Learning на пальцах - 2019
• Neural Networks: Zero to Hero

License

MIT License