torchy-nn 0.3.5.3

Overview

Torchy is a neural network framework implemented only using NumPy and based on PyTorch API but with manual backpropogation calculations. The main idea was to build a neural network from scratch for educational purposes.

Installation

pip install torchy-nn

Getting started

I suggest you to take a look at currently implemented stuff to be familiar with current possibilities for building neural network models with Torchy. Also I've created package structure in case if you stuck where to get specific layers.

Example usage

First we can define our model using Torchy with its PyTorch-like API

from nn.modules.sequential import Sequential  # Same as nn.Sequential
import nn.modules.module as layer

# Define 2-layer wtth 100 neurons hidden layer.
model = Sequential(
    layer.Linear(n_input=10, n_output=100),
    layer.BatchNorm1d(n_output=100),
    layer.ReLU(),
    layer.Linear(n_input=100, n_output=2)
)

Next step is to create instances of optimizer and criterion for loss function and scheduler for fun

import nn.modules.loss as loss
import optimizers.optim as optim
import optimizers.scheduler as sched

optimizer = optim.SGD(model.params(), lr=1e-3)
criterion = loss.CrossEntropyLoss()
scheduler = sched.StepLR(optimizer, step_size=10)

I won't cover whole training process like loops and stuff, just show you main differences while training

...
predictions = model(X)  # Nothing changed

loss, grad = criterion(predictions, y)  # Now return tuple of (loss, grad) instead of only loss 

optimizer.zero_grad()
model.backward(grad)  # Call backward on model object and pass gradient from loss as argument
optimizer.forward_step()

Demonstration

The demo notebook showcases what Torchy currently can do.

Roadmap

There is still a lot of work to be done, but here are the main points that will be completed soon

• Docstring every entity & add type hinting
• Add evaluation & inference for model

Resources

The opportunity to create such a project was given to me thanks to these people

• PyTorch
• CS231n - 2016
• Deep Learning на пальцах - 2019
• Neural Networks: Zero to Hero

License

MIT License