Skip to main content

A deep learning framework

Project description

PyNorch

Recreating PyTorch from scratch (C/C++, CUDA and Python, with GPU support and automatic differentiation!)

Project details explanations can also be found on medium.

1 - About

PyNorch is a deep learning framework constructed using C/C++, CUDA and Python. This is a personal project with educational purpose only! Norch means NOT PyTorch, and we have NO claims to rivaling the already established PyTorch. The main objective of PyNorch was to give a brief understanding of how a deep learning framework works internally. It implements the Tensor object, GPU support and an automatic differentiation system.

2 - Installation

Install this package from PyPi (you can test on Colab!)

$ pip install norch

or from cloning this repository

$ sudo apt install nvidia-cuda-toolkit
$ git clone https://github.com/lucasdelimanogueira/PyNorch.git
$ cd build
$ make
$ cd ..

3 - Get started

3.1 - Tensor operations

import norch

x1 = norch.Tensor([[1, 2], 
                  [3, 4]], requires_grad=True).to("cuda")

x2 = norch.Tensor([[4, 3], 
                  [2, 1]], requires_grad=True).to("cuda)

x3 = x1 @ x2
result = x3.sum()
result.backward

print(x1.grad)

3.2 - Create a model

import norch
import norch.nn as nn
import norch.optim as optim

class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.fc1 = nn.Linear(1, 10)
        self.sigmoid = nn.Sigmoid()
        self.fc2 = nn.Linear(10, 1)

    def forward(self, x):
        out = self.fc1(x)
        out = self.sigmoid(out)
        out = self.fc2(out)
        
        return out

3.3 - Example training

import norch
from norch.utils.data.dataloader import Dataloader
from norch.norchvision import transforms
import norch
import norch.nn as nn
import norch.optim as optim
import random
random.seed(1)

BATCH_SIZE = 32
device = "cpu"
epochs = 10

transform = transforms.Sequential(
    [
        transforms.ToTensor(),
        transforms.Reshape([-1, 784, 1])
    ]
)

target_transform = transforms.Sequential(
    [
        transforms.ToTensor()
    ]
)

train_data, test_data = norch.norchvision.datasets.MNIST.splits(transform=transform, target_transform=target_transform)
train_loader = Dataloader(train_data, batch_size = BATCH_SIZE)

class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.fc1 = nn.Linear(784, 30)
        self.sigmoid1 = nn.Sigmoid()
        self.fc2 = nn.Linear(30, 10)
        self.sigmoid2 = nn.Sigmoid()

    def forward(self, x):
        out = self.fc1(x)
        out = self.sigmoid1(out)
        out = self.fc2(out)
        out = self.sigmoid2(out)
        
        return out

model = MyModel().to(device)
criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=0.01)
loss_list = []

for epoch in range(epochs):    
    for idx, batch in enumerate(train_loader):

        inputs, target = batch

        inputs = inputs.to(device)
        target = target.to(device)

        outputs = model(inputs)
        
        loss = criterion(outputs, target)
        
        optimizer.zero_grad()
        
        loss.backward()

        optimizer.step()

    print(f'Epoch [{epoch + 1}/{epochs}], Loss: {loss[0]:.4f}')
    loss_list.append(loss[0])

4 - Progress

Development Status Feature
Operations in progress
  • [X] GPU Support
  • [X] Autograd
  • [X] Broadcasting
Loss in progress
  • [x] MSE
  • [X] Cross Entropy
Data in progress
  • [X] Dataset
  • [X] Batch
  • [X] Iterator
Convolutional Neural Network in progress
  • [ ] Conv2d
  • [ ] MaxPool2d
  • [ ] Dropout
Distributed in progress
  • [ ] Distributed Data Parallel

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

norch-0.0.3.tar.gz (87.6 kB view hashes)

Uploaded Source

Built Distribution

norch-0.0.3-py3-none-any.whl (93.5 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page