knoxtinytorch 0.1.1

A tiny PyTorch-like neural network library built from scratch using NumPy

TinyTorch

A tiny PyTorch-like neural network library built from scratch using NumPy.

This project was created to understand how deep learning frameworks like PyTorch work internally.

---

Features

• Linear layers
• Activation functions (ReLU, Sigmoid, Tanh, LeakyReLU, ELU, PReLU)
• Loss functions
• Optimizers (SGD, Momentum, RMSProp, Adam)
• Batch Normalization
• Dropout
• Sequential model API
• Automatic parameter detection

---

Installation

pip install knoxtinytorch

---

Quick Example

import numpy as np
from tinytorch import Sequential, Linear, ReLU
from tinytorch import MeanSquaredError, Adam

X = np.array([[0,0],[0,1],[1,0],[1,1]])
y = np.array([[0],[1],[1],[0]])

model = Sequential(
    Linear(2,8),
    ReLU(),
    Linear(8,1)
)

loss_fn = MeanSquaredError()
optimizer = Adam(model.parameters(), lr=0.05)

for epoch in range(5000):

    y_pred = model(X)

    loss = loss_fn.forward(y_pred, y)

    grad = loss_fn.backward()

    model.backward(grad)

    optimizer.step()
    model.zero_grad()

---

Example Project

Train a neural network to learn the XOR problem.

See:

examples/xor_example.py

---

Project Motivation

Most deep learning users rely on frameworks like PyTorch or TensorFlow without understanding how they work internally.

This project recreates the core components of a deep learning framework from scratch using only NumPy.

---

GitHub

https://github.com/KnoxCodes/TinyTorch