scratch3dnn 0.1.0

A neural network framework built from scratch using NumPy.

scratch3dnn

A lightweight deep learning framework built from scratch using only NumPy.
Designed for learning how neural networks really work — no PyTorch, no TensorFlow, just math and Python.

---

Features

• Modular architecture — every component extends a common Module base class
• Dense (fully-connected) layers with small-weight initialisation
• Activation functions — ReLU and Sigmoid
• Loss functions — Mean Squared Error (MSE)
• Optimiser — Stochastic Gradient Descent (SGD)
• Sequential network (NeuralNet) that chains layers and handles forward + backward passes automatically

---

Installation

pip install scratch3dnn

Requires Python >= 3.9 and NumPy >= 1.24.

---

Quick Start

import numpy as np
from scratch3dnn import NeuralNet, Layer, Relu, Sigmoid, MSELoss, SGDOptimizer

# Build a simple 2-layer network
model = NeuralNet(
    Layer(4, 8),   # 4 inputs -> 8 hidden units
    Relu(),
    Layer(8, 1),   # 8 hidden -> 1 output
    Sigmoid(),
)

loss_fn = MSELoss()
optimizer = SGDOptimizer(model.get_params(), learning_rate=0.01)

# Dummy dataset
X = np.random.randn(32, 4)   # 32 samples, 4 features
y = np.random.randint(0, 2, (32, 1)).astype(float)

# Training loop
for epoch in range(100):
    # Forward pass
    predictions = model.forward(X)
    loss = loss_fn.forward(predictions, y)

    # Backward pass
    grad = loss_fn.backward()
    model.backward(grad)

    # Update weights
    optimizer.step()
    optimizer.zero_grad()

    if (epoch + 1) % 10 == 0:
        print(f"Epoch {epoch + 1:3d} | Loss: {loss:.6f}")

---

API Reference

Module (base class)

All components inherit from Module.

Method	Description
forward(input_data)	Compute the forward pass
backward(gradient)	Compute the backward pass and return the upstream gradient
get_params()	Return a list of (param, grad) tuples (default: [])

---

Layer(input_size, output_size)

A fully-connected linear layer: y = x W + b.

• Weights initialised with N(0, 0.001), biases initialised to zero.
• Accumulates gradients in w_grad and b_grad during backward pass.

---

Activations

Class	Formula
Relu()	max(0, x)
Sigmoid()	1 / (1 + exp(-x))

---

MSELoss()

Mean Squared Error loss: L = 0.5 * mean((y_hat - y)^2)

loss = loss_fn.forward(predictions, targets)  # scalar
grad = loss_fn.backward()                     # gradient w.r.t. predictions

---

NeuralNet(*layers)

Sequential container — passes data through each layer in order during forward, and in reverse during backward.

model = NeuralNet(Layer(4, 8), Relu(), Layer(8, 1))
out   = model.forward(X)
model.backward(grad)
params = model.get_params()   # flat list of (param, grad) pairs

---

SGDOptimizer(parameters, learning_rate=0.001)

Vanilla Stochastic Gradient Descent.

optimizer = SGDOptimizer(model.get_params(), learning_rate=0.01)
optimizer.step()       # param -= lr * grad
optimizer.zero_grad()  # reset all gradients to 0

---

Project Structure

src/scratch3dnn/
├── __init__.py      # Public API exports
├── module.py        # Abstract Module base class
├── layers.py        # Dense Layer
├── activations.py   # Relu, Sigmoid
├── losses.py        # MSELoss
├── network.py       # NeuralNet (sequential container)
└── optimizers.py    # SGDOptimizer

---

License

MIT (c) Tridibesh Sarkar