custom-cnn 0.1.1

A convolutional neural network library built from scratch using NumPy — Conv1d, Conv2d, pooling, transposed convolution, activations, and loss functions with hand-derived forward and backward passes.

custom-cnn: Convolutional Neural Network Library from Scratch

A lightweight convolutional neural network library built entirely in NumPy and SciPy. This project implements forward and backward passes for all standard CNN components — convolution, pooling, transposed convolution, resampling, and more — without the use of high-level frameworks like PyTorch or TensorFlow.

Installation

pip install custom-cnn

Quick Start

import numpy as np
from mytorch.nn.Conv1d import Conv1d
from mytorch.nn.activation import ReLU
from mytorch.nn.loss import CrossEntropyLoss
from mytorch.flatten import Flatten
from mytorch.nn.linear import Linear

# Build a simple 1D CNN
conv = Conv1d(in_channels=3, out_channels=8, kernel_size=5, stride=1)
relu = ReLU()
flatten = Flatten()
fc = Linear(in_features=8 * 28, out_features=10)
criterion = CrossEntropyLoss()

# Forward pass
x = np.random.randn(4, 3, 32)  # (batch=4, channels=3, width=32)
z = conv.forward(x)
z = relu.forward(z)
z = flatten.forward(z)
z = fc.forward(z)

# Compute loss and backprop
labels = np.eye(10)[np.array([0, 1, 2, 3])]  # one-hot
loss = criterion.forward(z, labels)
grad = criterion.backward()
grad = fc.backward(grad)
grad = flatten.backward(grad)
grad = relu.backward(grad)
grad = conv.backward(grad)

Or use the pre-built CNN model:

from models.cnn import CNN
from mytorch.nn.activation import ReLU
from mytorch.nn.loss import CrossEntropyLoss

model = CNN(
    input_width=32,
    num_input_channels=3,
    num_channels=[8, 16, 32],
    kernel_sizes=[5, 3, 3],
    strides=[1, 1, 1],
    num_linear_neurons=10,
    activations=[ReLU(), ReLU(), ReLU()],
    conv_weight_init_fn=None,
    bias_init_fn=None,
    linear_weight_init_fn=None,
    criterion=CrossEntropyLoss(),
    lr=0.01,
)

# Single training step
output = model.forward(x)
model.backward(labels)
model.step()

Key Features

• Convolution Layers: Conv1d and Conv2d with arbitrary stride, implemented as stride-1 convolution followed by downsampling. Full cross-correlation forward and flipped-kernel backward passes.
• Transposed Convolution: ConvTranspose1d and ConvTranspose2d for learnable upsampling, implemented as upsample-then-convolve — the mathematical transpose of strided convolution.
• Pooling: MaxPool2d and MeanPool2d with both stride-1 and arbitrary-stride variants. Max pooling routes gradients to argmax positions; mean pooling distributes gradients uniformly.
• Resampling: Upsample1d/2d (zero-insertion) and Downsample1d/2d (strided selection), with transpose-pair backward passes.
• Activations: Seven activation functions — Identity, Sigmoid, Tanh, ReLU, GELU, Swish (with learnable beta), and Softmax — each with hand-derived derivatives.
• Loss Functions: MSELoss and CrossEntropyLoss with numerically stable Softmax integration using the max-subtraction method to prevent floating-point overflow.
• Pre-built Architectures: Configurable CNN classifier, MLP, and scanning MLP models (simple and distributed weight-sharing) demonstrating the convolution-as-scanning equivalence.

Technical Highlights

1. Vectorized Backpropagation

Every layer implements its own forward() and backward() pass. Gradients are calculated using vectorized matrix operations, with no autograd or computation graph. This makes the math behind backpropagation explicit and inspectable.

2. Strided Convolution via Decomposition

Strided convolution is decomposed into stride-1 convolution + downsampling. This simplifies the backward pass: upsample the gradient, then apply the stride-1 backward. The same principle applies to transposed convolution (upsample + convolve).

3. Numerical Stability

The CrossEntropyLoss implementation subtracts the row-wise maximum before exponentiation, preventing floating-point overflow in the softmax computation. The combined softmax + cross-entropy gradient simplifies to (softmax - labels) / N.

4. Cython-Compiled Distribution

Published on PyPI as pre-compiled binary wheels (.so/.pyd) using Cython. No Python source code is included in the distributed package. Wheels are built for Python 3.9–3.13 on Linux, macOS, and Windows via GitHub Actions and cibuildwheel.

Layers

Convolution

Class	Description
Conv1d_stride1	1D convolution with stride 1
Conv1d	1D convolution with arbitrary stride
Conv2d_stride1	2D convolution with stride 1
Conv2d	2D convolution with arbitrary stride
ConvTranspose1d	1D transposed (deconv) with upsampling
ConvTranspose2d	2D transposed (deconv) with upsampling

Pooling

Class	Description
MaxPool2d_stride1	2D max pooling with stride 1
MaxPool2d	2D max pooling with arbitrary stride
MeanPool2d_stride1	2D mean pooling with stride 1
MeanPool2d	2D mean pooling with arbitrary stride

Resampling

Class	Description
Upsample1d	1D upsampling (zero-insert)
Downsample1d	1D downsampling (strided select)
Upsample2d	2D upsampling
Downsample2d	2D downsampling

Core

Class	Description
Linear	Fully-connected layer
Flatten	Reshape (batch, C, W) to (batch, C*W)

Activations

Class	Description
Identity	Pass-through (no-op)
Sigmoid	Logistic sigmoid
Tanh	Hyperbolic tangent
ReLU	Rectified linear unit
GELU	Gaussian error linear unit
Swish	Swish/SiLU with learnable beta
Softmax	Softmax normalization

Loss Functions

Class	Description
MSELoss	Mean squared error
CrossEntropyLoss	Softmax + cross-entropy

Models

Class	Description
CNN	Configurable Conv1d network with linear head
MLP	Multi-layer perceptron with ReLU activations
CNN_SimpleScanningMLP	Conv1d network equivalent to a scanning MLP
CNN_DistributedScanningMLP	Conv1d network with weight-shared scanning

Project Structure

custom-cnn/
├── mytorch/
│   ├── nn/
│   │   ├── Conv1d.py        # 1D convolution (stride-1 and strided)
│   │   ├── Conv2d.py        # 2D convolution (stride-1 and strided)
│   │   ├── ConvTranspose.py # Transposed convolution (1D & 2D)
│   │   ├── pool.py          # Max & mean pooling (2D)
│   │   ├── resampling.py    # Up/downsampling (1D & 2D)
│   │   ├── linear.py        # Fully-connected layer
│   │   ├── activation.py    # 7 activation functions
│   │   └── loss.py          # MSE & cross-entropy loss
│   └── flatten.py           # Flatten layer
├── models/
│   ├── cnn.py               # CNN classifier
│   ├── mlp.py               # Multi-layer perceptron
│   └── mlp_scan.py          # Scanning MLP (simple & distributed)
├── sandbox/                  # Example/test scripts for each layer
├── pyproject.toml            # Package metadata & build config
└── setup.py                  # Cython compilation setup

Requirements

• Python >= 3.9
• NumPy >= 2.2.6
• SciPy >= 1.15.3

License

All rights reserved.