neurograd-vec 0.1.0

A micrograd-like autodiff library extended to vectors

NeuroGrad

A micrograd-like ({https://github.com/karpathy/micrograd}) autodiff library extended to support vector operations. NeuroGrad provides automatic differentiation capabilities for building and training neural networks from scratch.

Features

• Vector-based automatic differentiation: Extends micrograd's scalar operations to vectors
• Neural network components: Built-in Neuron, Layer, and MLP classes
• Activation functions: ReLU and Tanh activations with automatic gradient computation
• Simple API: Easy-to-use interface for building neural networks (MLPs only for now)

Installation

Clone the repository and install:

git clone https://github.com/Inomjonov/neurograd.git
cd neurograd
pip install -e .

Or install directly from GitHub:

pip install git+https://github.com/Inomjonov/neurograd.git

Quick Start

from neurograd import VectorValue, Neuron, Layer, MLP

# Create a simple neural network
mlp = MLP(3, [4, 4, 1])

# Forward pass
x = VectorValue([1.0, 2.0, 3.0])
out = mlp(x)

# Backward pass
loss = (out - VectorValue([1.0])) ** 2
loss.backward()

# Access gradients
for p in mlp.parameters():
    print(f"Parameter: {p.data}, Gradient: {p.grad}")

Components

VectorValue

The core class for automatic differentiation with vector support.

from neurograd import VectorValue

a = VectorValue([1.0, 2.0, 3.0])
b = VectorValue([4.0, 5.0, 6.0])

# Operations
c = a + b
d = a * b
e = a.dot(b)
f = a.sum()

Neuron

A single neuron with optional non-linearity.

from neurograd import Neuron

neuron = Neuron(nin=3, nonlin=True)
output = neuron(VectorValue([1.0, 2.0, 3.0]))

Layer

A layer of neurons.

from neurograd import Layer

layer = Layer(nin=3, nout=4)
output = layer(VectorValue([1.0, 2.0, 3.0]))

MLP

Multi-layer perceptron.

from neurograd import MLP

mlp = MLP(nin=3, nouts=[4, 4, 1])
output = mlp(VectorValue([1.0, 2.0, 3.0]))

Disclaimer

NeuroGrad was developed solely for educational purposes — to demonstrate how reverse-mode automatic differentiation and simple neural networks work from scratch. It is not intended for commercial or production use. For real workloads, use a mature framework such as PyTorch, TensorFlow, or JAX.

Known Limitations

This is a minimal teaching library and has several known limitations:

• No scalar broadcasting. Element-wise ops (+, *) require operands of equal length, so mixing a bare scalar with a multi-element vector (e.g. vec * 2, 10 - vec, vec + 1) raises an AssertionError. Negation and vector–vector subtraction do work.
• backward() accumulates across calls. Intermediate-node gradients are not reset, so calling backward() more than once on the same graph double-counts. Rebuild the graph (a fresh forward pass) for each backward, or zero gradients yourself.
• Non-scalar backward() seeds every output with 1.0, i.e. it computes the gradient of the sum of the output components, not a full Jacobian. Call backward() from a scalar (e.g. a loss) for meaningful gradients.
• Recursive topological sort. The backward pass builds its ordering recursively, so very deep computation graphs can hit Python's recursion limit.
• Limited op set. Only the operations needed for MLPs are implemented (no division, no convolutions, no batching, etc.).

License

See LICENSE file for details.

Author

Mironshoh Inomjonov