neuralkit

A lightweight, zero-dependency neural network framework built strictly from scratch in Python. No PyTorch, no TensorFlow — just NumPy.

Every forward pass, backward pass, gradient computation, parameter update, and initialization strategy is implemented from first principles. If you can build it from scratch, you can debug it, optimize it, and truly understand it.

Table of Contents

• Installation
• Quick Start
• Architecture Overview
• What's Implemented
  • Layers & Activations
  • Loss Functions & Regularization
  • Optimizers & Schedulers
  • Weight Initialization
  • Data Pipeline & Cross-Validation
  • Metrics & Evaluation
  • Visualization
  • Model Serialization
• Comparison with Other Frameworks
• Documentation
• Contributing
• License

Installation

git clone https://github.com/14k5hy4/neuralkit.git
cd neuralkit
pip install -e .

Core requirement: NumPy (>= 1.20.0). Optional: Matplotlib (for plotting loss curves, confusion matrices, and decision boundaries).

Quick Start

Train a neural network to solve the non-linear XOR classification problem:

import numpy as np
from neuralkit.model import Sequential
from neuralkit.layers import Dense
from neuralkit.activations import Sigmoid
from neuralkit.losses import MSELoss
from neuralkit.optimizers import SGD
from neuralkit.trainer import Trainer

# XOR dataset
X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
y = np.array([[0], [1], [1], [0]])

# Define model architecture
model = Sequential([
    Dense(2, 8, activation=Sigmoid()),
    Dense(8, 1, activation=Sigmoid()),
])

# Train with full-batch SGD
trainer = Trainer(model, SGD(lr=2.0), MSELoss())
history = trainer.fit(X, y, epochs=3000, verbose=False)

# Inference
predictions = model.predict(X)
print("Predictions:\n", np.round(predictions, 2))
# [[0.02], [0.98], [0.98], [0.02]]

Explore runnable examples in examples/:

• XOR Classification
• Iris Multi-Class Classification
• Synthetic Sine Wave Regression

Architecture Overview

                      +-------------------+
                      |   ArrayDataset    |
                      +---------+---------+
                                |
                                v
                      +-------------------+
                      |    DataLoader     |
                      +---------+---------+
                                |  (x_batch, y_batch)
                                v
+------------------+  +-------------------+  +-------------------+
|   Optimizer      |<--|     Trainer       |-->|   Loss Function   |
| (SGD/Adam/RMS)   |  +---------+---------+  | (MSE/CrossEnt)    |
+--------+---------+            |            +---------+---------+
         |                      v                      |
         |            +-------------------+            |
         +----------->|    Sequential     |<-----------+
                      +---------+---------+
                                |
                +---------------+---------------+
                |               |               |
                v               v               v
         +------------+  +------------+  +------------+
         |  Dense #1  |  |  Dropout   |  |  Dense #2  |
         +------------+  +------------+  +------------+

Execution Flow per Iteration:

1. DataLoader generates mini-batches.
2. Sequential.forward(x_batch) cascades activations through stacked Layer objects.
3. LossFunction.forward(pred, y_batch) evaluates scalar loss + penalty from Regularizer.
4. LossFunction.backward() computes initial upstream gradient $\frac{\partial L}{\partial y}$.
5. Sequential.backward(grad) propagates gradients via chain rule across all layers.
6. Optimizer.step(layers) updates parameters $\mathbf{W} \leftarrow \mathbf{W} - \eta \cdot \nabla_{\mathbf{W}} L$.

What's Implemented

Layers & Activations

Loss Functions & Regularization

Optimizers & Schedulers

Weight Initialization

Data Pipeline & Cross-Validation

Metrics & Evaluation

Visualization

Model Serialization

Save and reload fully trained model state without re-compiling:

# Save model definition and weights
model.save("saved_models/iris_classifier")

# Reload model
from neuralkit.model import Sequential
model = Sequential.load("saved_models/iris_classifier")

• architecture.json stores layer types, dimensions, and activation metadata.
• weights.npz stores binary compressed weight and bias tensors.

Comparison with Other Frameworks

Documentation

Full guides and detailed API documentation are available in the docs/ directory:

• Getting Started Guide: Detailed walkthrough for building models, using callbacks, metrics, and data pipelines.
• API Reference: Complete signature breakdown for all modules.

Contributing

Contributions are welcome! If you'd like to extend neuralkit:

1. Fork the repository.
2. Create a feature branch (git checkout -b feature/amazing-feature).
3. Add tests in tests/ for any new functionality.
4. Ensure all tests pass: python -m unittest discover tests.
5. Submit a Pull Request.

License

Distributed under the MIT License. See LICENSE for more details.

See CHANGELOG.md for version history.