scalargrad 1.0.5

A minimal scalar-valued autograd engine for automatic differentiation

ScalarGrad

A minimal scalar-valued automatic differentiation (autograd) engine with neural network capabilities, built from scratch in Python.

Features

• 🔥 Core Autograd Engine: Reverse-mode automatic differentiation for scalar values
• 🧠 Neural Networks: Full-featured MLP implementation with customizable architectures
• 📊 Optimizers: SGD with momentum and Adam optimizer
• 📉 Loss Functions: MSE and SVM loss implementations
• 🎨 Visualization: Computation graph and training metric visualization
• ✨ Rich Terminal Output: Themed console output, animated progress bars, and styled tables via rich
• 🌈 Theme System: Five built-in palettes (default, dark, cyberpunk, light, minimal) with live switching and custom theme registration
• 🛠️ Production-Ready: Comprehensive logging, error handling, and configuration management

Installation

From PyPI (when published)

pip install scalargrad

From Source

git clone https://github.com/THAMIZH-ARASU/scalargrad.git
cd scalargrad
pip install -e .

Dependencies

rich>=13.0.0 is a required runtime dependency and is installed automatically.

Optional Dependencies

For visualization features:

pip install scalargrad[viz]      # For computation graph visualization (requires Graphviz binary)
pip install scalargrad[plot]     # For training plots and decision boundaries
pip install scalargrad[all]      # Install all optional dependencies

Windows / Graphviz note: After pip install scalargrad[viz], also install the Graphviz binary:

winget install graphviz

Then add C:\Program Files\Graphviz\bin to your PATH.

For development:

pip install -r requirements-dev.txt

Quick Start

Basic Autograd Example

from scalargrad import Scalar

# Create scalar values
a = Scalar(2.0, label='a')
b = Scalar(3.0, label='b')

# Build computation graph
c = a * b + b ** 2
c.label = 'c'

# Compute gradients
c.backward()

print(f"c = {c.data}")       # c = 15.0
print(f"dc/da = {a.grad}")   # dc/da = 3.0
print(f"dc/db = {b.grad}")   # dc/db = 8.0

Neural Network Training

from scalargrad import MLP, Adam, MSELoss, Trainer

# Create a neural network
model = MLP(
    nin=2,                              # 2 input features
    layer_sizes=[16, 16, 1],            # Hidden layers and output
    activations=['relu', 'relu', 'linear']
)

# Setup training
optimizer = Adam(model.parameters(), lr=0.01)
loss_fn = MSELoss()
trainer = Trainer(model, optimizer, loss_fn)

# Training data
X = [[0, 0], [0, 1], [1, 0], [1, 1]]
y = [0, 1, 1, 0]  # XOR problem

# Train the model
history = trainer.fit(X, y, epochs=100)

Architecture

The package is organized into the following modules:

scalargrad/
├── core.py           # Core Scalar class and autograd engine
├── config.py         # Configuration, logging (RichHandler), theme/display fields
├── theme.py          # StyleTheme dataclass, THEMES dict, console proxy, theme API
├── nn/               # Neural network components
│   ├── module.py     # Base Module class
│   ├── neuron.py     # Neuron implementation
│   ├── layer.py      # Layer implementation
│   └── mlp.py        # MLP implementation
├── optim/            # Optimization algorithms
│   ├── optimizer.py  # Base Optimizer class
│   ├── sgd.py        # SGD with momentum
│   └── adam.py       # Adam optimizer
├── loss.py           # Loss functions
├── training.py       # Trainer with Progress / Live-table display modes
└── visualization.py  # Computation graph and training metric visualization

API Reference

Core Components

Scalar

The fundamental building block representing a scalar value with automatic differentiation.

from scalargrad import Scalar

x = Scalar(5.0, label='x')
y = x ** 2 + 3 * x + 1
y.backward()  # Compute gradients

Supported Operations:

• Arithmetic: +, -, *, /, **
• Activation functions: relu(), tanh(), sigmoid()
• Mathematical: exp(), log()

Module

Base class for all neural network components.

from scalargrad.nn import Module

class CustomLayer(Module):
    def __init__(self):
        super().__init__()
        # Initialize parameters
    
    def __call__(self, x):
        # Forward pass
        pass
    
    def parameters(self):
        # Return trainable parameters
        pass

Neural Networks

MLP (Multi-Layer Perceptron)

from scalargrad import MLP

model = MLP(
    nin=3,                           # Input features
    layer_sizes=[16, 16, 1],         # Layer sizes
    activations=['relu', 'relu', 'linear'],  # Activations per layer
    init_scale=1.0                   # Weight initialization scale
)

Optimizers

SGD (Stochastic Gradient Descent)

from scalargrad import SGD

optimizer = SGD(
    parameters=model.parameters(),
    lr=0.01,          # Learning rate
    momentum=0.9      # Momentum coefficient
)

Adam

from scalargrad import Adam

optimizer = Adam(
    parameters=model.parameters(),
    lr=0.001,         # Learning rate
    beta1=0.9,        # First moment decay
    beta2=0.999,      # Second moment decay
    eps=1e-8          # Numerical stability
)

Loss Functions

MSELoss (Mean Squared Error)

from scalargrad import MSELoss

loss_fn = MSELoss()
loss = loss_fn(predictions, targets)

SVMLoss (Support Vector Machine Loss)

from scalargrad import SVMLoss

loss_fn = SVMLoss()
loss = loss_fn(predictions, targets)  # targets should be -1 or 1

Training

Trainer

from scalargrad import Trainer

trainer = Trainer(
    model=model,
    optimizer=optimizer,
    loss_fn=loss_fn
)

history = trainer.fit(
    X=training_data,
    y=training_labels,
    epochs=100,
    batch_size=32,      # None for full batch
    verbose=True
)

Visualization

Computation Graph

from scalargrad import Visualizer

# Visualize computation graph
graph = Visualizer.draw_graph(output_scalar, format='svg')
graph.render('computation_graph', view=True)

Training History

# Plot training metrics
Visualizer.plot_training_history(trainer.history)

Decision Boundary

# Plot decision boundary (for 2D data)
Visualizer.plot_decision_boundary(model, X, y)

Examples

Check the examples/ directory for complete examples:

• basic_operations.py: Demonstrates basic autograd functionality
• neural_network.py: Full neural network training example

Run examples:

python examples/basic_operations.py
python examples/neural_network.py

Running Tests

# Install development dependencies
pip install -r requirements-dev.txt

# Run tests
pytest

# Run with coverage
pytest --cov=scalargrad --cov-report=html

Publishing to PyPI

1. Build the package:

python -m build

2. Upload to TestPyPI (optional):

python -m twine upload --repository testpypi dist/*

3. Upload to PyPI:

python -m twine upload dist/*

Theming

ScalarGrad ships with five built-in colour palettes:

Name	Description
default	Blue-toned professional look
dark	Muted greys for dark terminals
cyberpunk	High-contrast neon cyan/magenta
light	Subdued colours for light terminals
minimal	Monochrome, no colour

from scalargrad import set_theme, list_themes, register_theme, StyleTheme

# Switch theme globally
set_theme("cyberpunk")

# List available themes
list_themes()

# Register a custom theme
register_theme("my_theme", StyleTheme(
    header="bold yellow",
    value="cyan",
    success="green",
    # ... other fields use defaults
))

You can also switch the theme via the global config object:

from scalargrad import config
config.theme = "dark"   # live switch — no re-import needed

Configuration

Global configuration can be modified:

from scalargrad import config, LogLevel

config.log_level = LogLevel.DEBUG
config.seed = 42
config.gradient_clip = 1.0   # Clip gradients to this magnitude

# Theme ("default", "dark", "cyberpunk", "light", "minimal", or custom)
config.theme = "cyberpunk"

# Training display mode: "progress" (animated bar) or "table" (live epoch table)
config.training_display = "table"

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

Inspired by:

• micrograd by Andrej Karpathy
• PyTorch's autograd system
• Modern deep learning frameworks

Citation

If you use ScalarGrad in your research, please cite:

@software{scalargrad2024,
  title = {ScalarGrad: A Production-Grade Scalar-Valued Autograd Engine},
  author = {Thamizharasu Saravanan},
  year = {2026},
  url = {https://github.com/THAMIZH-ARASU/scalargrad}
}

Support

For questions and support:

• Open an issue on GitHub
• Check the documentation

---

Made with ❤️ for the deep learning community