TensorLib 🚀

A zero-dependency open-source Machine Learning & Deep Learning library built completely from scratch in Python.

TensorLib delivers a PyTorch and Scikit-Learn style interface for tensor operations, automatic differentiation (autograd), neural network building blocks, optimizers, data loaders, and classical machine learning algorithms—all powered by a custom pure-Python numerical engine.

🌟 Key Features

• ⚡ Core Tensor Engine (tensorlib.Tensor)
  • N-dimensional array storage with row-major memory layouts, stride indexing, slicing, reshaping, matrix multiplication, and broadcasting.
• 🔄 Reverse-Mode Automatic Differentiation (tensorlib.autograd)
  • Dynamic computational graph tracking (DAG), topological sorting, and automated reverse backpropagation (.backward()).
• 🧠 Neural Network Framework (tensorlib.nn)
  • Modular Module & Parameter architecture.
  • Layers: Linear (Dense), Conv2D, MaxPool2D, Sequential, Flatten, Dropout, BatchNorm1d.
  • Activations: ReLU, Sigmoid, Tanh, Softmax, LeakyReLU, GELU.
  • Losses: MSELoss, CrossEntropyLoss, BCEWithLogitsLoss, L1Loss.
• 🛠️ Optimizers (tensorlib.optim)
  • SGD (with momentum & weight decay), Adam, AdamW, RMSprop.
• 🤖 Classical Machine Learning Suite (tensorlib.ml)
  • Built directly on top of Tensor primitives:
    • Regression: LinearRegression, LogisticRegression.
    • Trees & Ensembles: DecisionTreeClassifier, RandomForestClassifier.
    • Clustering: KMeans.
    • Neighbors: KNeighborsClassifier.
    • Dimensionality Reduction: PCA.
• 📊 Data Loading & Preprocessing (tensorlib.data)
  • TensorDataset, DataLoader (mini-batching & shuffling), StandardScaler, MinMaxScaler, OneHotEncoder.
• 📈 Metrics & Utilities (tensorlib.metrics & tensorlib.utils)
  • Classification & regression metrics (accuracy_score, f1_score, r2_score, confusion_matrix).
  • Model serialization (save, load) and ASCII computational graph renderer (render_graph).

📁 Repository Structure

TensorLib/
├── pyproject.toml
├── README.md
├── tensorlib/
│   ├── __init__.py
│   ├── tensor.py            # N-dimensional Tensor & math operations
│   ├── autograd.py          # Reverse-mode automatic differentiation engine
│   ├── ops.py               # Pure-Python matrix, broadcasting, and stride operations
│   ├── nn/                  # Neural network layers, activations, and losses
│   ├── optim/               # SGD, Adam, AdamW, RMSprop optimizers
│   ├── ml/                  # Classical ML suite (Regression, Trees, K-Means, KNN, PCA)
│   ├── data/                # Dataset, DataLoader, StandardScaler, OneHotEncoder
│   ├── metrics/             # Accuracy, F1, R2, Confusion Matrix
│   └── utils/               # Model saving/loading & graph rendering
├── tests/                   # Unit test suite (100% standard library unittest)
└── examples/                # Runnable demonstration scripts

⚡ Quickstart

1. Tensor Math & Autograd Computation

from tensorlib import Tensor
from tensorlib.utils import render_graph

# Create tensors with autograd enabled
x = Tensor([[1.0, 2.0], [3.0, 4.0]], requires_grad=True)
W = Tensor([[0.5, -0.5], [1.0, 2.0]], requires_grad=True)

# Forward pass
y = x @ W
loss = (y ** 2).sum()

# Print computational graph
print(render_graph(loss))

# Reverse Backpropagation
loss.backward()

print("x.grad:", x.grad)
print("W.grad:", W.grad)

2. Training a Neural Network (MLP)

from tensorlib import Tensor
from tensorlib.nn import Sequential, Linear, ReLU, CrossEntropyLoss
from tensorlib.optim import Adam
from tensorlib.data import TensorDataset, DataLoader

# Define dataset
X = Tensor([[1.0, 2.0], [1.5, 1.8], [5.0, 5.0], [6.0, 7.0]])
y = Tensor([0.0, 0.0, 1.0, 1.0])

dataset = TensorDataset(X, y)
loader = DataLoader(dataset, batch_size=2, shuffle=True)

# Build model
model = Sequential(
    Linear(in_features=2, out_features=8),
    ReLU(),
    Linear(in_features=8, out_features=2)
)

optimizer = Adam(model.parameters(), lr=0.05)
criterion = CrossEntropyLoss()

# Training loop
for epoch in range(20):
    for batch_X, batch_y in loader:
        optimizer.zero_grad()
        logits = model(batch_X)
        loss = criterion(logits, batch_y)
        loss.backward()
        optimizer.step()

3. Classical Machine Learning

from tensorlib import Tensor
from tensorlib.ml import LogisticRegression, RandomForestClassifier, KMeans, PCA

X = Tensor([[1.0, 1.0], [1.5, 2.0], [6.0, 6.0], [7.0, 8.0]])
y = Tensor([0.0, 0.0, 1.0, 1.0])

# Logistic Regression
clf = LogisticRegression(lr=0.1, epochs=100).fit(X, y)
print("LogReg Predictions:", clf.predict(X).data)

# Random Forest Classifier
rf = RandomForestClassifier(n_estimators=5, max_depth=3).fit(X, y)
print("Random Forest Predictions:", rf.predict(X).data)

# K-Means Clustering
kmeans = KMeans(n_clusters=2, random_state=42).fit(X)
print("Cluster Assignments:", kmeans.predict(X).data)

# Principal Component Analysis
pca = PCA(n_components=1).fit(X)
X_reduced = pca.transform(X)
print("PCA Reduced Shape:", X_reduced.shape)

🧪 Running Tests & Examples

To run the complete automated test suite:

python -m unittest discover -s tests -p "test_*.py"

To run example scripts:

python -m examples.01_tensor_autograd_basics
python -m examples.02_mlp_mnist_classification
python -m examples.03_classical_ml_regression_clustering
python -m examples.04_cnn_image_classifier

📜 License

MIT License. Open-source and free for educational, research, and production use!