ttorch 0.1.4

A lightweight C++ tensor and autograd library with Python bindings

TTorch

TTorch is a lightweight tensor and automatic differentiation library written in C++ with Python bindings.

The project explores the internal architecture of modern deep learning frameworks by implementing core components from scratch: tensor operations, a dynamic computation graph, backpropagation, and Python bindings via pybind11.

---

Features

• C++17 core — tensors, math ops, shape manipulation
• Dynamic autograd engine — computation graph built during the forward pass, traversed in reverse on backward()
• Activation functions — relu, sigmoid with gradient tracking
• Python bindings — full API exposed via pybind11
• scikit-build-core build backend — pip install works out of the box
• CI/CD — wheels built via cibuildwheel and published to PyPI on push to main

---

Project Structure

TTorch/
├── pyproject.toml        # Python build config (scikit-build-core + pybind11)
├── CMakeLists.txt        # C++ build config
├── README.md
├── AUTOGRAD.md           # Deep-dive into the autograd design
├── LICENSE
│
├── include/
│   ├── tensor.h          # Tensor class declaration + autograd fields
│   └── autograd.h        # GradFn base class + all GradFn subclasses
│
├── src/
│   ├── tensor.cpp        # Tensor ops + forward graph building
│   ├── autograd.cpp      # Backward engine + gradient rules
│   └── bindings.cpp      # pybind11 Python bindings
│
├── ttorch/
│   └── __init__.py       # Python package entry point
│
└── tests/
    └── test.cpp          # C++ unit tests

---

Installation

From PyPI (recommended)

pip install ttorch

From Source

git clone https://github.com/DuongAnh1212/TTorch.git
cd TTorch
pip install .

Or build a wheel manually:

pip install build
python -m build
pip install dist/*.whl

Requirements: Python 3.11+, a C++17-capable compiler, CMake.

---

Quick Start

Python

import ttorch

# Create tensors
a = ttorch.Tensor.form([2, 3], [1, 2, 3, 4, 5, 6])
b = ttorch.Tensor.ones([2, 3])

# Math ops
c = a.add(b)          # element-wise addition
d = a.multiply(b)     # element-wise multiplication
e = a.dot(a.T())      # matrix multiply → shape (2, 2)
s = a.sum(0)          # sum along axis 0 → shape (3,)

# Autograd
a.requires_grad = True
y = ttorch.relu(a)
y.backward()
print(a.grad)         # gradient of a
a.zero_grad()

C++

#include "tensor.h"
#include "autograd.h"

Tensor a = Tensor::form({2, 3}, {1, 2, 3, 4, 5, 6});
a.requires_grad = true;

Tensor y = relu(a);
y.backward();

a.grad->print();  // gradient of a
a.zero_grad();

---

Tensor API

Factory Methods

Method	Description
Tensor::zeros(dims)	Tensor filled with 0.0
Tensor::ones(dims)	Tensor filled with 1.0
Tensor::custom(dims, val)	Tensor filled with a scalar
Tensor::form(dims, data)	Tensor from a vector<double>

Shape & Access

Method	Description
dims()	Returns shape as vector<int>
ndim()	Number of dimensions
size(dim)	Size of a given dimension
at(index)	Element access by multi-index
reshape(newshape)	Returns reshaped tensor
view(newshape)	Alias for reshape
flatten()	Returns 1D tensor

Math Operations

Method	Description
add(Tensor)	Element-wise addition
add_int(double)	Scalar broadcast addition
scale_int(double)	Scalar broadcast multiply
multiply(Tensor)	Element-wise multiplication
dot(Tensor)	Matrix multiplication (1D and 2D)
transpose() / T()	Transpose a 2D tensor
sum(axis)	Sum along axis
mean(axis)	Mean along axis

Autograd

Method	Description
backward()	Backpropagate gradients through the computation graph
zero_grad()	Reset accumulated gradient to zero

---

Autograd API

TTorch implements a dynamic computation graph — the graph is built automatically during the forward pass as a chain of GradFn pointers.

Gradient Functions

GradFn	Forward op	Backward rule
AddBackward	add(a, b)	grad flows to a and b unchanged
AddScalarBackward	add_int(a, s)	grad flows to a unchanged
MulBackward	multiply(a, b)	grad_a = grad * b, grad_b = grad * a
ScaleBackward	scale_int(a, s)	grad_a = grad * s
DotBackward	dot(a, b)	grad_a = grad @ b.T, grad_b = a.T @ grad
TransposeBackward	transpose(a)	grad_a = grad.T
FlattenBackward	flatten(a)	grad_a = grad.reshape(original_shape)
SumBackward	sum(a, axis)	broadcast grad back to original shape
MeanBackward	mean(a, axis)	broadcast grad / n back to original shape
ReLUBackward	relu(a)	grad_a = grad * (a > 0)
SigmoidBackward	sigmoid(a)	grad_a = grad * sigmoid(a) * (1 - sigmoid(a))

Activation Functions

Function	Description
relu(Tensor& x)	Element-wise ReLU with gradient tracking
sigmoid(Tensor& x)	Element-wise sigmoid with gradient tracking

---

Architecture

Python API (ttorch)
      ↓
pybind11 bindings  (src/bindings.cpp)
      ↓
C++ Core
  ├── Tensor + math ops   (include/tensor.h, src/tensor.cpp)
  └── Autograd engine     (include/autograd.h, src/autograd.cpp)

The backward engine uses topological sort to visit nodes in reverse dependency order, accumulating gradients into leaf tensors. See AUTOGRAD.md for a detailed design walkthrough with worked examples.

---

Build from C++ Only

mkdir build && cd build
cmake ..
make

---

Roadmap

• Core tensor data structure
• Tensor math operations (add, multiply, dot, transpose, sum, mean)
• Autograd computation graph (GradFn architecture)
• Backpropagation engine (backward, zero_grad)
• Activation functions (relu, sigmoid)
• Python bindings (pybind11 via scikit-build-core)
• PyPI publishing via CI/CD (cibuildwheel + GitHub Actions)
• Neural network modules (Linear, loss functions)
• Optimizers (SGD, Adam)
• GPU backend

---

Dependencies

Dependency	Purpose
C++17	Core library
CMake	Build system
Python 3.11+	Python interface
pybind11	Python bindings
scikit-build-core	Python build backend

---

License

MIT — see LICENSE.

---

Inspiration

• PyTorch
• tinygrad
• micrograd