The Simple Pytorch-Like Auto Differentiation Toolkit is an automatic differentiation package for calculating gradients of a function in forward and reverse mode.
Project description
Introduction
With the rapid development of deep learning, auto differentiation has become an indispensable part of multiple optimization algorithms like gradient descent. Numerical means such as Newton's Method and finite-difference method is useful in some situations, we desire to compute the analytical solutions by applying chain rules with our automatic differentiation SPLADTool (Simple Pytorch-Like Auto Differentiation Toolkit), which will be faster and more accurate than numerical methods.
Usage
-
Create a virtual environment: Conda
conda create --name spladtool_env python
Activate the environment:
conda activate spladtool_env
Deactivate the envrionment after use:
conda deactivate
-
Install spladtool
pip install spladtool
-
Try out an example from
test.py
on arithmetic functions:import spladtool.spladtool_forward as st x = st.tensor([[1., 2.], [3., 4.]]) # Define output functions y(x) and z(x) y = 2 * x + 1 z = - y / (x ** 3) w = st.cos((st.exp(z) + st.exp(-z)) / 2) # Print out the values calculated by our forward mode automatic differentiation SPLADTool print('x : ', x) print('y : ', y) print('y.grad : ', y.grad) print('z: ', z) print('z.grad: ', z.grad) print('w: ', w) print('w.grad: ', w.grad)
-
Try out an example training a linear classifier on a dataset
import spladtool.spladtool_reverse as str from spladtool.utils import SGD import numpy as np # We chose a simple classification model with decision boundary being 4x1 - 3x2 > 0 x = np.random.randn(200, 2) y = ((x[:, 0] - 3 * x[:, 1]) > 0).astype(float) # define a linear regression module np.random.seed(42) class MyModel(str.Module): def __init__(self): super().__init__() self.register_param(w1=str.tensor(np.random.randn())) self.register_param(w2=str.tensor(np.random.randn())) self.register_param(b=str.tensor(np.random.randn())) def forward(self, x): w1 = self.params['w1'].repeat(x.shape[0]) w2 = self.params['w2'].repeat(x.shape[0]) b = self.params['b'].repeat(x.shape[0]) y = w1 * str.tensor(x[:, 0]) + w2 * str.tensor(x[:, 1]) + b return y # define loss function and optimizer model = MyModel() criterion = str.BCELoss() opt = SGD(model.parameters(), lr=0.1, momentum=0.9) # training for epoch in range(100): outputs = model(x) targets = str.tensor(y) loss = criterion(targets, outputs) opt.zero_grad() loss.backward() opt.step() print(loss.data)
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for spladtool-0.0.5-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3465246b905b7ec5ab3dae8628fb1c142953739bf236bc3c1971529806e8cdf2 |
|
MD5 | d5341f5f123f5610ee51ac03e242735c |
|
BLAKE2b-256 | 4245ff3cd3bf3b7dc72fde4655ec04df2ea86a6f86be431aee22b3f3a01fcaf2 |