Skip to main content

autodiff engine inspired by tinygrad

Project description

SkinnyGrad

python pypi license tests

SkinnyGrad is a tensor autodifferentiation library that I wrote as a side project for fun and learning. By default, a computational graph is built and evaluated lazily with NumPy. GPU acceleration is also available with the CuPy backend extension. At ~1300 lines, skinnygrad is written with simplicity and extensibility in mind. It nevertheless covers a good subset of the features of a torch.Tensor. Kudos to tinygrad which inspired the RISC-like design of mapping all operations to 19 low level ops that the runtime engine optimizes and executes.

Try it out!

pip install skinnygrad
import skinnygrad

a = skinnygrad.Tensor(((1, 2, 3)))
b = skinnygrad.Tensor(10)
x = skinnygrad.Tensor(((4,), (5,), (6,)))
y = a @ x + b
print(y)
# <skinnygrad.tensors.Tensor(
#   <skinnygrad.llops.Symbol(UNREALIZED <Op(ADD)>, shape=(1, 1))>,
#   self.requires_grad=False,
#   self.gradient=None,
# )>
print(y.realize())
# [[42]]

LeNet-5 as a convergence test

As an end-to-end test for the engine, I replicated the LeNet-5 paper -- a convolutional neural network (CNN) designed for handwritten digit recognition. Trained on MNIST, the model recovers 98% accuracy on the evaluation set after about 5 epochs. With a batch size of 64 it takes a few minutes per training epoch (60k images) using the CuPy GPU acceleration backend on a Nvidia A100 GPU. The code for the experiment can be found in the examples folder.

BONUS: The computational graph pass built up by the skinnygrad engine for LeNet-5

lenet-fwd

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

skinnygrad-0.1.3.tar.gz (17.3 kB view details)

Uploaded Source

Built Distribution

skinnygrad-0.1.3-py3-none-any.whl (19.3 kB view details)

Uploaded Python 3

File details

Details for the file skinnygrad-0.1.3.tar.gz.

File metadata

  • Download URL: skinnygrad-0.1.3.tar.gz
  • Upload date:
  • Size: 17.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.3 CPython/3.12.4 Darwin/23.5.0

File hashes

Hashes for skinnygrad-0.1.3.tar.gz
Algorithm Hash digest
SHA256 d2a2db4460e681ae72777233bc82e38d324ec2dbc308cd6008dc0eaf7fbfd81f
MD5 5e3872480b40bccbb93aa2aa939a6193
BLAKE2b-256 dfdeb5220f4a10bc6df1dc0f56690d2f0364d826d2a8bf4dc816edbf6565047c

See more details on using hashes here.

File details

Details for the file skinnygrad-0.1.3-py3-none-any.whl.

File metadata

  • Download URL: skinnygrad-0.1.3-py3-none-any.whl
  • Upload date:
  • Size: 19.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.3 CPython/3.12.4 Darwin/23.5.0

File hashes

Hashes for skinnygrad-0.1.3-py3-none-any.whl
Algorithm Hash digest
SHA256 1977d9a3f10975bee39911999d375e8964974342f79767cad844b35478d8a623
MD5 161f414e5f5e1fa7d90275f69f981012
BLAKE2b-256 d40e1bc7e06bb3c0453b0d46827e308619c42af70740835aa5ca4fa398c75be3

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page