Paper - Pytorch
Project description
TeraGPT
Train a production grade GPT in less than 400 lines of code. Better than Karpathy's verison and GIGAGPT
Install
pip3 install
Usage
import torch
from teragpt.main import TeraGPT
model = TeraGPT(
dim=4096,
depth=6,
heads=8,
num_tokens=20000,
)
x = torch.randint(0, 20000, (1, 4096))
out = model(x)
print(out.shape)
License
MIT
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
teragpt-0.0.1.tar.gz
(3.2 kB
view hashes)