Paper - Pytorch
Project description
TeraGPT
Train a production grade GPT in less than 400 lines of code. Better than Karpathy's verison and GIGAGPT
Install
pip3 install teragpt
Usage
import torch
from teragpt.main import TeraGPT
model = TeraGPT(
dim=4096,
depth=6,
heads=8,
num_tokens=20000,
)
x = torch.randint(0, 20000, (1, 4096))
out = model(x)
print(out.shape)
Train
from teragpt import train
train()
License
MIT
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
teragpt-0.0.2.tar.gz
(11.0 kB
view details)
Built Distribution
teragpt-0.0.2-py3-none-any.whl
(11.0 kB
view details)
File details
Details for the file teragpt-0.0.2.tar.gz.
File metadata
- Download URL: teragpt-0.0.2.tar.gz
- Upload date:
- Size: 11.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 23f94541f50e3d37579df1f6ab3884826413d80aa82588d6fb2703d3cd8d1cc3 |
|
| MD5 | 335dda00f472c6ba8c9ab8cb655b7f96 |
|
| BLAKE2b-256 | 4541d635027f7f2cec638e3753ee390fd1c69fecb5227fc1ad4820c23d38b9a9 |
File details
Details for the file teragpt-0.0.2-py3-none-any.whl.
File metadata
- Download URL: teragpt-0.0.2-py3-none-any.whl
- Upload date:
- Size: 11.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 783abfac663f08b14fa0ba066fe76b5b8674a535b9c40f0218eaab3f815040e3 |
|
| MD5 | d8a080037d6d2c09d738c2b0af353229 |
|
| BLAKE2b-256 | d3276b015b09099c1a02c5e0b0742626c935e642096ab08c138da8f2a54eb669 |
