Paper - Pytorch
Project description
TeraGPT
Train a production grade GPT in less than 400 lines of code. Better than Karpathy's verison and GIGAGPT
Install
pip3 install
Usage
import torch
from teragpt.main import TeraGPT
model = TeraGPT(
dim=4096,
depth=6,
heads=8,
num_tokens=20000,
)
x = torch.randint(0, 20000, (1, 4096))
out = model(x)
print(out.shape)
License
MIT
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
teragpt-0.0.1.tar.gz
(3.2 kB
view details)
Built Distribution
File details
Details for the file teragpt-0.0.1.tar.gz.
File metadata
- Download URL: teragpt-0.0.1.tar.gz
- Upload date:
- Size: 3.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 08e91dd453d66afbefdf57e0e42ef76bd6d0c22d1b0eec7f55601ab6d41fa021 |
|
| MD5 | b6c0f88ce26412a142678856f287c194 |
|
| BLAKE2b-256 | 53c73972e050a28f12a65427ea4fad968bab67076534e0046ad2b1e96de25e1a |
File details
Details for the file teragpt-0.0.1-py3-none-any.whl.
File metadata
- Download URL: teragpt-0.0.1-py3-none-any.whl
- Upload date:
- Size: 3.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | b8b88e003ce58d0aa7ff2397fc34bf0d763188687fa4ae65f4b39bdce70b305a |
|
| MD5 | e18507fe539ad04e498048a956fa6123 |
|
| BLAKE2b-256 | 99fcdb9c5600eba29bf441445b0228614135cc3338111ba9ce17ff862306ca9e |
