A toolkit for building an LLM from scratch
Project description
llm-from-scratch-lab
Lab and python package for building a small GPT model. This code was adapted from the Build a Large Language Model (From Scratch) repository. This lab is intended to be a learning exercise, not a production-ready implementation. It is still very much a work in progress, and in some regards, a little incomplete. However, it is a good starting point for understanding how to build a small GPT model from scratch.
Requirements
- pytorch < 3
tiktokenBPE tokenizer < 1
Usage
There is a run_lab.py script that can be used to run the lab. It will send a piece of text to the lab and the lab will perform the preprocessing, tokenization, and model inference steps. The output will be printed to the console.
Package Installation
TBD
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file llm_scratch_lab-0.0.1.tar.gz.
File metadata
- Download URL: llm_scratch_lab-0.0.1.tar.gz
- Upload date:
- Size: 6.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.1.0 CPython/3.12.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
849382ef945805e0ed24a3fa5f1e58e70329c0f2c0237f5e1f1c736c639f7586
|
|
| MD5 |
2f3b4479147d74fc551a5b80424579d5
|
|
| BLAKE2b-256 |
217410860382fe3421aeb3b2fb0106788c4cf677de0d2dc677b00034f20df4d7
|
File details
Details for the file llm_scratch_lab-0.0.1-py3-none-any.whl.
File metadata
- Download URL: llm_scratch_lab-0.0.1-py3-none-any.whl
- Upload date:
- Size: 7.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.1.0 CPython/3.12.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
471cee66c6641745eed20083504d8abe961b76b29bf3d98b93e21f207af41c95
|
|
| MD5 |
a5d24e6e300b1496ce753270d63600f0
|
|
| BLAKE2b-256 |
ccc10fe6d46824ef3feb373f3215f075b61fbe258d317d4c652262d5f985e2e3
|
