Toy-GPT: train-200-bigram

Demonstrates, at very small scale, how a language model is trained.

This repository is part of a series of toy training repositories plus a companion client repository:

Training repositories produce pretrained artifacts (vocabulary, weights, metadata).
A web app loads the artifacts and provides an interactive prompt.

a small, declared text corpus
a tokenizer and vocabulary builder
a simple next-token prediction model
a repeatable training loop
committed, inspectable artifacts for downstream use

Scope

This is:

an intentionally inspectable training pipeline
a next-token predictor trained on an explicit corpus

This is not:

a production system
a full Transformer implementation
a chat interface
a claim of semantic understanding

Outputs

This repository produces and commits pretrained artifacts under artifacts/.

Training logs and evidence are written under outputs/ (for example, outputs/train_log.csv).

Quick start

See SETUP.md for full setup and workflow instructions.

Run the full training script:

uv run python src/toy_gpt_train/d_train.py

Run individually:

a/b/c are demos (can be run alone if desired)
d_train produces artifacts
e_infer consumes artifacts

uv run python src/toy_gpt_train/a_tokenizer.py
uv run python src/toy_gpt_train/b_vocab.py
uv run python src/toy_gpt_train/c_model.py
uv run python src/toy_gpt_train/d_train.py
uv run python src/toy_gpt_train/e_infer.py

Provenance and Purpose

The primary corpus used for training is declared in SE_MANIFEST.toml.

This repository commits pretrained artifacts so the client can run without retraining.

Annotations

ANNOTATIONS.md - REQ/WHY/OBS annotations used

Citation

CITATION.cff

License

MIT

SE Manifest

SE_MANIFEST.toml - project intent, scope, and role

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

toy_gpt_train_200_bigram-0.9.8.tar.gz (83.8 kB view details)

Uploaded Apr 3, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

toy_gpt_train_200_bigram-0.9.8-py3-none-any.whl (24.1 kB view details)

Uploaded Apr 3, 2026 Python 3

File details

Details for the file toy_gpt_train_200_bigram-0.9.8.tar.gz.

File metadata

Download URL: toy_gpt_train_200_bigram-0.9.8.tar.gz
Upload date: Apr 3, 2026
Size: 83.8 kB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for toy_gpt_train_200_bigram-0.9.8.tar.gz
Algorithm	Hash digest
SHA256	`5b034ea8c70f6605b29c37959d2147d66e028f876e5ad2fe052e5c3d293dfa58`
MD5	`f49be88ad7f7cd032d29b04b560255f6`
BLAKE2b-256	`fe1f96f5f469bbf1ca4cb61b1f554a325a5371bcaaf30950dfc03c6e81a26f04`

See more details on using hashes here.

Provenance

The following attestation bundles were made for toy_gpt_train_200_bigram-0.9.8.tar.gz:

Publisher: release-pypi-mkdocs-shared.yml on toy-gpt/train-200-bigram

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: toy_gpt_train_200_bigram-0.9.8.tar.gz
- Subject digest: 5b034ea8c70f6605b29c37959d2147d66e028f876e5ad2fe052e5c3d293dfa58
- Sigstore transparency entry: 1221998477
- Sigstore integration time: Apr 3, 2026
Source repository:
- Permalink: toy-gpt/train-200-bigram@d60f24144a2cde2a02a34b80c9d7165bb42ed813
- Branch / Tag: refs/tags/v0.9.8
- Owner: https://github.com/toy-gpt
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: release-pypi-mkdocs-shared.yml@d60f24144a2cde2a02a34b80c9d7165bb42ed813
- Trigger Event: push