Toy-GPT: train-400-context-3

PyPI version Latest Release Docs License: MIT CI Deploy-Docs Check Links Dependabot

Demonstrates, at very small scale, how a language model is trained.

This repository is part of a series of toy training repositories plus a companion client repository:

  • Training repositories produce pretrained artifacts (vocabulary, weights, metadata).
  • The client repository loads those artifacts and provides an interactive prompt.

Contents

  • a small, declared text corpus
  • a tokenizer and vocabulary builder
  • a simple next-token prediction model
  • a repeatable training loop
  • committed, inspectable artifacts for downstream use

Scope

This is:

  • an intentionally inspectable training pipeline
  • a next-token predictor trained on an explicit corpus

This is not:

  • a production system
  • a full Transformer implementation
  • a chat interface
  • a claim of semantic understanding

Outputs

This repository produces and commits pretrained artifacts under artifacts/.

Training logs and evidence are written under outputs/ (for example, outputs/train_log.csv).

Quick start

See SETUP.md for full setup and workflow instructions.

Run the full training script:

uv run python src/toy_gpt_train/d_train.py

Run individually:

  • a/b/c are demos (can be run alone if desired)
  • d_train produces artifacts
  • e_infer consumes artifacts
uv run python src/toy_gpt_train/a_tokenizer.py
uv run python src/toy_gpt_train/b_vocab.py
uv run python src/toy_gpt_train/c_model.py
uv run python src/toy_gpt_train/d_train.py
uv run python src/toy_gpt_train/e_infer.py

Provenance and Purpose

The primary corpus used for training is declared in SE_MANIFEST.toml.

This repository commits pretrained artifacts so the client can run without retraining.

Annotations

ANNOTATIONS.md - REQ/WHY/OBS annotations used

Citation

CITATION.cff

License

MIT

SE Manifest

SE_MANIFEST.toml - project intent, scope, and role

Project links

Data verified by PyPI on Jan 17, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for denisecase from gravatar.com denisecase

Credits

Author: Denise Case

GitHub Statistics

Data verified by PyPI on Jan 17, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

MIT License

Requires

Python >=3.14

Provides Extra

dev docs

Tags

toy gpt training context window language model

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

toy_gpt_train_400_context_3-0.9.2.tar.gz (85.3 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

toy_gpt_train_400_context_3-0.9.2-py3-none-any.whl (22.3 kB view details)

Uploaded Python 3

File details

Details for the file toy_gpt_train_400_context_3-0.9.2.tar.gz.

File metadata

File hashes

Hashes for toy_gpt_train_400_context_3-0.9.2.tar.gz
Algorithm Hash digest
SHA256 031e7eb11d3c6203fe069dd229ea687e4fc06a16a9623cdaafe5adf76d69723b
MD5 6fb8c02d340c894ae4fa9e804f164ad2
BLAKE2b-256 99c17e2186285c5863eddee0e948f3936f717aa138392c916bf069485b66edc1

See more details on using hashes here.

Provenance

The following attestation bundles were made for toy_gpt_train_400_context_3-0.9.2.tar.gz:

Publisher: release-pypi.yml on toy-gpt/train-400-context-3

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file toy_gpt_train_400_context_3-0.9.2-py3-none-any.whl.

File metadata

File hashes

Hashes for toy_gpt_train_400_context_3-0.9.2-py3-none-any.whl
Algorithm Hash digest
SHA256 4a99c60ef1485de1d0e34db8c8f3d7e724cca12639514c35d69d1461aadd7014
MD5 5bff97b6dd0ea23584b89a61d51b638a
BLAKE2b-256 ca3d92cadbdcb83a58b81915e6b1b846172d8f41f7c02edb63b9103b7e1478db

See more details on using hashes here.

Provenance

The following attestation bundles were made for toy_gpt_train_400_context_3-0.9.2-py3-none-any.whl:

Publisher: release-pypi.yml on toy-gpt/train-400-context-3

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.9.2 This release

2 files

0.9.1

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page