Skip to main content

teapottrainer

teapottrainer is a config-driven library for pretraining and supervised fine-tuning of causal language models. You describe the model, the dataset mix, and the token budgets. TeapotTrainer runs pretraining, then supervised training, and can resume from the last checkpoint.

pip install teapottrainer

Train

from teapottrainer import (
    DatasetSpec,
    FINEWEB_EDU,
    GSM8K,
    ModelConfig,
    TeapotTrainer,
    TrainingConfig,
)

config = TrainingConfig(
    model=ModelConfig(
        model_type="llama",
        hidden_size=768,
        layers=12,
        heads=12,
        kv_heads=4,
        intermediate=2048,
        seq_len=2048,
    ),
    pretrain_datasets=[
        DatasetSpec(source=FINEWEB_EDU, target_tokens=1_000_000_000),
    ],
    sft_datasets=[
        DatasetSpec(source=GSM8K, target_tokens=4_000_000),
    ],
    tokenizer="HuggingFaceTB/SmolLM2-135M",
    batch_size=4,
    grad_accum=8,
    lr=5e-4,
)

trainer = TeapotTrainer(config)
trainer.train(fresh=True)

train(fresh=False) continues from checkpoints/resume in the run directory: weights, optimizer, and the sample cursor. Pass init_from to start from an existing Hugging Face model or a local checkpoint instead of randomly initialized weights.

Set sft_at_checkpoints=True to fine-tune a copy of the base model at each pretrain eval and save that copy beside the base weights. Pretraining keeps going from the base model.

Runs are written under ./runs/. Loss and task scores go to TensorBoard and to metrics.jsonl in the run directory.

Datasets

Preset sources live on the package: FINEWEB_EDU, DCLM_EDU, STACK_EDU, FINEMATH, GSM8K, SQUAD_V2, HOTPOT, SYNTHQA, TEAPOT_CHAT, the SmolTalk subsets, and the reading, tool-calling, and instruction-following evals. A DatasetSpec assigns a source a token budget and a mix weight.

To add a source, build a DatasetSourceConfig with a chat_template_fn that returns a prompt and an answer. The trainer supervises the answer tokens.

Develop

The train.py script in this repository is one example run. It is not part of the installed package.

pip install -e ".[dev]"
pytest

Metadata

Release files for teapottrainer 0.1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for teapottrainer 0.1.1
File Size Uploaded Provenance Actions
teapottrainer-0.1.1.tar.gz 44.8 kB With provenance Details for teapottrainer-0.1.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for teapottrainer 0.1.1
File Interpreter ABI Platform Actions
teapottrainer-0.1.1-py3-none-any.whl Python 3 none any Details for teapottrainer-0.1.1-py3-none-any.whl

Total release size: 92.0 kB

Release files / teapottrainer-0.1.1.tar.gz

Download URL teapottrainer-0.1.1.tar.gz
Size 44.8 kB
Tags Source
SHA-256 checksum
How to use checksums
3045c17781951cf43a21f5c6be68cdac04f3c51b7ff85bfcde11c17f3df074d2
BLAKE2b-256 checksum
How to use checksums
20889811597fb398b48bb72d41c828a5c5dac4d78e56bfcffd47a6f33ce6fdb4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 4, 2026.

Transparency log

Release files / teapottrainer-0.1.1-py3-none-any.whl

Download URL teapottrainer-0.1.1-py3-none-any.whl
Size 47.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
482ece758b6a14a1b03a2f924f36e97af0f2a63ba2d6a73f73875c55ed51507c
BLAKE2b-256 checksum
How to use checksums
fa87ea5eb0357b6c2012ed67acfc90ef2f70004e1a815a83ca80bf27b511d697
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 4, 2026.

Transparency log

Security for teapottrainer 0.1.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for teapottrainer-0.1.1.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 4, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page