Skip to main content

teapottrainer

teapottrainer is a config-driven library for pretraining and supervised fine-tuning of causal language models. You describe the model, the dataset mix, and the token budgets. TeapotTrainer runs pretraining, then supervised training, and can resume from the last checkpoint.

pip install teapottrainer

Train

from teapottrainer import (
    DatasetSpec,
    FINEWEB_EDU,
    GSM8K,
    ModelConfig,
    TeapotTrainer,
    TrainingConfig,
)

config = TrainingConfig(
    model=ModelConfig(
        model_type="llama",
        hidden_size=768,
        layers=12,
        heads=12,
        kv_heads=4,
        intermediate=2048,
        seq_len=2048,
    ),
    pretrain_datasets=[
        DatasetSpec(source=FINEWEB_EDU, target_tokens=1_000_000_000),
    ],
    sft_datasets=[
        DatasetSpec(source=GSM8K, target_tokens=4_000_000),
    ],
    tokenizer="HuggingFaceTB/SmolLM2-135M",
    batch_size=4,
    grad_accum=8,
    lr=5e-4,
)

trainer = TeapotTrainer(config)
trainer.train(fresh=True)

train(fresh=False) continues from checkpoints/resume in the run directory: weights, optimizer, and the sample cursor. Pass init_from to start from an existing Hugging Face model or a local checkpoint instead of randomly initialized weights.

Set sft_at_checkpoints=True to fine-tune a copy of the base model at each pretrain eval and save that copy beside the base weights. Pretraining keeps going from the base model.

Runs are written under ./runs/. Loss and task scores go to TensorBoard and to metrics.jsonl in the run directory.

Datasets

Preset sources live on the package: FINEWEB_EDU, DCLM_EDU, STACK_EDU, FINEMATH, GSM8K, SQUAD_V2, HOTPOT, SYNTHQA, TEAPOT_CHAT, the SmolTalk subsets, and the reading, tool-calling, and instruction-following evals. A DatasetSpec assigns a source a token budget and a mix weight.

To add a source, build a DatasetSourceConfig with a chat_template_fn that returns a prompt and an answer. The trainer supervises the answer tokens.

Develop

The train.py script in this repository is one example run. It is not part of the installed package.

pip install -e ".[dev]"
pytest

Metadata

Release files for teapottrainer 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for teapottrainer 0.1.0
File Size Uploaded Provenance Actions
teapottrainer-0.1.0.tar.gz 44.8 kB With provenance Details for teapottrainer-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for teapottrainer 0.1.0
File Interpreter ABI Platform Actions
teapottrainer-0.1.0-py3-none-any.whl Python 3 none any Details for teapottrainer-0.1.0-py3-none-any.whl

Total release size: 92.0 kB

Release files / teapottrainer-0.1.0.tar.gz

Download URL teapottrainer-0.1.0.tar.gz
Size 44.8 kB
Tags Source
SHA-256 checksum
How to use checksums
a9e3849d4eeab6d34b1fb4b746981229d241184f15af2dbc7d750087d7d995b2
BLAKE2b-256 checksum
How to use checksums
c1bb84fabf608ba1c11fed358c28794c2715380766716fc5bf1d978b5011e23b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 4, 2026.

Transparency log

Release files / teapottrainer-0.1.0-py3-none-any.whl

Download URL teapottrainer-0.1.0-py3-none-any.whl
Size 47.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
6012048d790108147e7d15ff666a446f89d0f5424f7ca94364dea7cbded2010f
BLAKE2b-256 checksum
How to use checksums
e73350f3c54b390408b7bcc3e1eb73a60745a2847438c629025dacb97369c836
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 4, 2026.

Transparency log

Security for teapottrainer 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for teapottrainer-0.1.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 4, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page