arclm 0.1.1

A compact PyTorch language-model training and fine-tuning library.

ArcLM

ArcLM is a small PyTorch framework for developers who want a readable language-model training stack without giving up the pieces that matter in real projects: data preparation, tokenizer handling, checkpointing, fine-tuning, and inference.

It is designed for local experiments, compact domain models, teaching, and framework-style workflows where you want to understand and control the training loop.

Why Use ArcLM

• Train compact causal language models from plain text
• Fine-tune ArcLM checkpoints on new domain data
• Load ArcLM checkpoints, raw PyTorch state dicts, safetensors files, and Hugging Face sources through one loader API
• Save resumable checkpoints by epoch or every N training batches
• Keep tokenizer metadata inside checkpoints for safer reloads
• Use a high-level train_model() API or drop down to Config, build_model(), and Trainer
• Run inference with a simple load_model() and predict() flow

ArcLM is not trying to replace large production training systems. It is a practical framework for building, testing, and adapting small transformer language models with code that stays easy to inspect.

Installation

pip install arclm

ArcLM requires Python 3.9+ and PyTorch.

Quick Start

Train a model from text:

from arclm import train_model

result = train_model(
    mode="pretrain",
    data="data/data.txt",
    output="models/arclm.pth",
    tokenizer_type="word",
    max_vocab=2000,
    embed_dim=64,
    num_blocks=2,
    block_size=32,
    batch_size=8,
    num_epochs=3,
    checkpoint_batch_interval=100,
)

print(result.model_path)

Generate text from the saved checkpoint:

from arclm import load_model

model = load_model("models/arclm.pth")
print(model.predict("machine learning", max_new_tokens=50, top_p=0.9))

Fine-Tuning

from arclm import train_model

result = train_model(
    mode="finetune",
    checkpoint="models/arclm.pth",
    data="data/domain_text.txt",
    output="models/arclm_domain.pth",
    num_epochs=2,
    learning_rate=2e-5,
    freeze_backbone=True,
    checkpoint_batch_interval=50,
)

For native ArcLM checkpoints, tokenizer compatibility is checked before fine-tuning so token IDs are not silently mixed up.

Lower-Level Control

Use the framework pieces directly when you need custom training logic:

import torch
from arclm import (
    Config,
    build_model,
    build_trainer,
    create_checkpoint_callback,
    prepare_data,
)

config = Config(
    data_path="data/data.txt",
    model_path="models/arclm.pth",
    tokenizer_type="sentencepiece",
    max_vocab=8000,
    embed_dim=128,
    num_blocks=4,
    block_size=128,
    batch_size=32,
    num_epochs=5,
    learning_rate=3e-4,
    validation_split=0.1,
    checkpoint_batch_interval=100,
    device="cuda" if torch.cuda.is_available() else "cpu",
)

data = prepare_data(config)
config.vocab_size = data.vocab_size

model = build_model(config, data.vocab_size)
trainer = build_trainer(model, config)

trainer.train(
    data.train_loader,
    config.num_epochs,
    val_loader=data.val_loader,
    early_stopping_patience=2,
    min_delta=1e-4,
    checkpoint_callback=create_checkpoint_callback(config, data.tokenizer, data.vocab_size),
)

Checkpointing

ArcLM checkpoints include model weights, optimizer state, config, training history, tokenizer mappings, and tokenizer metadata.

Useful settings:

• checkpoint_batch_interval=100 saves every 100 training batches
• checkpoint_interval=1 saves every completed epoch
• early_stopping_patience=2 stops training when validation loss stops improving

Checkpoint writes are atomic: ArcLM writes to a temporary file first and replaces the checkpoint only after the save succeeds.

External Model Loading

ArcLM can normalize several checkpoint formats before training:

• ArcLM checkpoints saved by Trainer.save
• raw PyTorch state dict files: .pth, .pt, .bin, .ckpt
• .safetensors files when safetensors is installed
• Hugging Face model folders or model IDs when transformers is installed

from arclm import adapt_for_training, load_external_model

loaded = load_external_model("external/model.pth")
bundle = adapt_for_training(loaded)

print(loaded.source_type)
print(bundle.config)

Project Layout

arclm/      Framework source code
docs/       Usage guides and references
examples/   Example scripts
tests/      Test suite

Documentation

See docs/USAGE.md for a longer guide covering training, loading checkpoints, fine-tuning, and external sources.

License

MIT