Modal Dojo

Modal Dojo is an open-source library to get started with production-grade LLM training on Modal.

To get started, you can go through the Quickstart section below, or paste the prompt below into your favorite agent:

Install the `modal-dojo` library. Then, install the skill bundle into the local project with `modal-dojo skills install`. Finally, tell the user that they can ask you to do something like: "Walk through tutorials/rl_basics.py and train Qwen3.5-4B to write 5-7-5 haikus from statworx/haiku."

Quickstart

Requirements:

• Python 3.12

Install the package directly:

uv pip install modal-dojo

Or pin it in pyproject.toml:

modal-dojo = { git = "https://github.com/modal-projects/modal-dojo.git", branch = "main" }

Set up Modal and the dashboard:

modal-dojo setup

And empower your agents with the Modal Dojo's skill bundle:

modal-dojo skills install

Then, it's as easy as:

import re

from modal_dojo import (
    HuggingFaceDataset,
    Qwen3_5_4B,
    Qwen3_5_4B_Recipe,
    TrainConfig,
)

model = Qwen3_5_4B()


async def gsm8k_rm(args, sample, **kwargs) -> float:
    text = model.parse_response(sample.response or "").content
    boxed = re.findall(r"\\boxed\{([^}]+)\}", text)
    pred = boxed[-1] if boxed else (re.findall(r"-?[\d,]+(?:\.\d+)?", text) or [""])[-1]
    try:
        return float(float(pred.replace(",", "")) == float(sample.label))
    except ValueError:
        return 0.0


config = TrainConfig(
    model=model,
    dataset=HuggingFaceDataset(
        hf_repo="skrishna/gsm8k_only_answer",
        hf_split="train[:120]",
        input_column="text",
        output_column="label",
        input_format="text",
    ),
    recipe=Qwen3_5_4B_Recipe(
        custom_rm_function=gsm8k_rm,
    ),
)

if __name__ == "__main__":
    run = config.launch()
    print(run.training_run_id)

For a step-by-step walkthrough, see the Getting started tutorial.

Supported models