dqntrain

Headless, config-driven fine-tuning for LLMs on cloud GPUs.

A programmatic fine-tuning toolkit inspired by Unsloth Studio's workflow, built for headless environments like Kaggle, Google Colab, and remote servers. No GUI required — just YAML/JSON recipes, clean Python APIs, and automatic GPU-aware defaults.

Quick Start (Kaggle / Colab Notebook)

1. Install

%%capture
!pip install dqntrain

2. Define a Config

from dqntrain import auto_config_for_model

config = auto_config_for_model("unsloth/Qwen2.5-7B-Instruct-bnb-4bit")
config.training.num_train_epochs = 1
config.dataset.path = "yahma/alpaca-cleaned"
config.export.formats = ["lora", "gguf"]

3. Train

from dqntrain import (
    ModelLoader, DatasetBuilder, DQNTrainer,
    TrainingMonitor, ModelExporter
)

loader = ModelLoader(config)
model, tokenizer = loader.load()
model = loader.apply_lora()

dataset = DatasetBuilder(config).build()

monitor = TrainingMonitor()
trainer = DQNTrainer(config, monitor=monitor)
trainer.create_trainer(model, tokenizer, dataset)
trainer.train()
monitor.plot()

ModelExporter(config).export(model, tokenizer)

That's it. The entire pipeline in ~15 lines.

Features

Project Structure

dqntrain/
├── dqntrain/
│   ├── config.py          # YAML/JSON recipes + auto GPU presets
│   ├── models.py          # Model loader + LoRA applier
│   ├── data.py            # Dataset builder (JSON/CSV/HF/conversations)
│   ├── trainer.py         # SFTTrainer wrapper with cloud paths
│   ├── observability.py   # Live metrics, plots, JSONL logging
│   ├── export.py          # GGUF, merged, Ollama, Hub push
│   └── utils.py           # Environment detection + GPU info
├── configs/
│   ├── quick_lora.yaml    # T4-optimized 4-bit recipe
│   └── full_finetune.yaml # P100+ 16-bit recipe
└── notebooks/
    └── kaggle_template.ipynb  # Ready-to-run Kaggle notebook

Config Recipes

Save a YAML file and pass it to load_config():

model:
  name: "unsloth/Qwen2.5-7B-Instruct-bnb-4bit"
  max_seq_length: 2048
  load_in_4bit: true

lora:
  r: 16
  lora_alpha: 32
  target_modules: [q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj]

training:
  num_train_epochs: 1
  per_device_train_batch_size: 1
  gradient_accumulation_steps: 4
  learning_rate: 2.0e-4
  output_dir: "./outputs"

dataset:
  path: "yahma/alpaca-cleaned"
  instruction_column: "instruction"
  response_column: "output"

export:
  formats: [lora, gguf]
  quantization: "q4_k_m"

Supported Dataset Formats

DatasetBuilder auto-detects:

• HuggingFace datasets (dataset.path = "yahma/alpaca-cleaned")
• JSON / JSONL files with instruction/response or conversations arrays
• CSV files with text columns
• In-memory lists of dicts

GPU-Aware Defaults

auto_config_for_model() inspects your GPU and adjusts:

• load_in_4bit — forced on for T4 (15 GB), optional on P100/V100
• per_device_train_batch_size — smaller for big models
• max_seq_length — capped to fit VRAM
• gradient_accumulation_steps — scaled up to maintain effective batch size

Export Formats

ModelExporter can produce any combination of:

• lora — LoRA adapter (smallest, fastest to save)
• merged / safetensors — Full 16-bit merged model
• gguf — Quantized GGUF for llama.cpp / Ollama
• ollama — Modelfile ready for ollama create

CLI Usage

# Install deps
pip install dqntrain

# Run from a config file
dqntrain --config configs/quick_lora.yaml

# Override model and dataset
dqntrain --model unsloth/Phi-4 --dataset my_data.jsonl --epochs 2

Kaggle-Specific Tips

1. Enable Internet in notebook settings for pip install and HF downloads.
2. Persist outputs — everything in /kaggle/working survives the session.
3. Save Versions — click "Save Version" → "Run" to execute headlessly. Logs still write to /kaggle/working.
4. Dataset inputs — upload custom JSON/CSV via Kaggle Datasets, then reference them with config.dataset.path = "/kaggle/input/your-dataset/file.json".

License

MIT. This package depends on Unsloth (Apache 2.0 core), TRL, and Transformers.