peft-doctor 0.1.0

A practical pre-flight doctor for PEFT, LoRA, and QLoRA fine-tuning runs.

PEFT Doctor

PEFT Doctor is a pre-flight checker for LoRA and QLoRA fine-tuning. It catches the boring problems before they burn a training run: CUDA memory pressure, risky learning rates, missing tokenizer padding, weak LoRA target modules, broken prompt formats, NaN losses, and adapter save/load mistakes.

The package works in two ways:

• Use peft-doctor from the terminal before training.
• Use diagnose_peft(...) inside your training script with real model, tokenizer, peft_config, training_args, and dataset objects.

Install

Minimal install:

python -m pip install peft-doctor

Install with the normal fine-tuning stack:

python -m pip install "peft-doctor[ml]"

In Google Colab:

%pip install -U "peft-doctor[ml]"
!peft-doctor env

Use a GPU runtime in Colab before loading a model: Runtime -> Change runtime type -> T4, L4, A100, or another GPU.

Development install from this repository:

git clone https://github.com/awais-akhtar/peft-doctor.git
cd peft-doctor
python -m pip install -e ".[dev,ml]"

Quick Start

Run a pre-flight check from the terminal:

peft-doctor check \
  --model meta-llama/Llama-3-8B \
  --dataset data.jsonl \
  --batch-size 4 \
  --sequence-length 4096 \
  --learning-rate 2e-4

Use it in Python:

from peft_doctor import diagnose_peft

report = diagnose_peft(
    model=model,
    tokenizer=tokenizer,
    peft_config=peft_config,
    training_args=training_args,
    train_dataset=train_dataset,
    sequence_length=2048,
)

print(report.to_markdown())

Generate a safe starter config:

from peft_doctor import create_safe_lora_config, create_safe_bnb_config

peft_config = create_safe_lora_config(model)
bnb_config = create_safe_bnb_config()

What It Checks

Area	Common problem	Typical fix
GPU memory	CUDA out of memory	Use QLoRA, batch size 1, gradient checkpointing, shorter sequence length
Target modules	LoRA attached to the wrong layers	Use model-aware targets like q_proj, v_proj, c_attn, or query_key_value
Prompt format	Dataset does not teach the response shape	Use instruction/response text or a proper chat template
Learning rate	Loss spikes or NaN	Try 1e-4, 5e-5, bf16, cleaner samples, and label checks
Tokenizer	Padding crash during batching	Set tokenizer.pad_token = tokenizer.eos_token when appropriate
Evaluation	Eval OOM after training works	Disable eval or use a tiny eval batch
Adapter flow	Adapter not found after training	Use model.save_pretrained() and PeftModel.from_pretrained()

Commands

peft-doctor check

Runs the main pre-flight check.

peft-doctor check --model meta-llama/Llama-3-8B --dataset data.jsonl

Useful options:

peft-doctor check \
  --model Qwen/Qwen2.5-7B \
  --dataset train.jsonl \
  --batch-size 2 \
  --grad-accum 8 \
  --sequence-length 2048 \
  --learning-rate 2e-4 \
  --load-in-4bit \
  --bf16 \
  --gradient-checkpointing

Machine-readable output:

peft-doctor check --model mistralai/Mistral-7B-v0.1 --dataset train.jsonl --output json

Markdown output for issues or pull requests:

peft-doctor check --model gpt2 --dataset train.jsonl --output markdown

peft-doctor targets

Recommends LoRA target_modules.

peft-doctor targets --model meta-llama/Llama-3-8B
peft-doctor targets --family gpt2

Print as JSON:

peft-doctor targets --family qwen --output json

peft-doctor safe-config

Prints a safe LoRA or QLoRA starter config.

peft-doctor safe-config --model meta-llama/Llama-3-8B

Only LoRA:

peft-doctor safe-config --family gpt2 --no-qlora

JSON:

peft-doctor safe-config --family llama --output json

peft-doctor inspect-dataset

Checks a local .json, .jsonl, .csv, or .txt dataset sample.

peft-doctor inspect-dataset data.jsonl

The command looks for common training shapes:

• messages chat rows with role and content
• instruction and response style columns
• single text rows containing instruction/response markers
• pre-tokenized input_ids and labels

peft-doctor scan-log

Scans a training log for NaN, infinity, CUDA OOM, overflow, and unstable loss jumps.

peft-doctor scan-log trainer_log.jsonl
peft-doctor scan-log run.log --output markdown

peft-doctor adapter-check

Checks a LoRA adapter merge plan without loading the full model.

peft-doctor adapter-check \
  --base-model meta-llama/Llama-2-7b-hf \
  --adapter awaisakhtar/llama-2-7b-summarization-finetuned-on-xsum-lora \
  --output-dir merged-llama

peft-doctor merge-adapter

Merges a PEFT LoRA adapter into the base model and saves a normal Transformers model.

peft-doctor merge-adapter \
  --base-model meta-llama/Llama-2-7b-hf \
  --adapter awaisakhtar/llama-2-7b-summarization-finetuned-on-xsum-lora \
  --output-dir Llama-2-7b-summarization-finetuned-on-xsum \
  --dtype fp16

Push the merged model and tokenizer to the Hugging Face Hub:

huggingface-cli login

peft-doctor merge-adapter \
  --base-model meta-llama/Llama-2-7b-hf \
  --adapter your-user/your-lora-adapter \
  --output-dir merged-model \
  --push-to-hub \
  --hub-model-id your-user/merged-model \
  --dtype fp16

For Colab or private/gated models, store your own Hugging Face token as a secret named HF_TOKEN and read it from the notebook environment. Do not paste access tokens into notebooks, scripts, shell history, or GitHub issues.

For final exports, do not merge from a 4-bit or 8-bit loaded model unless you know your PEFT/Transformers versions support it. The safest export path is fp16, bf16, or fp32, then save_pretrained(..., safe_serialization=True).

peft-doctor scan-notebook

Scans a notebook for common PEFT and Colab mistakes, including exposed Hugging Face tokens.

peft-doctor scan-notebook model_merge.ipynb

peft-doctor env

Checks the local Python, CUDA, and fine-tuning package stack.

peft-doctor env
peft-doctor env --output json

This is especially useful in Colab because many setup problems come from the notebook runtime, not the training script.

peft-doctor colab

Prints a notebook-friendly setup cell.

peft-doctor colab

peft-doctor version

Prints the installed version.

peft-doctor version

Python API

Diagnose a training setup

from peft_doctor import diagnose_peft

report = diagnose_peft(
    model=model,
    tokenizer=tokenizer,
    peft_config=peft_config,
    training_args={
        "per_device_train_batch_size": 1,
        "gradient_accumulation_steps": 8,
        "learning_rate": 2e-4,
        "num_train_epochs": 3,
        "bf16": True,
        "gradient_checkpointing": True,
    },
    train_dataset=train_dataset,
)

if report.has_errors:
    raise RuntimeError(report.to_markdown())

Generate target modules

from peft_doctor import recommend_target_modules

targets = recommend_target_modules(model_name="meta-llama/Llama-3-8B")

Generate safe LoRA and QLoRA configs

from peft_doctor import create_safe_bnb_config, create_safe_lora_config

peft_config = create_safe_lora_config(model, r=16, lora_alpha=32)
bnb_config = create_safe_bnb_config()

When peft, transformers, and torch are installed, these helpers return real LoraConfig and BitsAndBytesConfig objects. Without those packages, they return plain dictionaries so you can still inspect the recommendation.

Guard training logs

from peft_doctor import NanLossGuard

guard = NanLossGuard()

for log in trainer_state_log_history:
    issues = guard.update(log)
    for issue in issues:
        print(issue.title, issue.fix)

Merge a LoRA adapter

from peft_doctor import merge_lora_adapter

result = merge_lora_adapter(
    base_model="meta-llama/Llama-2-7b-hf",
    adapter="your-user/your-lora-adapter",
    output_dir="merged-model",
    torch_dtype="fp16",
)

print(result.to_dict())

Colab Notebook Pattern

%pip install -U "peft-doctor[ml]"

from peft_doctor import diagnose_peft, create_safe_lora_config, create_safe_bnb_config

peft_config = create_safe_lora_config(model_name="meta-llama/Llama-3-8B")
bnb_config = create_safe_bnb_config()

report = diagnose_peft(
    model_name="meta-llama/Llama-3-8B",
    peft_config=peft_config,
    training_args={
        "per_device_train_batch_size": 1,
        "gradient_accumulation_steps": 8,
        "learning_rate": 2e-4,
        "bf16": True,
        "gradient_checkpointing": True,
        "load_in_4bit": True,
    },
    train_dataset=train_dataset,
    tokenizer=tokenizer,
)

print(report.to_markdown())

Dependency Note

PEFT Doctor uses open-source Python packages from the normal PyData and Hugging Face fine-tuning stack: torch, transformers, peft, datasets, accelerate, rich, typer, and related optional packages. Model weights and datasets can have their own licenses, so always check the license of the model and data you fine-tune.

Common Safe Config

from peft import LoraConfig

peft_config = LoraConfig(
    r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
    target_modules=[
        "q_proj", "k_proj", "v_proj", "o_proj",
        "gate_proj", "up_proj", "down_proj",
    ],
)

from transformers import BitsAndBytesConfig
import torch

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

Publishing

The repository includes GitHub Actions for CI and PyPI publishing.

1. Push this repository to awais-akhtar/peft-doctor.
2. On PyPI, create a trusted publisher for package peft-doctor:
   • owner: awais-akhtar
   • repository: peft-doctor
   • workflow: publish.yml
   • environment: pypi
3. On TestPyPI, create the same trusted publisher with environment testpypi.
4. Push a version tag to publish to PyPI:

git tag v0.1.0
git push origin v0.1.0

Manual TestPyPI publishing is available from the Publish Python Package workflow in GitHub Actions.

Project Status

This is alpha software. The checks are deliberately conservative: the package should warn early, explain the reason, and give a fix that a developer can actually try.

License

MIT