amazingvmsloth

Blazing-fast LLM fine-tuning with minimal VRAM.

  \   / |    amazingvmsloth - Fast LLM Fine-Tuning
   O^O / \_/ \   Minimal VRAM. Maximum Speed.
  \        /
   "-____-"

Train 14B models on a 4GB GPU. Multi-GPU, 4-bit quantization, LoRA, CPU offloading, gradient checkpointing, and sequence packing — all built for speed on consumer hardware.

Install

pip install amazingvmsloth

Or from source:

git clone https://github.com/CollabVMgamez/amazingvmsloth.git
cd amazingvmsloth
pip install -e .

Requirements: Python 3.9+, PyTorch 2.1+, CUDA 11.8+ (optional, CPU training supported)

Quick Start

1. Wizard — let it pick settings for your hardware

amazingvmsloth wizard --model Qwen/Qwen2.5-0.5B

Analyzes your GPU/CPU and prints a ready-to-run command.

2. Train

amazingvmsloth train \
  --model Qwen/Qwen2.5-0.5B \
  --dataset tatsu-lab/alpaca \
  --epochs 3 \
  --batch-size 2 \
  --grad-accum 4 \
  --lora-r 16 \
  --output-dir ./output

Supports chat-format datasets too:

amazingvmsloth train \
  --model Qwen/Qwen2.5-0.5B \
  --dataset angrygiraffe/claude-opus-4.6-4.7-reasoning-8.7k \
  --dataset-format chat \
  --max-samples 1000 \
  --output-dir ./thinking_lora

3. Convert LoRA to merged model

amazingvmsloth merge \
  --model Qwen/Qwen2.5-0.5B \
  --lora ./output \
  --output ./merged_model

4. Run inference

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("./merged_model", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B")

messages = [{"role": "user", "content": "What is 2+2?"}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0]))

CLI Commands

Command	Description
`wizard`	Interactive config generator based on your hardware
`train`	Fine-tune a model with LoRA
`merge`	Merge LoRA weights into base model
`convert`	Merge + convert to GGUF (requires llama.cpp)
`info`	Show model info and VRAM estimates
`bench`	Benchmark vs unsloth

Hardware Tiers

GPU VRAM	Strategy
4-6 GB	4-bit quant, batch=1, grad accum=8, seq=512, tiny LoRA
6-12 GB	4-bit quant, batch=1-2, grad accum=4, seq=1024
12-24 GB	4-bit or full precision, batch=2-4, torch.compile
24+ GB	Full precision, no grad checkpointing, large batch
CPU only	fp32/bf16, torch.compile, physical-core threading

Key Features

Model Architectures: Qwen 3.5 (e.g. Qwen/Qwen3.5-0.8B), Gemma 3 (1B/4B), Gemma 2, LLaMA 3.2/3.1, Mistral, DeepSeek v2/v3/R1, and multimodal/TTS models
RTX 3050 & Low-VRAM Tuned: Ampere TensorFloat-32 (TF32) acceleration, fp16 Tensor Core speedup, non-blocking CUDA transfers, and automated 4-bit memory management
rsLoRA scaling for stable training at any rank
4-bit/8-bit quantization via bitsandbytes
XFormers/SDPA attention patching (Flash Attention on Linux)
Sequence packing for 2-3x throughput
Gradient checkpointing with selective layer skipping
Multi-GPU: DDP, FSDP, DeepSpeed, pipeline parallelism
Layer offloading via accelerate.dispatch_model
CPU training with IPEX, pre-packing, torch.compile
PagedAdamW8bit optimizer for low-VRAM training
Checkpoint resume with full RNG/optimizer state
Tqdm progress bar with live loss + VRAM display

Example: 500 Steps on Dolly

amazingvmsloth train \
  --model Qwen/Qwen2.5-0.5B \
  --dataset databricks/databricks-dolly-15k \
  --dataset-format alpaca \
  --epochs 1 --batch-size 2 --grad-accum 2 \
  --max-samples 1000 --max-seq-length 512 \
  --lora-r 16 --output-dir ./dolly_lora --packing

This runs ~500 steps in ~10 minutes on a 4GB RTX 3050.

Project Structure

amazingvmsloth/
├── lora.py              # LoRA with rsLoRA, device-aware init
├── quantization.py      # 4-bit/8-bit quant, kbit training prep
├── attention.py         # SDPA/XFormers patching
├── trainer.py           # AmazingTrainer with tqdm, packing, offloading
├── cpu_trainer.py       # CpuTrainer for CPU-only training
├── packing.py           # Sequence packing collators
├── gradient.py          # GradientAccumulator
├── optimizer.py         # PagedAdamW8bit, CpuOffloadedAdamW
├── offload.py           # Layer offloading via accelerate
├── cli.py               # CLI entrypoint
├── wizard.py            # Hardware-aware config generator
├── bench.py             # Benchmark vs unsloth
└── utils/
    ├── banner.py        # Startup banner with GPU info
    ├── memory.py        # VRAM estimation
    ├── patching.py      # LoRA save/load helpers
    └── save_load.py     # Model save/merge

Benchmarks

On RTX 3050 4GB Laptop GPU:

Library	Time (1 epoch, 500 samples)	Peak VRAM
amazingvmsloth	5.3s	1.07 GB
unsloth	10.1s	0.96 GB

1.91x faster on small runs with pre-quantized models.

License

MIT

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

amazingvmsloth-0.7.4.tar.gz (104.9 kB view details)

Uploaded Sep 13, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

amazingvmsloth-0.7.4-py3-none-any.whl (106.0 kB view details)

Uploaded Sep 13, 2026 Python 3

File details

Details for the file amazingvmsloth-0.7.4.tar.gz.

File metadata

Download URL: amazingvmsloth-0.7.4.tar.gz
Upload date: Sep 13, 2026
Size: 104.9 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: twine/6.2.0 CPython/3.12.10

File hashes

Hashes for amazingvmsloth-0.7.4.tar.gz
Algorithm	Hash digest
SHA256	`88ea0a81819b7dede02c640ca918033efd1b44e6b716b393b9b2672eedf2bdd7`
MD5	`5f2db91ab203c610b779753cc2a94564`
BLAKE2b-256	`2f2f391b3578c605ebfcfcf5458e79fafe50263e876e842467e579a485441441`

See more details on using hashes here.

File details

Details for the file amazingvmsloth-0.7.4-py3-none-any.whl.

File metadata

Download URL: amazingvmsloth-0.7.4-py3-none-any.whl
Upload date: Sep 13, 2026
Size: 106.0 kB
Tags: Python 3
Uploaded using Trusted Publishing? No
Uploaded via: twine/6.2.0 CPython/3.12.10

File hashes

Hashes for amazingvmsloth-0.7.4-py3-none-any.whl
Algorithm	Hash digest
SHA256	`62860482abd086644da8cd3e5d7b93d39c3465bd7cd43f12b77bd7f3fff0f5a2`
MD5	`f4946c3024e2cb6a9d07b4f80f6419fb`
BLAKE2b-256	`2c2333e3f5af2b327cefe27dc296ccc9b1ed7814aebe5424f0af75873a630b47`