amazingvmsloth
Blazing-fast LLM fine-tuning with minimal VRAM.
\ / | amazingvmsloth - Fast LLM Fine-Tuning
O^O / \_/ \ Minimal VRAM. Maximum Speed.
\ /
"-____-"
Train 14B models on a 4GB GPU. Multi-GPU, 4-bit quantization, LoRA, CPU offloading, gradient checkpointing, and sequence packing — all built for speed on consumer hardware.
Install
pip install amazingvmsloth
Or from source:
git clone https://github.com/CollabVMgamez/amazingvmsloth.git
cd amazingvmsloth
pip install -e .
Requirements: Python 3.9+, PyTorch 2.1+, CUDA 11.8+ (optional, CPU training supported)
Quick Start
1. Wizard — let it pick settings for your hardware
amazingvmsloth wizard --model Qwen/Qwen2.5-0.5B
Analyzes your GPU/CPU and prints a ready-to-run command.
2. Train
amazingvmsloth train \
--model Qwen/Qwen2.5-0.5B \
--dataset tatsu-lab/alpaca \
--epochs 3 \
--batch-size 2 \
--grad-accum 4 \
--lora-r 16 \
--output-dir ./output
Supports chat-format datasets too:
amazingvmsloth train \
--model Qwen/Qwen2.5-0.5B \
--dataset angrygiraffe/claude-opus-4.6-4.7-reasoning-8.7k \
--dataset-format chat \
--max-samples 1000 \
--output-dir ./thinking_lora
3. Convert LoRA to merged model
amazingvmsloth merge \
--model Qwen/Qwen2.5-0.5B \
--lora ./output \
--output ./merged_model
4. Run inference
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("./merged_model", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B")
messages = [{"role": "user", "content": "What is 2+2?"}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0]))
CLI Commands
| Command | Description |
|---|---|
wizard |
Interactive config generator based on your hardware |
train |
Fine-tune a model with LoRA |
merge |
Merge LoRA weights into base model |
convert |
Merge + convert to GGUF (requires llama.cpp) |
info |
Show model info and VRAM estimates |
bench |
Benchmark vs unsloth |
Hardware Tiers
| GPU VRAM | Strategy |
|---|---|
| 4-6 GB | 4-bit quant, batch=1, grad accum=8, seq=512, tiny LoRA |
| 6-12 GB | 4-bit quant, batch=1-2, grad accum=4, seq=1024 |
| 12-24 GB | 4-bit or full precision, batch=2-4, torch.compile |
| 24+ GB | Full precision, no grad checkpointing, large batch |
| CPU only | fp32/bf16, torch.compile, physical-core threading |
Key Features
- Model Architectures: Qwen 3.5 (e.g. Qwen/Qwen3.5-0.8B), Gemma 3 (1B/4B), Gemma 2, LLaMA 3.2/3.1, Mistral, DeepSeek v2/v3/R1, and multimodal/TTS models
- RTX 3050 & Low-VRAM Tuned: Ampere TensorFloat-32 (TF32) acceleration, fp16 Tensor Core speedup, non-blocking CUDA transfers, and automated 4-bit memory management
- rsLoRA scaling for stable training at any rank
- 4-bit/8-bit quantization via bitsandbytes
- XFormers/SDPA attention patching (Flash Attention on Linux)
- Sequence packing for 2-3x throughput
- Gradient checkpointing with selective layer skipping
- Multi-GPU: DDP, FSDP, DeepSpeed, pipeline parallelism
- Layer offloading via
accelerate.dispatch_model - CPU training with IPEX, pre-packing, torch.compile
- PagedAdamW8bit optimizer for low-VRAM training
- Checkpoint resume with full RNG/optimizer state
- Tqdm progress bar with live loss + VRAM display
Example: 500 Steps on Dolly
amazingvmsloth train \
--model Qwen/Qwen2.5-0.5B \
--dataset databricks/databricks-dolly-15k \
--dataset-format alpaca \
--epochs 1 --batch-size 2 --grad-accum 2 \
--max-samples 1000 --max-seq-length 512 \
--lora-r 16 --output-dir ./dolly_lora --packing
This runs ~500 steps in ~10 minutes on a 4GB RTX 3050.
Project Structure
amazingvmsloth/
├── lora.py # LoRA with rsLoRA, device-aware init
├── quantization.py # 4-bit/8-bit quant, kbit training prep
├── attention.py # SDPA/XFormers patching
├── trainer.py # AmazingTrainer with tqdm, packing, offloading
├── cpu_trainer.py # CpuTrainer for CPU-only training
├── packing.py # Sequence packing collators
├── gradient.py # GradientAccumulator
├── optimizer.py # PagedAdamW8bit, CpuOffloadedAdamW
├── offload.py # Layer offloading via accelerate
├── cli.py # CLI entrypoint
├── wizard.py # Hardware-aware config generator
├── bench.py # Benchmark vs unsloth
└── utils/
├── banner.py # Startup banner with GPU info
├── memory.py # VRAM estimation
├── patching.py # LoRA save/load helpers
└── save_load.py # Model save/merge
Benchmarks
On RTX 3050 4GB Laptop GPU:
| Library | Time (1 epoch, 500 samples) | Peak VRAM |
|---|---|---|
| amazingvmsloth | 5.3s | 1.07 GB |
| unsloth | 10.1s | 0.96 GB |
1.91x faster on small runs with pre-quantized models.
License
MIT
Metadata
Release files for amazingvmsloth 0.7.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| amazingvmsloth-0.7.4.tar.gz | 104.9 kB | Without provenance | Details for amazingvmsloth-0.7.4.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| amazingvmsloth-0.7.4-py3-none-any.whl | Python 3 | none | any | Details for amazingvmsloth-0.7.4-py3-none-any.whl |
Total release size: 210.9 kB
Release files / amazingvmsloth-0.7.4.tar.gz
| Download URL | amazingvmsloth-0.7.4.tar.gz |
|---|---|
| Size | 104.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
88ea0a81819b7dede02c640ca918033efd1b44e6b716b393b9b2672eedf2bdd7
|
|
BLAKE2b-256 checksum How to use checksums |
2f2f391b3578c605ebfcfcf5458e79fafe50263e876e842467e579a485441441
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.10
|
Release files / amazingvmsloth-0.7.4-py3-none-any.whl
| Download URL | amazingvmsloth-0.7.4-py3-none-any.whl |
|---|---|
| Size | 106.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
62860482abd086644da8cd3e5d7b93d39c3465bd7cd43f12b77bd7f3fff0f5a2
|
|
BLAKE2b-256 checksum How to use checksums |
2c2333e3f5af2b327cefe27dc296ccc9b1ed7814aebe5424f0af75873a630b47
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.10
|
Security for amazingvmsloth 0.7.4
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.