axolotl 0.19.0

A Free and Open Source LLM Fine-tuning Framework

🎉 Latest Updates

• 2026/08:
  • New model support has been added in Axolotl for Ling 3.0, Muse Glimmer, North Micro Vision Instruct and Shieldstral.
• 2026/07:
  • NVFP4 (4-bit) MoE LoRA training is now supported via ScatterMoE (W4A16) and SonicMoE (W4A4), including adapter merge back into a plain NVFP4 checkpoint.
• 2026/06:
  • Expert Parallelism (EP) for distributed MoE training via DeepEP, remote training through Tinker-compatible APIs, Context Parallelism for hybrid SSM models (Nemotron-H, Falcon-H1, Bamba), BitNet 1.58-bit fine-tuning, and a multimodal assistant-only loss-masking fix.
• 2026/04:
  • New model support has been added in Axolotl for Mistral Medium 3.5 and Gemma 4.
  • New RL and kernels: Async GRPO (up to 58% faster steps), Flash Attention 4, NeMo Gym, and EBFT.
  • Axolotl is now uv-first and has SonicMoE fused LoRA support.
• 2026/03:
  • New model support has been added in Axolotl for Mistral Small 4, Qwen3.5, Qwen3.5 MoE, GLM-4.7-Flash, GLM-4.6V, and GLM-4.5-Air.
  • MoE expert quantization support (via quantize_moe_experts: true) greatly reduces VRAM when training MoE models (FSDP2 compat).

✨ Overview

Axolotl is a free and open-source tool designed to streamline post-training and fine-tuning for the latest large language models (LLMs).

Features:

• Multiple Model Support: Train various models like GPT-OSS, LLaMA, Mistral, Mixtral, Pythia, and many more models available on the Hugging Face Hub.
• Multimodal Training: Fine-tune vision-language models (VLMs) including LLaMA-Vision, Qwen2-VL, Pixtral, LLaVA, SmolVLM2, GLM-4.6V, InternVL 3.5, Gemma 3n, PaddleOCR-VL, Muse Glimmer, and audio models like Voxtral with image, video, and audio support.
• Training Methods: Full fine-tuning, LoRA, QLoRA, GPTQ, QAT (int8/int4/FP8/NVFP4/MXFP4), FP8 mixed-precision training, NVFP4/MXFP4 MoE LoRA, Preference Tuning (DPO, IPO, KTO, ORPO), RL (GRPO, GDPO), and Reward Modelling (RM) / Process Reward Modelling (PRM).
• Easy Configuration: Re-use a single YAML configuration file across the full fine-tuning pipeline: dataset preprocessing, training, evaluation, quantization, and inference.
• Performance Optimizations: Multipacking, Flash Attention 2/3/4, Xformers, Flex Attention, SageAttention, Liger Kernel, Cut Cross Entropy, ScatterMoE, Sequence Parallelism (SP), LoRA optimizations, Multi-GPU training (FSDP1, FSDP2, DeepSpeed), Multi-node training (Torchrun, Ray), and many more!
• Flexible Dataset Handling: Load from local, HuggingFace, and cloud (S3, Azure, GCP, OCI) datasets.
• Cloud Ready: We ship Docker images and also PyPI packages for use on cloud platforms and local hardware.

🚀 Quick Start - LLM Fine-tuning in Minutes

Requirements:

• NVIDIA GPU (Ampere or newer for bf16 and Flash Attention) or AMD GPU
• Python >=3.11 (3.12 recommended)
• PyTorch ≥2.11.0 (2.12.1 recommended)

Google Colab

Installation

# install uv if you don't already have it installed (restart shell after)
curl -LsSf https://astral.sh/uv/install.sh | sh

# change depending on system
export UV_TORCH_BACKEND=cu130

# create a new virtual environment
uv venv --python 3.12
source .venv/bin/activate

uv pip install torch==2.12.1 torchvision
uv pip install --no-build-isolation axolotl[deepspeed]

# Download example axolotl configs, deepspeed configs
axolotl fetch examples
axolotl fetch deepspeed_configs  # OPTIONAL

Using Docker

Installing with Docker can be less error prone than installing in your own environment.

docker run --gpus '"all"' --ipc=host --rm -it axolotlai/axolotl:main-latest

Other installation approaches are described here.

Cloud Providers

Your First Fine-tune

# Fetch axolotl examples
axolotl fetch examples

# Or, specify a custom path
axolotl fetch examples --dest path/to/folder

# Train a model using LoRA
axolotl train examples/llama-3/lora-1b.yml

That's it! Check out our Getting Started Guide for a more detailed walkthrough.

📚 Documentation

• Installation Options - Detailed setup instructions for different environments
• Support Matrix - Feature support, compatibility, and known gaps
• Configuration Guide - Full configuration options and examples
• Dataset Loading - Loading datasets from various sources
• Dataset Guide - Supported formats and how to use them
• Multi-GPU Training
• Multi-Node Training
• Multipacking
• API Reference - Auto-generated code documentation
• FAQ - Frequently asked questions

AI Agent Support

Axolotl ships with built-in documentation optimized for AI coding agents (Claude Code, Cursor, Copilot, etc.). These docs are bundled with the pip package, no repo clone needed.

# Show overview and available training methods
axolotl agent-docs

# Topic-specific references
axolotl agent-docs sft                 # supervised fine-tuning
axolotl agent-docs grpo                # GRPO online RL
axolotl agent-docs preference_tuning   # DPO, KTO, ORPO, SimPO
axolotl agent-docs reward_modelling    # outcome and process reward models
axolotl agent-docs pretraining         # continual pretraining
axolotl agent-docs --list              # list all topics

# Dump config schema for programmatic use
axolotl config-schema
axolotl config-schema --field adapter

If you're working with the source repo, agent docs are also available at docs/agents/ and the project overview is in AGENTS.md.

🤝 Getting Help

• Join our Discord community for support
• Check out our Examples directory
• Read our Debugging Guide
• Need dedicated support? Please contact ✉️wing@axolotl.ai for options

🌟 Contributing

Contributions are welcome! Please see our Contributing Guide for details.

📈 Telemetry

Axolotl has opt-out telemetry that helps us understand how the project is being used and prioritize improvements. We collect basic system information, model types, and error rates, never personal data or file paths. Telemetry is enabled by default. To disable it, set AXOLOTL_DO_NOT_TRACK=1. For more details, see our telemetry documentation.

❤️ Sponsors

Interested in sponsoring? Contact us at wing@axolotl.ai

📝 Citing Axolotl

If you use Axolotl in your research or projects, please cite it as follows:

@software{axolotl,
  title = {Axolotl: Open Source LLM Post-Training},
  author = {{Axolotl maintainers and contributors}},
  url = {https://github.com/axolotl-ai-cloud/axolotl},
  license = {Apache-2.0},
  year = {2023}
}

📜 License

This project is licensed under the Apache 2.0 License - see the LICENSE file for details.