auto-lora 0.1.0

Automated Hyperparameter Optimization Platform for Efficient LLM Fine-Tuning

Auto-LoRA

🚀 Automated Hyperparameter Optimization Platform for Efficient LLM Fine-Tuning

Auto-LoRA automates parameter-efficient fine-tuning of Large Language Models using LoRA/QLoRA. It reduces trial-and-error by automatically selecting optimal training configurations based on dataset characteristics and available hardware.

Features

• ⚡ Unsloth Powered — 2x–5x faster training and 70% less VRAM (automatic detection)
• 📊 Scientific Benchmarking — Journal-grade metrics (ROUGE-L, TPS, Latency, Peak VRAM)
• 🎨 Dynamic Feedback — Real-time row-by-row dataset loading and analysis progress bars
• 🎯 Automatic Tuning — Bayesian optimization via Optuna finds the best LoRA configuration
• 🔍 Hardware-Aware — Auto-detects GPU/VRAM and adjusts strategy (LoRA vs QLoRA)
• 📈 Rich Reports — HTML dashboards, publication-quality charts, CSV exports
• 🔧 Presets — Optimized defaults for chatbot, coding, QA, summarization, domain tasks
• 💾 Run History — SQLite tracking of all runs, trials, and artifacts

Quick Start

Installation

# Clone the repository
git clone https://github.com/shrey1720/alora.git
cd alora

# Install in development mode
pip install -e ".[dev]"

# Optional: Install Unsloth for 2-5x speedup
# (Recommended for NVIDIA GPUs)
pip install unsloth xformers

Usage

# 1. Check system health
auto-lora doctor

# 2. Initialize project
auto-lora init

# 3. Train with automatic tuning
auto-lora train --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --data datasets/sample_faq.json --max-trials 5

# 4. View results
auto-lora runs list
auto-lora runs show <run_id>

# 5. Generate report
auto-lora report --run <run_id>

# 6. Scientific Benchmark
# Generates a journal-ready technical profile (TPS, ROUGE-L, VRAM)
auto-lora benchmark --run <run_id> --references gold_answers.txt

# 7. Export adapter
auto-lora export --run <run_id>

Using Presets

# Chatbot preset (conversational focus)
auto-lora train --model mistralai/Mistral-7B-v0.1 --data chat_data.json --preset chatbot

# Coding preset (lower LR, code tokenization)
auto-lora train --model codellama/CodeLlama-7b-hf --data code_data.json --preset coding

# QA preset (fact-oriented)
auto-lora train --model meta-llama/Llama-2-7b-hf --data qa_data.json --preset qa

CLI Commands

Command	Description
auto-lora init	Initialize project structure
auto-lora doctor	System health check
auto-lora train	Train with automatic tuning
auto-lora runs list	List all training runs
auto-lora runs show <id>	Show run details
auto-lora report --run <id>	Generate HTML report
auto-lora benchmark --run <id>	Benchmark trained adapter
auto-lora export --run <id>	Export adapter for deployment

Architecture

auto_lora/
├── cli/          # Typer CLI commands
├── core/         # Config, presets, constants
├── hardware/     # GPU/CPU detection
├── dataset/      # Loading, validation, formatting
├── tuner/        # Optuna hyperparameter search
├── trainer/      # LoRA/QLoRA training engine
├── metrics/      # Scoring, benchmarking
├── reports/      # HTML reports, charts, exports
├── db/           # SQLite run/trial storage
└── utils/        # Logging, display, filesystem

Requirements

• Python >= 3.10
• PyTorch >= 2.1.0
• NVIDIA GPU with CUDA (recommended, CPU supported)
• 8GB+ VRAM for QLoRA, 16GB+ for LoRA

License

MIT