forgelm 0.4.0

Config-driven LLM fine-tuning with safety evaluation, EU AI Act compliance, and 6 alignment methods.

ForgeLM

ForgeLM is a config-driven, enterprise-ready LLM fine-tuning toolkit. It supports the full modern post-training stack — from supervised fine-tuning to preference alignment to reasoning RL — with integrated safety evaluation, EU AI Act compliance, and CI/CD-native design.

Features

Training

• 6 Trainer Types: SFT, DPO, SimPO, KTO, ORPO, GRPO — the complete alignment stack
• Unsloth & Transformers: 2-5x faster training with unsloth backend, or standard transformers
• 4-Bit QLoRA & DoRA: NF4 quantization with LoRA, DoRA, PiSSA, and rsLoRA support
• GaLore: Optimizer-level memory optimization — full-parameter training via gradient low-rank projection (alternative to LoRA)
• Long-Context Training: RoPE scaling, NEFTune noise injection, sliding window attention, sample packing
• Multi-Dataset Training: Mix multiple datasets with configurable ratios
• Synthetic Data Pipeline: Teacher-to-student distillation with --generate-data CLI flag
• DeepSpeed & FSDP: Multi-GPU distributed training with ZeRO-2/3 presets
• MoE Support: Fine-tune Mixture of Experts models (Qwen3, Mixtral, DeepSeek)
• GPU Cost Estimation: Auto-detection for 18 GPU models with per-run cost tracking

Evaluation & Safety

• Automated Benchmarking: Post-training evaluation via lm-evaluation-harness
• Safety Evaluation: Llama Guard classifier with confidence-weighted scoring, S1-S14 harm categories, severity levels, cross-run trend tracking, and auto-revert
• LLM-as-Judge: API-based (OpenAI) or local model scoring for quality assessment
• Auto-Revert: Automatically discard models that fail loss, benchmark, or safety thresholds

Post-Training (v0.4.0)

• Interactive Chat: forgelm chat ./model — streaming REPL with /reset, /save, /temperature, /system commands; optional Llama Guard safety routing
• GGUF Export: forgelm export ./model --quant q4_k_m — wraps llama-cpp-python converter; 6 quant levels; SHA-256 appended to integrity manifest
• Deployment Configs: forgelm deploy ./model --target ollama|vllm|tgi|hf-endpoints — generates ready-to-use config files; does not start the server
• VRAM Fit Check: forgelm --config my.yaml --fit-check — pre-flight memory estimator; FITS / TIGHT / OOM / UNKNOWN verdict with recommendations

Enterprise & MLOps

• Config-Driven: Declarative YAML — built for CI/CD pipelines, not notebooks
• EU AI Act Compliance: Auto-generated audit trails, data provenance (SHA-256), training manifests
• Docker: Official Dockerfile and docker-compose for portable deployment
• Offline / Air-Gapped: Full operation without internet for regulated industries
• JSON Output: Machine-readable results with --output-format json for pipeline integration
• Webhook Notifications: Slack/Teams alerts on training start, success, or failure
• W&B / MLflow / TensorBoard: Flexible experiment tracking via report_to
• Model Card Generation: Auto-generated HF-compatible model cards with metrics and benchmarks
• Model Merging: TIES, DARE, SLERP, linear merge of multiple adapters via --merge

---

Quick Start

# Install
pip install -e .
pip install -e ".[export]"   # GGUF export (optional, non-Windows)

# Generate config interactively
forgelm --wizard

# Validate without training
forgelm --config my_config.yaml --dry-run

# Check VRAM before a long run
forgelm --config my_config.yaml --fit-check

# Train
forgelm --config my_config.yaml

# After training: chat, export, deploy
forgelm chat ./checkpoints/final_model
forgelm export ./checkpoints/final_model --output model.gguf --quant q4_k_m
forgelm deploy ./checkpoints/final_model --target ollama --output ./Modelfile

See the Quick Start Guide for a complete walkthrough.

---

Guides

Guide	Description
Quick Start	First fine-tuned model in 5 minutes
Alignment (DPO/SimPO/KTO/GRPO)	Complete post-training stack
CI/CD Pipeline Integration	GitHub Actions, GitLab CI, Docker
Enterprise Deployment	Docker, air-gapped, multi-GPU
Safety & Compliance	EU AI Act, safety evaluation
Distributed Training	DeepSpeed ZeRO, FSDP, multi-node
Troubleshooting & FAQ	Common issues and solutions

Reference Documentation

1. Architecture Overview (Türkçe)
2. Configuration Guide (Türkçe)
3. Usage & Execution (Türkçe)
4. Data Preparation Format (Türkçe)
5. Product Strategy (Türkçe)
6. Roadmap (Türkçe)

Notebooks

• Quick Start — SFT Fine-Tuning
• DPO Preference Alignment
• KTO Binary Feedback
• GRPO Reasoning RL
• Multi-Dataset Training

---

Installation

# From PyPI
pip install forgelm

# Or from source
git clone https://github.com/cemililik/ForgeLM.git
cd ForgeLM
pip install -e .

Optional Dependencies

pip install -e ".[qlora]"        # 4-bit quantization (Linux)
pip install -e ".[unsloth]"      # Unsloth backend (Linux)
pip install -e ".[eval]"         # lm-evaluation-harness benchmarks
pip install -e ".[tracking]"     # W&B experiment tracking
pip install -e ".[distributed]"  # DeepSpeed multi-GPU
pip install -e ".[merging]"      # mergekit model merging
pip install -e ".[dev]"          # pytest, ruff (development)

---

Docker

# Build (with benchmarking support)
docker build -t forgelm --build-arg INSTALL_EVAL=true .

# Train
docker run --gpus all \
  -v $(pwd)/my_config.yaml:/workspace/config.yaml \
  -v $(pwd)/output:/workspace/output \
  forgelm --config /workspace/config.yaml

# Multi-GPU
docker run --gpus all --shm-size=16g \
  forgelm:latest \
  torchrun --nproc_per_node=4 -m forgelm.cli --config /workspace/config.yaml

---

CLI

forgelm --config job.yaml                    # Train
forgelm --config job.yaml --dry-run          # Validate config
forgelm --config job.yaml --output-format json  # JSON output for CI/CD
forgelm --config job.yaml --resume           # Resume from checkpoint
forgelm --config job.yaml --offline          # Air-gapped mode
forgelm --config job.yaml -q                 # Quiet mode (warnings only)
forgelm --config job.yaml --benchmark-only /path/to/model  # Evaluate only
forgelm --config job.yaml --merge            # Merge models
forgelm --config job.yaml --compliance-export ./audit/  # Export audit artifacts
forgelm --wizard                             # Interactive config generator
forgelm --version                            # Show version

---

Project Structure

forgelm/
├── cli.py           # CLI with 10+ modes (train, dry-run, merge, benchmark, wizard...)
├── config.py        # Pydantic config (19 models: training, evaluation, distributed...)
├── data.py          # Dataset loading (SFT, DPO, KTO, GRPO formats + multi-dataset)
├── model.py         # Model loading (transformers, unsloth, MoE, PEFT)
├── trainer.py       # Training orchestration (6 trainer types via TRL, GaLore, long-context)
├── inference.py     # Shared inference primitives (load, generate, stream, adaptive sampling)
├── chat.py          # Interactive terminal REPL with streaming and slash commands
├── export.py        # GGUF export via llama-cpp-python
├── fit_check.py     # Pre-flight VRAM estimator (FITS / TIGHT / OOM / UNKNOWN)
├── deploy.py        # Deployment config generator (Ollama, vLLM, TGI, HF Endpoints)
├── results.py       # TrainResult dataclass
├── benchmark.py     # lm-evaluation-harness integration
├── safety.py        # Post-training safety evaluation (Llama Guard)
├── judge.py         # LLM-as-Judge evaluation (API + local)
├── compliance.py    # EU AI Act compliance export & data provenance
├── model_card.py    # Auto-generated HF model cards
├── merging.py       # Model merging (TIES, DARE, SLERP, linear)
├── synthetic.py     # Synthetic data generation (teacher→student distillation)
├── wizard.py        # Interactive configuration wizard
├── webhook.py       # Slack/Teams webhook notifications
└── utils.py         # Authentication & checkpoint management

configs/deepspeed/   # ZeRO-2, ZeRO-3, ZeRO-3+Offload presets
notebooks/           # Colab-ready Jupyter notebooks
tests/               # 430 passed (+34 skipped) across 30 test files
docs/guides/         # Quickstart, alignment, CI/CD, enterprise, safety guides

---

License

Apache License 2.0