What Sparse Does

Sparse compresses fine-tuned models and datasets as deltas from their base versions.

Key benefit: Works on models you've already trained - no LoRA required during training.

Works with: Full fine-tunes, RLHF, model merges, translated/augmented datasets

Quick Start

pip install sparse-llm

Compress a Fine-tune

# Lossless compression (~1.4GB for 7B model)
sparse compress meta-llama/Llama-2-7b-hf ./my-finetune -o ./my-delta

# OR: Lossy compression (~50MB, LoRA-equivalent quality)
sparse compress-lossy meta-llama/Llama-2-7b-hf ./my-finetune -o ./my-delta --rank 16

Reconstruct from Delta

# From lossless delta
sparse reconstruct meta-llama/Llama-2-7b-hf ./my-delta -o ./reconstructed-model

# From lossy delta
sparse reconstruct-lossy meta-llama/Llama-2-7b-hf ./my-delta -o ./reconstructed-model

Dataset Delta

# Compress derivative dataset
sparse dataset-compress squad squad_v2 -o ./squad_v2_delta

# Reconstruct
sparse dataset-reconstruct ./squad_v2_delta

How It Works

Fine-tuned Model (14GB)  -  Base Model (14GB)  =  Delta (1.4GB or 50MB)
                                                        ↓
                                              Reconstruct: Base + Delta

• Lossless: Sparse + INT8 encoding → ~10% of original size, 100% quality
• Lossy (SVD): Low-rank approximation → ~0.4% of original, ~95-99% quality

CLI Reference

# Lossless compression (100% quality)
sparse compress <base> <finetune> -o <output>
sparse reconstruct <base> <delta> [-o <output>]

# Lossy compression (~50MB, LoRA-equivalent quality)
sparse compress-lossy <base> <finetune> -o <output> [--rank 16]
sparse reconstruct-lossy <base> <delta> [-o <output>]

# Dataset commands
sparse dataset-compress <base> <derivative> -o <output>
sparse dataset-reconstruct <delta_dir>
sparse dataset-estimate <base> <derivative>

# Info
sparse info <path>

Python API

from core import compress_delta, reconstruct_from_delta
from core import compress_delta_svd_full, reconstruct_from_svd_delta

# Lossless compression
manifest = compress_delta(
    base_model_id="meta-llama/Llama-2-7b-hf",
    finetune_model_id="./my-finetune",
    output_path="./my-delta"
)
print(f"Compression: {manifest.compression_ratio:.1f}x")  # ~10x

# Extract LoRA (lossy, LoRA-equivalent)
manifest = compress_delta_svd_full(
    base_model_id="meta-llama/Llama-2-7b-hf",
    finetune_model_id="./my-finetune",
    output_path="./my-svd-delta",
    rank=16  # Like LoRA rank
)
print(f"Compression: {manifest.compression_ratio:.1f}x")  # ~280x

# Reconstruct (lossless)
model = reconstruct_from_delta("meta-llama/Llama-2-7b-hf", "./my-delta")

# Reconstruct from extracted LoRA
model = reconstruct_from_svd_delta("meta-llama/Llama-2-7b-hf", "./my-lora-delta")

Dataset API

from core import compress_dataset_delta, reconstruct_from_dataset_delta

# Compress
manifest = compress_dataset_delta("squad", "squad_v2", "./squad_v2_delta")
print(f"Savings: {manifest['size_stats']['savings_pct']:.1f}%")

# Reconstruct
dataset = reconstruct_from_dataset_delta("./squad_v2_delta")

Performance

All optimizations are automatic - no configuration needed:

• Rust SIMD acceleration: 5-10x faster compression
• Base model caching: ~20s saved per compression
• Smart heuristics: 10-20% better compression ratios
• GPU reconstruction: 2-3x faster on CUDA
• Lazy loading: 50-70% memory reduction for 30B+ models

Typical speedup: ~60s → ~8-12s (5-8x faster)

📚 Advanced optimizations: See API_REFERENCE.md for MmapDeltaStorage, DifferentialCompressor, and other utilities.

Sparse vs LoRA

Key insight: sparse compress-lossy gives you LoRA-sized files (~50MB) from models that weren't trained with LoRA.

Requirements

• Python 3.9+
• PyTorch 2.0+
• transformers
• Rust (included in wheel, no setup needed)

License

Apache 2.0 - See LICENSE for details.

Free for personal and commercial use.