isagellm-compression 0.5.2.8

Model Compression & Acceleration Module for sageLLM

sagellm-compression

Model Compression & Acceleration Module for SageLLM (Task 3).

📌 Architecture & Responsibility

sagellm-compression is the core module responsible for model compression (quantization, sparsity) and inference acceleration strategies (speculative decoding, CoT optimizations).

Dependency Graph

graph TD
    Protocol[isagellm-protocol] --> Backend[isagellm-backend]
    Protocol --> Compression[isagellm-compression]
    Backend --> Compression
    Compression --> Core[isagellm-core]
    Compression --> KVCache[isagellm-kv-cache]

• Depends on:
  • isagellm-protocol: Shared schemas and definitions.
  • isagellm-backend: Hardware acceleration kernels (for Quantization).
• Used by:
  • isagellm-core: Main inference engine (integrates acceleration strategies).
  • isagellm-kv-cache: Uses compression techniques for KV storage.

✨ Features

• Chain-of-Thought (CoT): Template management and prompt engineering strategies (Zero-shot, Few-shot, Self-Consistency).
• Quantization (Task 3.1, Implemented): INT8/INT4 weight and activation quantization.
• Sparsity (Planned Task 3.2): Structured and unstructured pruning support.
• Speculative Decoding (Task 3.3, Implemented): Draft-verify orchestration with acceptance-rate metrics.
• Kernel Fusion (Planned Task 3.4): Operator fusion optimization.

📦 Installation

pip install isagellm-compression

Requirements

• Python >= 3.10
• isagellm-protocol >= 0.4.0.0
• isagellm-backend >= 0.4.0.0

🚀 Quick Start

Chain-of-Thought (CoT) Templates

Currently, the CoT module provides template management for reasoning tasks.

from sagellm_compression.cot import CoTTemplateManager

# Initialize the manager
# Note: Ensure you have the templates directory available
manager = CoTTemplateManager(template_dir="templates/cot")

# Load a specific strategy template (e.g., zero-shot reasoning)
try:
    template_content = manager.load_template("zero_shot")
    # Render the prompt with a question
    prompt = manager.render(template_content, question="What is the result of 25 * 14?")
    print("--- Generated Prompt ---")
    print(prompt)
except FileNotFoundError:
    print("Template not found. Please ensure 'templates/cot' exists.")

🛠️ Development

Setup

git clone [e-mail]:intellistream/sagellm-compression.git
cd sagellm-compression
./quickstart.sh

# Install in editable mode with dev dependencies
pip install -e ".[dev]"

Testing & Linting

# Run all tests
pytest -v

# Check code style
ruff check .
ruff format .

Core Principles

• Protocol-First: Changes involving schemas must update isagellm-protocol first.
• CPU-First: All compression logic must reference CPU implementation by default.
• Fail-Fast: Missing configurations must raise explicit errors.

📚 Documentation

• Protocol Specification (v0.1)
• Team & Roles
• Related Papers

🔄 贡献指南

请遵循以下工作流程：

1. 创建 Issue - 描述问题/需求

   gh issue create --title "[Bug] 描述" --label "bug,sagellm-compression"
   

2. 开发修复 - 在本地 fix/#123-xxx 分支解决

   git checkout -b fix/#123-xxx origin/main-dev
   # 开发、测试...
   pytest -v
   ruff format . && ruff check . --fix
   

3. 发起 PR - 提交到 main-dev 分支

   gh pr create --base main-dev --title "Fix: 描述" --body "Closes #123"
   

4. 合并 - 审批后合并到 main-dev

更多详情见 .github/copilot-instructions.md

📅 Versioning & Changelog

Current Version: 0.4.0.10

See CHANGELOG.md for full history.

License

Private - IntelliStream Research Project