isagellm-compression 0.4.0.5

Model Compression & Acceleration Module for sageLLM

sagellm-compression

Protocol Compliance (Mandatory)

• MUST follow Protocol v0.1: https://github.com/intellistream/sagellm-docs/blob/main/docs/specs/protocol_v0.1.md
• Any globally shared definitions (fields, error codes, metrics, IDs, schemas) MUST be added to Protocol first.

Inference acceleration tools for LLM: quantization, sparsity, speculative decoding, kernel fusion, and more.

Features

• Quantization (INT8/INT4)
• Sparsity (structured and unstructured pruning)
• Speculative decoding
• Kernel fusion
• Chain-of-Thought acceleration

Installation

pip install isagellm-compression

Quick Start

from sagellm_compression import QuantizationConfig, apply_quantization

config = QuantizationConfig(method="int8", per_channel=True)
quantized_model = apply_quantization(model, config)

Development

git clone [e-mail]:intellistream/sagellm-compression.git
cd sagellm-compression
./quickstart.sh

pip install -e ".[dev]"
pytest tests/ -v

Documentation

• docs/

🔄 贡献指南

请遵循以下工作流程：

1. 创建 Issue - 描述问题/需求

   gh issue create --title "[Bug] 描述" --label "bug,sagellm-compression"
   

2. 开发修复 - 在本地 fix/#123-xxx 分支解决

   git checkout -b fix/#123-xxx origin/main-dev
   # 开发、测试...
   pytest -v
   ruff format . && ruff check . --fix
   

3. 发起 PR - 提交到 main-dev 分支

   gh pr create --base main-dev --title "Fix: 描述" --body "Closes #123"
   

4. 合并 - 审批后合并到 main-dev

更多详情见 .github/copilot-instructions.md

License

Private - IntelliStream Research Project