sagellm-compression
Model Compression & Acceleration Module for SageLLM (Task 3).
📌 Architecture & Responsibility
sagellm-compression is the core module responsible for model compression (quantization, sparsity) and inference acceleration strategies (speculative decoding, CoT optimizations).
Dependency Graph
graph TD
Protocol[isagellm-protocol] --> Backend[isagellm-backend]
Protocol --> Compression[isagellm-compression]
Backend --> Compression
Compression --> Core[isagellm-core]
Compression --> KVCache[isagellm-kv-cache]
- Depends on:
isagellm-protocol: Shared schemas and definitions.isagellm-backend: Hardware acceleration kernels (for Quantization).
- Used by:
isagellm-core: Main inference engine (integrates acceleration strategies).isagellm-kv-cache: Uses compression techniques for KV storage.
✨ Features
- Chain-of-Thought (CoT) Acceleration (Task 3.5, Implemented): CoT detection + prompt compression + chain shortening with metrics.
- Quantization (Task 3.1, Implemented): INT8/INT4 weight and activation quantization.
- Sparsity (Task 3.2, Implemented): Structured (N:M / block) and unstructured pruning support.
- Speculative Decoding (Task 3.3, Implemented): Draft-verify orchestration with acceptance-rate metrics.
- Kernel Fusion (Task 3.4, Implemented): Attention/MLP fusion interfaces with CPU stub for CI.
📦 Installation
pip install isagellm-compression
Requirements
- Python >= 3.10
isagellm-protocol >= 0.4.0.0isagellm-backend >= 0.4.0.0
🚀 Quick Start
Chain-of-Thought (CoT) Templates
Currently, the CoT module provides template management for reasoning tasks.
from sagellm_compression.cot import CoTTemplateManager
# Initialize the manager
# Note: Ensure you have the templates directory available
manager = CoTTemplateManager(template_dir="templates/cot")
# Load a specific strategy template (e.g., zero-shot reasoning)
try:
template_content = manager.load_template("zero_shot")
# Render the prompt with a question
prompt = manager.render(template_content, question="What is the result of 25 * 14?")
print("--- Generated Prompt ---")
print(prompt)
except FileNotFoundError:
print("Template not found. Please ensure 'templates/cot' exists.")
🛠️ Development
Setup
git clone git@github.com:intellistream/sagellm-compression.git
cd sagellm-compression
./quickstart.sh --standard
# 开发联调(在 standard 基础上,本地 editable 覆盖)
./quickstart.sh --dev
# 查看帮助
./quickstart.sh --help
# quickstart 会在安装前自动清理同前缀历史包(isagellm-*)
# 并且不会创建 venv/.venv,复用当前非-venv Python 环境
模式说明:
--standard:优先安装 PyPI 稳定包(默认模式)--dev:先执行 standard,再执行本地 editable 覆盖(--no-deps)
Testing & Linting
# Run all tests
pytest -v
# Run issue #8 unit-test subset
pytest -v tests/unit/test_quantization.py tests/unit/test_sparsity.py tests/unit/test_speculative_config.py
# Check code style
ruff check .
ruff format .
Core Principles
- Protocol-First: Changes involving schemas must update
isagellm-protocolfirst. - CPU-First: All compression logic must reference CPU implementation by default.
- Fail-Fast: Missing configurations must raise explicit errors.
📚 Documentation
🔄 贡献指南
请遵循以下工作流程:
-
创建 Issue - 描述问题/需求
gh issue create --title "[Bug] 描述" --label "bug,sagellm-compression"
-
开发修复 - 在本地
fix/#123-xxx分支解决git checkout -b fix/#123-xxx origin/main-dev # 开发、测试... pytest -v ruff format . && ruff check . --fix
-
发起 PR - 提交到
main-dev分支gh pr create --base main-dev --title "Fix: 描述" --body "Closes #123"
-
合并 - 审批后合并到
main-dev
更多详情见 .github/copilot-instructions.md
📅 Versioning & Changelog
Current Version: 0.4.0.10
See CHANGELOG.md for full history.
License
Private - IntelliStream Research Project
Metadata
Release files for isagellm-compression 0.5.4.12
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| isagellm_compression-0.5.4.12.tar.gz | 69.4 kB | Without provenance | Details for isagellm_compression-0.5.4.12.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| isagellm_compression-0.5.4.12-py2.py3-none-any.whl | Python 2, Python 3 | none | any | Details for isagellm_compression-0.5.4.12-py2.py3-none-any.whl |
Total release size: 160.0 kB
Release files / isagellm_compression-0.5.4.12.tar.gz
| Download URL | isagellm_compression-0.5.4.12.tar.gz |
|---|---|
| Size | 69.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
3088a2770261d53f03b2de1d573adfcde359ed58c1179731c5c6df506b8d1202
|
|
BLAKE2b-256 checksum How to use checksums |
02cf8cd2e41d18abb318fe7c0766f6cbe50b8a7fdb2767b891acf2bc8f2e0a41
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.15
|
Release files / isagellm_compression-0.5.4.12-py2.py3-none-any.whl
| Download URL | isagellm_compression-0.5.4.12-py2.py3-none-any.whl |
|---|---|
| Size | 90.6 kB |
| Tags | Python 2 Python 3 |
|
SHA-256 checksum How to use checksums |
c7be6c089b600fcb40043a3873dc5064d34d19e682a5ecc70f1aca1041ccb38f
|
|
BLAKE2b-256 checksum How to use checksums |
88de431d2c8564daeeeb0772aa1532929a32b0c570d097fbf1caf352a898cdd9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.15
|
Security for isagellm-compression 0.5.4.12
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.