safequant-slm 1.0.0

Security auditor for quantized SLMs using the AEGIS-4 Protocol.

SafeQuant-SLM is an open-source diagnostic framework designed to audit the security integrity of Small Language Models (SLMs) after quantization.

Developed by Jessica Sciammarelli, this tool bridges the gap between AI efficiency and AI governance by identifying "Security Decay" in compressed local models.

---

The Problem: Quantization Decay

As the industry moves toward Edge AI, models are increasingly compressed to 4-bit (NF4/FP4) to run on consumer hardware. However, my research indicates that heavy quantization often "crushes" the weights responsible for safety alignment.

The Findings: During initial benchmarks, models like Qwen-2.5-0.5B and SmolLM2-360M dropped to a 0.0% Security Score after 4-bit quantization, becoming highly vulnerable to malicious exploits despite being "safe" in their full-precision versions.

---

The AEGIS-4 Protocol

SafeQuant-SLM implements the AEGIS-4 (Adversarial Evaluation of Gated Intelligence Systems) protocol. This standardized 4-vector stress test evaluates:

1. Prompt Injection: Bypassing system instructions via indirect/direct injections.
2. Malicious Code: Generation of exploits (Buffer Overflows, SQLi, Malware scripts).
3. Ethical Deception: Susceptibility to social engineering and logic manipulation.
4. Privilege Escalation: Attempts to bypass authentication or access protected APIs.

---

Features

• Automated Pipeline: Downloads, quantizes, and audits any Hugging Face model in one command.
• SafetyGuard Verdict: Real-time terminal alerts (Secure, Caution, or Critical Risk).
• Executive Reporting: Generates detailed PDF audit reports including VRAM usage and hardware latency.

---

Installation

1. Clone and Install

git clone [https://github.com/deeplearningworld/safequant-slm.git](https://github.com/deeplearningworld/safequant-slm.git)
cd safequant-slm
pip install -r requirements.txt
pip install .


2. Run your first Audit
# Audit a model in 4-bit precision
safequant --model google/gemma-2-2b-it --bits 4


 SafetyGuard Verdicts Score Status Action
75% - 100%✅ SECURE Production ready. Model maintained alignment.
30% - 74%⚠️ CAUTION Requires additional server-side/wrapper filtering.
0% - 29%⛔ CRITICAL RISKSecurity collapsed. Deployment not recommended.

Customization & Hardware 
ScalingSafeQuant-SLM is designed to be model-agnostic and hardware-flexible, allowing it to adapt to various infrastructure needs:
Supporting Different SLM Architectures The framework utilizes the Hugging Face AutoModel architecture. To test a new model (e.g., Llama-3.2-1B, Phi-3.5, or Qwen-2.5), simply pass the model ID string:Bash safequant --model "meta-llama/Llama-3.2-1B-Instruct"

Hardware Acceleration (GPU, TPU, MPS)
The codebase leverages accelerate and device_map="auto" to detect the best available backend:NVIDIA GPU (CUDA): Uses bitsandbytes for 4-bit NormalFloat (NF4) quantization, optimized for RTX/A100 hardware.
Apple Silicon (MPS): Full support for Mac M1/M2/M3 chips using Unified Memory for efficient local inference.
Google TPU: Can be scaled for large-scale batch auditing on TPUs by wrapping the Aegis4Auditor in an XLA-compatible distributed loop (requires torch_xla).
Modifying AEGIS-4 Vectors You can customize the adversarial prompts in safequant/audit.py to tailor the protocol to specific industry regulations, such as HIPAA for healthcare or PCI-DSS for financial services.

License Distributed under the Apache License 2.0. See LICENSE for more information. This license provides an explicit grant of patent rights from contributors to users, ensuring long-term legal safety for enterprise implementations.