EdgeGuard

Sub-microsecond, On-Device AI Guardrails Engine in Rust and C++

EdgeGuard provides low-latency, zero-network security scanning and token-by-token guardrails for on-device LLMs (such as llama.cpp, vLLM, and Apple MLX).

Performance Benchmarks (Apple Silicon)

Features

• Sub-Microsecond Latency: Native zero-allocation Rust core.
• Streaming Guardrails: Real-time token streaming validation with rolling window and immediate abort.
• Dynamic Policy: Update and hot-reload rules (PII, Prompt Injection, Secrets) from policy.yaml without recompilation.
• Shannon Entropy: Unsupervised detection of high-entropy raw secret keys and passwords.
• Base64 Sniffer: Automatic decoding and scanning of obfuscated payloads in memory.

Quick Start

from edgeguard import EdgeGuard

guard = EdgeGuard(policy_path="policy.yaml")
result = guard.scan("Contact me at user@example.com")
print(result)

License

MIT