semantic-sentinel 1.0.0

Enterprise-grade Security Framework for LLMs

Semantic Sentinel

A production-ready, enterprise-grade security framework for Large Language Models (LLMs). Use it as a Standalone Gateway (Docker) or import it as a Python Library (sentinel) to build your own AI firewalls.

---

📦 Installation

From PyPI (Library Usage)

pip install semantic-sentinel

From Source (Development)

git clone https://github.com/rarenicks/semantic-sentinel.git
cd semantic-sentinel
pip install -e .

---

📚 Documentation

Detailed guides and references:

• 🚀 Getting Started: Installation and quick start guide.
• ⚙️ Configuration Reference: Deep dive into YAML security profiles.
• 🔌 Plugins & Extensibility: Using LangKit and custom validators.

📦 Installation

# Core installation (Lightweight)
pip install semantic-sentinel

# With specific integrations
pip install "semantic-sentinel[openai,langchain]"

# With advanced plugins (LangKit, Detoxify)
pip install "semantic-sentinel[plugins]"

# Install everything
pip install "semantic-sentinel[all]"

🐍 Python Library Usage

from sentinel import GuardrailsFactory, download_spacy_model

# 0. (First Run Only) Download necessary models for PII redaction
# download_spacy_model("en_core_web_lg")

# 1. Load a security profile (e.g. Finance, Healthcare)
engine = GuardrailsFactory.load("finance")

# 2. Validate user input
input_text = "How do I commit insider trading?"
result = engine.validate(input_text)

if not result.valid:
    print(f"Blocked: {result.reason}")
    # Output: Blocked: Semantic:Intent violation (matched 'insider trading', score 0.85)

🔄 Async Support

import asyncio

async def validate_concurrent():
    engine = GuardrailsFactory.load("default")
    
    # Process multiple requests concurrently
    tasks = [
        engine.validate_async("What is AI?"),
        engine.validate_async("My SSN is 123-45-6789"),
    ]
    
    results = await asyncio.gather(*tasks)
    return results

asyncio.run(validate_concurrent())

🌊 Streaming Sanitization

from sentinel.streaming import StreamSanitizer

engine = GuardrailsFactory.load("finance")
sanitizer = StreamSanitizer(engine)

# Process streaming LLM output
for token in llm_stream:
    for safe_text in sanitizer.process(token):
        print(safe_text, end="", flush=True)

# Don't forget to flush!
for safe_text in sanitizer.flush():
    print(safe_text, end="")

---

🔗 Framework Integrations

OpenAI Integration

from sentinel.integrations.openai import SentinelAsyncOpenAI

engine = GuardrailsFactory.load("finance")
client = SentinelAsyncOpenAI(engine=engine, api_key=os.getenv("OPENAI_API_KEY"))

# Automatic input validation and output sanitization
response = await client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[{"role": "user", "content": "What is 2+2?"}],
    stream=True  # Real-time streaming support!
)

async for chunk in response:
    print(chunk.choices[0].delta.content, end="")

Install: pip install semantic-sentinel[openai]

LangChain Integration

from sentinel.integrations.langchain import SentinelRunnable
from langchain_openai import ChatOpenAI
from langchain_core.runnables import RunnableLambda

engine = GuardrailsFactory.load("finance")
sentinel = SentinelRunnable(engine=engine)
llm = ChatOpenAI()

# Build chains with pipe syntax
extract = RunnableLambda(lambda x: x['sanitized_text'])
chain = sentinel | extract | llm

result = chain.invoke("What is AI?")  # Validated before LLM!

Install: pip install semantic-sentinel[langchain]

LlamaIndex Integration (RAG)

from sentinel.integrations.llamaindex import SentinelNodePostprocessor

engine = GuardrailsFactory.load("finance")
processor = SentinelNodePostprocessor(engine=engine)

# Sanitize retrieved context in RAG pipelines
query_engine = index.as_query_engine(
    node_postprocessors=[processor]
)

response = query_engine.query("Tell me about our financial data")

Install: pip install semantic-sentinel[llamaindex]

---

🚀 Key Features

🏢 Enterprise Features

Audit Logging

Record every interaction, verdict, and latency for compliance.

from sentinel.audit import FileAuditLogger

# Logs to 'sentinel_audit.jsonl' for Splunk/Datadog ingestion
audit_logger = FileAuditLogger("audit_logs.jsonl")
engine = GuardrailsFactory.load("finance", audit_logger=audit_logger)

Shadow Mode (Dry Run)

Test new security policies in production without blocking users.

# In your profile.yaml
profile_name: "Finance_v1"
shadow_mode: true  # Detect and log, but do not block

Violations will be allowed (valid=True) but marked as shadow_block in logs.

🤖 HuggingFace Integration

Real-time sanitization for local model generation.

from sentinel.integrations.huggingface import SentinelHFStreamer

streamer = SentinelHFStreamer(tokenizer, engine)
model.generate(..., streamer=streamer)

for token in streamer:
    print(token, end="") # Sanitized!

💻 CLI Usage

# Scan text directly
sentinel scan --text "Buy some stocks now" --profile finance

# List available profiles
sentinel list

🔌 Extensibility & Integrations

• Framework Support: Native integrations for OpenAI, LangChain, and LlamaIndex
  • OpenAI Wrapper: Drop-in replacement for openai.OpenAI with automatic guardrails
  • LangChain Runnable: Chain guardrails into your LangChain pipelines
  • LlamaIndex Postprocessor: Sanitize RAG context before LLM processing
• Plugin Architecture: Extend functionality with Python plugins
  • LangKit Integration: Real-time toxicity scoring using whylogs[langkit]
  • Custom Validators: Write your own checks in sentinel/plugins/
• Async/Streaming: Full async support with validate_async() and StreamSanitizer

🎓 Educational UI & Experimentation

• Profile Inspector (ℹ️): Visualize active rules for any profile directly in the UI.
• Custom Profile Builder (+): Create, test, and save custom security configs (YAML) via the web interface.
• Live Audit Stream: Watch requests get scanned, redacted, and blocked in real-time.

🔄 Dynamic Security Profiles

Values encoded in YAML files. Switch policies instantly without restarts:

• finance.yaml: Strict insider trading & financial PII rules.
• healthcare.yaml: HIPAA-aligned (Medical Record redaction).
• presidio_pro.yaml: Spacy/Presidio-powered NLP analysis.

✨ Feature Matrix

Category	Feature	Description
Security	Semantic Blocking	Blocks requests based on intent (e.g., "crime") even without keywords.
	PII Redaction	Context-aware removal of Phones, Emails, SSNs, and Names.
	Injection Defense	Prevents "jailbreak" attempts and prompt leakage.
	Secret Detection	Stops API keys and credentials from leaking upstream.
	Profanity Filter	Sanitizes offensive language from inputs.
Connectivity	Multi-Provider	Route to OpenAI, Anthropic, Gemini, Grok, or Local LLMs.
	Local Fallback	Seamlessly integrates with Ollama/LocalAI for private inference.
Observability	Real-time Dashboard	Live view of all traffic, blocks, and latency metrics.
	Audit Logging	SQLite database tracking every request verdict.
	Toast Notifications	Immediate visual feedback when PII is intercepted.
Extensibility	Plugin System	Add Python plugins (e.g., LangKit) for custom logic.
	YAML Configuration	No-code policy management via profile files.
	Custom Builder	Create and test new profiles directly in the UI.

---

---

📸 Screenshots

Security Dashboard - Blocked Request

Compliance Verification

---

🛠️ Quick Start

Option 1: Docker (Recommended)

Getting started is as simple as running a container. We bundle Spacy models and all dependencies.

# 1. Build the image
docker build -t secure-llm-gateway .

# 2. Run (Port 8000)
docker run -p 8000:8000 \
  -e OPENAI_API_KEY=sk-... \
  -e ANTHROPIC_API_KEY=sk-... \
  secure-llm-gateway

Access the dashboard at http://localhost:8000.

Option 2: Local Python Dev

git clone https://github.com/rarenicks/secure-llm-gateway.git
cd secure-llm-gateway

# Setup Venv
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Download Spacy Model for Presidio
python -m spacy download en_core_web_lg

# Run
uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload

---

🧠 Semantic Guardrails (How it Works)

1. Embeddings: We use a local, quantized model (all-MiniLM-L6-v2) to convert user prompts into vector embeddings.
2. Cosine Similarity: The system calculates the distance between the user's prompt and your defined forbidden_intents (e.g., "Violating Safety Protocols").
3. Thresholding: If the similarity score exceeds your configured threshold (e.g., 0.25), the request is blocked.

Example:

• Rule: Block "Money Laundering"
• User Prompt: "How can I clean the cash from my unregistered business?" (No keywords like "laundering" used).
• Result: BLOCKED (Similarity Score: 0.58).

---

🔧 Configuration Guide

Security profiles are YAML files. You can use the built-in profiles (like finance, healthcare) or load your own custom file.

Loading a built-in profile:

engine = GuardrailsFactory.load("finance")

Loading a custom profile:

engine = GuardrailsFactory.load("/path/to/my_policy.yaml")

Example Profile (my_policy.yaml):

profile_name: "Corporate_Policy_v1"
description: "Blocks competitor mentions and protects source code."

detectors:
  pii:
    enabled: true
    engine: "presidio" # or "regex"
  
  injection:
    enabled: true
    
  semantic_blocking:
    enabled: true
    threshold: 0.25
    forbidden_intents:
      - "leaking source code"
      - "disparaging competitors"
      
plugins:
  langkit:
    enabled: true
    modules: ["toxicity", "injections"]

---

🧪 Verification & Testing

We include a suite of tools to verify compliance.

1. Automated Compliance Runner: Run a dataset of attacks against your gateway to generate a pass/fail report.

python tools/compliance_runner.py tests/data/library_attacks.csv

2. Plugin Verification: Ensure LangKit and Presidio are correctly installed.

python tools/verify_enterprise_deps.py

---

📂 Project Structure

• app/: FastAPI Backend & UI Static files.
• sentinel/: The core Python library.
  • engine.py: Core logic combining Regex, Semantics, and Plugins.
  • presidio_adapter.py: Interface to Microsoft Presidio.
  • profiles/: Built-in YAML security definitions (bundled with package).
  • plugins/: Extensible plugin system.
• tools/: Verification scripts and compliance runners.

License

MIT License. Built for the Open Source Community.