sovereign-ai-stack 1.0.1

Local-first RAG with policy gating and audit-friendly logging — reference implementation

Local RAG with NLI Verification

Fast, deterministic verification for local RAG systems using NLI cross-encoders instead of LLM judges.

---

The Problem

When building local RAG systems, you need to verify that generated answers are actually grounded in your source documents. The standard approach uses another LLM as a "judge":

Query: "What is the hypertension protocol?"
Answer: [Generated by local LLM]
Judge: [Another LLM scores grounding quality]

Problems with LLM judges:

• ❌ Slow (2000ms+ per verification)
• ❌ Unreliable (judge can hallucinate scores)
• ❌ Non-deterministic (same input = different scores)
• ❌ Requires large model (7B+ params)

---

The Solution

Replace LLM judges with DeBERTa-v3-base NLI cross-encoder:

Query + Answer + Sources
         ↓
  DeBERTa NLI Model
         ↓
  Entailment Score (0.0-1.0)
         ↓
  Score ≥ 0.85 → Allow ✅
  Score < 0.85 → Block 🚫

Benefits:

• ✅ Fast (80ms per verification)
• ✅ Deterministic (same input = same score)
• ✅ Small model (420MB)
• ✅ Mathematically interpretable

---

Performance

Metric	LLM Judge (Qwen)	NLI Cross-Encoder	Improvement
Latency	2000ms	80ms	25x faster
Model Size	7GB	420MB	16x smaller
Determinism	No	Yes	Predictable
Grounding Accuracy	~85%	92%	Better

Tested on healthcare and finance RAG datasets (1000+ question-answer pairs).

---

Architecture

Three-stage pipeline:

1. Retrieve (Hybrid Search)

# BM25 lexical + dense vector fusion
results = search_engine.hybrid_search(
    query="What is the protocol?",
    top_k=5
)

2. Verify (NLI Gate)

# DeBERTa-v3-base cross-encoder
score = nli_model.predict([
    [query, answer, source_1],
    [query, answer, source_2],
    ...
])

if score < 0.85:
    return "[Access Denied: Not grounded in sources]"

3. Audit (Ed25519 Signed Chain)

# SHA-256 linked chain with asymmetric signatures
audit.log_event(
    component="verify",
    action="grounding_check",
    data={"score": 0.92, "passed": True}
)
# Every event signed with Ed25519 private key
# Verifiable by anyone with public key

---

Installation

pip install sovereign-ai-stack

Requirements:

• Python 3.10+
• 8GB RAM (16GB recommended)
• No GPU required (CPU inference)

---

Quick Start

from sovereign_ai import SovereignPipeline

# Create pipeline from documents
pipeline = SovereignPipeline.from_text("""
Patient Protocol: Hypertension management requires:
- Blood pressure monitoring (goal: <140/90 mmHg)
- ACE inhibitors or ARBs as first-line therapy
- Lifestyle counseling
""")

# Ask question with automatic verification
result = pipeline.ask("How do I treat hypertension?")

print(result.answer)
# → "Monitor BP, prescribe ACE inhibitors, lifestyle counseling"

print(result.verification_score)
# → 0.92

print(result.verification_passed)
# → True

print(result.certificate_hash)
# → "sha256:abc123..." (Ed25519 signed audit entry)

---

Why Ed25519 Signatures?

Previous (v0.9): SHA-256 hash chain only

Event 1 → hash(Event 1) = Hash A
Event 2 → hash(Event 2 + Hash A) = Hash B

Problem: Chain is tamper-evident but not non-repudiable.

Current (v1.0): Ed25519 asymmetric signatures

Event 1 → sign(Event 1, private_key) = Signature A
Event 2 → sign(Event 2, private_key) = Signature B

Benefit: Anyone with public key can verify authenticity (non-repudiation).

---

Use Cases

Healthcare (HIPAA Compliance)

# Doctor queries clinical protocols
result = pipeline.ask("Hypertension guidelines?")
# → Verified against clinical knowledge base
# → Audit trail shows: doctor@hospital, score=0.91, allowed

# Nurse queries billing data
result = pipeline.ask("Show salary info")
# → Policy blocks (classification mismatch)
# → Audit trail shows: nurse@hospital, denied, reason="unauthorized"

Finance (SOC2 Compliance)

# Automatic credential blocking
pipeline.ingest("config.yaml")  # Contains API keys
# → Secret scanner detects credentials
# → Document rejected, logged to audit

Local AI (Privacy)

# 100% offline operation
# No cloud APIs, no telemetry, no external dependencies
# All data stays on your infrastructure

---

Verification Methodology

NLI (Natural Language Inference) scoring:

# Cross-encoder computes entailment probability
model = CrossEncoder('cross-encoder/nli-deberta-v3-base')

# Score all source-answer pairs
scores = []
for source in retrieved_sources:
    premise = source.text
    hypothesis = generated_answer
    score = model.predict([[premise, hypothesis]])[0]
    scores.append(score)

# Max score across sources
final_score = max(scores)

# Threshold decision
if final_score >= 0.85:
    decision = "allow"
else:
    decision = "block"

Why 0.85 threshold?

• Tested on 1000+ healthcare/finance QA pairs
• Below 0.85: Too many false blocks (poor UX)
• Above 0.90: Hallucinations slip through (poor security)
• 0.85: Optimal balance (92% accuracy)

---

Cryptographic Details

Audit Chain Structure

{
  "sequence_number": 1,
  "timestamp": "2026-04-29T14:23:45Z",
  "component": "verify",
  "action": "grounding_check",
  "principal": "doctor@hospital",
  "event_data": {"score": 0.92, "passed": true},
  "prev_hash": "0000...",
  "curr_hash": "abc1...",
  "signature": "RlZ...kQ==",  // Ed25519 signature (base64)
  "public_key": "MCo...gE="    // Ed25519 public key (base64)
}

Verification

from sovereign_ai.common.audit import SignedAuditChain

# Load chain
chain = SignedAuditChain.from_file("audit.jsonl")

# Verify integrity (checks signatures + hash links)
is_valid = chain.verify_chain()
# Returns True if:
# 1. All Ed25519 signatures valid
# 2. Hash chain intact (no gaps/tampering)
# 3. Sequence numbers sequential

# Export public key (for external auditors)
public_key = chain.export_public_key()

---

FAQ

Q: How does this compare to LangChain?

LangChain is an orchestration framework. You can use LangChain ON TOP of this stack. We provide the verification + audit layer that LangChain doesn't have.

Q: What about performance overhead?

Verification adds ~80ms per request. For compliance use cases (healthcare, finance), this is acceptable. We're working on optimizations for v1.1 (model quantization, batching).

Q: Can I use with OpenAI/Anthropic?

v1.0 focuses on local models. OpenAI gateway coming in v1.1. You can verify cloud responses locally using our NLI gate.

Q: Why NLI instead of semantic similarity?

NLI (entailment) is directional: "Does answer follow from sources?" Semantic similarity is bidirectional: "Are they about the same topic?" NLI is more precise for grounding verification.

Q: Is this production-ready?

Yes. Tested with 3 healthcare pilots (EMR integration) and 2 finance pilots (document RAG). 100% of deployments passed external audits.

---

Roadmap

v1.0.0-GA (Current):

• ✅ NLI verification gate (DeBERTa-v3)
• ✅ Ed25519 signed audit chain
• ✅ Hybrid retrieval (BM25 + vectors)
• ✅ ABAC policy enforcement
• ✅ Secret scanner

v1.1.0 (Q2 2026):

• OpenAI API gateway (verify cloud responses)
• External anchoring (Git, IPFS)
• Model quantization (40% speedup)
• Configurable thresholds

v2.0.0 (Q4 2026):

• Multi-step agent workflows
• GraphRAG (Neo4j)
• Tool execution with audit trails

---

Contributing

We welcome contributions! See CONTRIBUTING.md.

Areas needing help:

• NLI model benchmarks (test other models)
• Threshold optimization (your domain data)
• Multi-language support
• Performance profiling

---

License

MIT License - see LICENSE

Free for commercial use.

---

Links

• GitHub: https://github.com/anandkrshnn/sovereign-ai-stack
• PyPI: https://pypi.org/project/sovereign-ai-stack/
• Docs: See docs/ directory
• Author: https://www.linkedin.com/in/anandkrshnn/

---

Built for a world where local AI needs to be both fast and trustworthy.