director-ai 0.8.2

Coherence Engine — AI output verification and safety oversight

Director AI

Drop-in coherence guardrail for llama.cpp, vLLM, OpenAI, and any LLM.
NLI + vector RAG fact-checking. 4.56 µs scoring. Rust hot path.

---

What It Does

Score any LLM output for hallucination. Halt incoherent token streams in real time. Fact-check responses against your own knowledge base.

from director_ai.core import CoherenceScorer

scorer = CoherenceScorer(threshold=0.6, use_nli=True)
approved, score = scorer.review(
    "What year was the Eiffel Tower built?",
    "The Eiffel Tower was completed in 1889."
)
print(f"Approved: {approved}, Coherence: {score.score:.2f}")
# Approved: True, Coherence: 0.94

Key Metric

Coherence = 1 - (0.6 * H_logical + 0.4 * H_factual)

• H_logical: NLI contradiction probability (DeBERTa-v3-mnli-fever-anli)
• H_factual: RAG ground truth deviation (ChromaDB + sentence-transformers)
• Score < 0.6 = rejected. Score < 0.5 = emergency stop.

Installation

pip install director-ai                    # Lightweight (~5 MB, no torch)
pip install director-ai[nli]               # + DeBERTa NLI model (~2 GB)
pip install director-ai[vector]            # + ChromaDB + sentence-transformers
pip install director-ai[server]            # + FastAPI server
pip install director-ai[nli,vector,server] # Everything

Quick Start

1. Score a response

from director_ai.core import CoherenceScorer

scorer = CoherenceScorer(threshold=0.6, use_nli=True)
approved, score = scorer.review("Capital of France?", "The capital is Paris.")

2. Stream-gate a chatbot

from director_ai.core import CoherenceAgent
from director_ai.integrations import OpenAIProvider

provider = OpenAIProvider(api_key="sk-...")
agent = CoherenceAgent()
session = agent.process_streaming("Explain gravity", provider=provider)

if session.halted:
    print(f"Halted at token {session.halt_index}: {session.halt_reason}")
else:
    print(session.output)

3. Batch-audit Q&A pairs

from director_ai.core import BatchProcessor, CoherenceAgent

agent = CoherenceAgent()
processor = BatchProcessor(agent)
result = processor.process_batch(["What is 2+2?", "Explain DNA"])
print(f"Passed: {result.succeeded}/{result.total}")

4. Ingest your own knowledge base

from director_ai.core import VectorGroundTruthStore, ChromaBackend

store = VectorGroundTruthStore(
    backend=ChromaBackend(persist_directory="./my_facts"),
)
store.ingest([
    "The speed of light is 299,792,458 m/s.",
    "Water boils at 100 degrees Celsius at sea level.",
    "Earth orbits the Sun at 149.6 million km.",
])

# Now score against your facts
from director_ai.core import CoherenceScorer
scorer = CoherenceScorer(threshold=0.6, ground_truth_store=store)
approved, score = scorer.review("How fast is light?", "Light is about 300,000 km/s.")

5. Ingest a directory of documents

from director_ai.core import VectorGroundTruthStore, ChromaBackend

store = VectorGroundTruthStore(
    backend=ChromaBackend(persist_directory="./kb"),
)
n = store.ingest_from_directory("./docs", glob="**/*.md")
print(f"Indexed {n} documents")

scorer = CoherenceScorer(threshold=0.6, ground_truth_store=store)

6. CLI

director-ai review "What color is the sky?" "The sky is blue."
director-ai process "Explain photosynthesis"
director-ai batch input.jsonl --output results.jsonl
director-ai ingest facts.txt --persist ./chroma_data
director-ai serve --port 8080

Integrations

LLM Providers

Built-in adapters with real SSE streaming for OpenAI, Anthropic, HuggingFace, and local servers (llama.cpp, vLLM, Ollama):

from director_ai.integrations import OpenAIProvider, AnthropicProvider, LocalProvider

provider = OpenAIProvider(api_key="sk-...", model="gpt-4o-mini")
for token in provider.stream_generate("Explain gravity"):
    print(token, end="", flush=True)

LangChain

from director_ai.integrations.langchain_callback import CoherenceCallbackHandler

chain = LLMChain(llm=llm, callbacks=[CoherenceCallbackHandler(threshold=0.6)])

Install with: pip install director-ai[langchain]

Architecture

                    ┌─────────────────────────┐
                    │   Coherence Agent        │
                    │   (Main Orchestrator)    │
                    └──────────┬──────────────┘
                               │
              ┌────────────────┼────────────────┐
              │                │                │
    ┌─────────▼──────┐ ┌──────▼──────┐ ┌───────▼────────┐
    │  Generator     │ │ Coherence   │ │  Streaming     │
    │  (LLM Provider │ │ Scorer      │ │  Kernel        │
    │   + Streaming) │ │ (NLI+RAG)   │ │  (Token Gate)  │
    └────────────────┘ └──────┬──────┘ └────────────────┘
                              │
                    ┌─────────▼─────────┐
                    │  Vector Store     │
                    │  (Chroma/Memory)  │
                    └───────────────────┘

Module	Purpose
CoherenceScorer	Dual-entropy scorer: NLI contradiction + RAG fact-check
StreamingKernel	Token-by-token gate: hard limit + sliding window + trend detection
CoherenceAgent	Orchestrator: generate candidates, score, emit best
VectorGroundTruthStore	Semantic retrieval via ChromaDB + sentence-transformers
BatchProcessor	Bulk scoring with concurrency control
OpenAI/Anthropic/HF/LocalProvider	LLM adapters with SSE streaming

Benchmarks

Scored with DeBERTa-v3-base-mnli-fever-anli, threshold 0.5, CPU inference.

Benchmark	Metric	Score
TruthfulQA (MC, n=50)	Accuracy	66.0%
HaluEval QA (n=100)	F1	73.6%
HaluEval Dialogue (n=100)	F1	69.6%
HaluEval Summarization (n=100)	F1	21.1%
HaluEval Overall (n=300)	F1	60.2%

Summarization recall is low — the NLI model struggles to detect hallucinations in paraphrased summaries where sentence structure is preserved. QA and dialogue detection work well.

pip install director-ai[nli]
python -m benchmarks.truthfulqa_eval 50
python -m benchmarks.halueval_eval 100

Performance (Rust Hot Path)

The backfire-kernel/ Rust workspace reimplements safety-critical paths with PyO3 FFI. All operations clear the 50 ms deadline by orders of magnitude:

Operation	Latency	Headroom
Safety kernel (10 tokens)	265 ns	188,679x
Full scoring pipeline	4.56 µs	10,965x
SSGF gradient (analytic Jacobian)	4.14 µs	12,077x

Testing

pytest tests/ -v                 # 397+ tests
pytest tests/test_consumer_api.py -v  # Consumer API only
pytest benchmarks/ -v -m slow    # Benchmarks (requires NLI)
cd backfire-kernel && cargo test  # 153 Rust tests

Research Extensions

Optional SCPN research modules (Lyapunov stability, UPDE integrators, consciousness gate). Zero cross-imports with core.

pip install director-ai[research]

See docs/RESEARCH_GUIDE.md for details.

License

Dual-licensed:

1. Open-Source: GNU AGPL v3.0 — academic, personal, open-source
2. Commercial: ANULUM — closed-source

Citation

@software{sotek2026director,
  author  = {Sotek, Miroslav},
  title   = {Director AI: Coherence Engine},
  year    = {2026},
  url     = {https://github.com/anulum/director-ai},
  version = {0.8.2},
}

Contributing

See CONTRIBUTING.md. By contributing you agree to AGPL v3 terms.