NomosGuard

LLM produces evidence. Deterministic machinery decides.

NomosGuard is a deterministic security reasoning core: an append-only evidence ledger, a rule engine that derives threat/access paths from that evidence, and a fail-closed policy gate. Language models may read events and write structured claims with cited evidence, and may later explain decisions in human language — but a model never crosses the decision boundary. Every claim without reproducible evidence is discarded; every decision is hash-chained; the gate defaults to deny.

The name: nomos (νόμος) — Greek for rational law, order derived from reason rather than brute force (physis) — plus guard, the fail-closed gate that enforces it. Together: reasoning that decides, evidence that must prove itself.

Why deterministic core + probabilistic edge

A language model's confidence is not evidence — it is an assertion. This project treats it as such:

• The LLM edge (top): reads raw events, configuration, CVE records, logs; emits structured claims (entities, relations, vulnerabilities, tool calls), each citing the exact evidence fragment (log line hash, config path, CVE ID) that supports it.
• The deterministic core: an append-only, SHA-256 hash-chained ledger stores every claim; a rule engine derives attack/access paths (MulVAL-style logical inference — pure logic, no randomness, no model); a policy gate decides ALLOW / ALERT / BLOCK, fail-closed.
• The LLM edge (bottom): explains the decision, drafts remediation suggestions, and proposes next analysis steps. It narrates; it never decides. If the explanation contradicts the evidence chain, the chain wins.

This is the same separation that makes a court work: a lawyer (the model) presents evidence, but the verdict comes from the rules of law (the core), and the transcript is tamper-evident.

Status

Early development — see VISION.md for the architecture, scope, and first-release boundary. The core (evidence ledger, rule engine, policy gate) is implemented with 51 passing tests and a reproducible demo scenario, real log ingestion, and a measured benchmark corpus.

Components

Evidence production pipeline

The LLM edge is where hallucinations live — every model in the 16-model benchmark invented at least one claim. nomosguard.benchmark.model_compare.pipeline wraps model output in four stages, ordered by how much they can be trusted:

1. SAMPLE   N independent draws from the model
2. VOTE     keep a claim only if >= M draws produced it
            (one-off claims are likely hallucinations)
3. AUDIT    a second prompt asks the model to FLAG suspicious claims
            — it never deletes them (measured: deletion cost recall)
4. GROUND   deterministic: a claim survives only if its evidence
            field is a verbatim substring of the raw logs

Only stage 4 removes claims by hard rule. Stage 3 attaches suspicion as metadata (result.flagged); the claim survives so no correct evidence is ever silently lost. A first measured pass (step-5-preview-free, 4 scenarios) showed the pipeline trading −9 recall points for +20 precision points against single-pass extraction — the flag-not-delete audit design exists to close that recall gap.

from nomosguard.benchmark.model_compare.pipeline import run_pipeline

result = run_pipeline(model_fn, raw_logs, draws=3, min_votes=2)
# result.claims  — surviving claims, each with vote support
# result.flagged — suspicious claims, kept but marked
# result.report  — per-stage removals, for measurement

Benchmark (measured, not asserted)

Run with python -m nomosguard.benchmark.run_suite_main --output DIR. The corpus is committed and deterministic; the expected outcomes are hand-derived from the facts, not produced by the engine. Latest run:

What the corpus does NOT cover (stated honestly): real-world logs, evidence tampering inside the ledger (that is the ledger's own test suite's job), and any scenario requiring an LLM to judge.

Ingestion (three evidence types, one discipline)

Three adapters, each with the same contract: every claim cites sha256:<hex> of the raw source line; malformed records are rejected with line number and reason (fail-closed); duplicate lines are counted, never re-appended.

Committed samples: examples/sample_toolcalls.jsonl, examples/sample_cves.jsonl, examples/sample_policies.jsonl.

Run it

pip install -e .
python -m nomosguard.demo    # committed scenario: derivation + BLOCK
pytest tests/                # 107 tests incl. security suite
python -m nomosguard.benchmark.run_suite_main          # tool-call corpus (11 scenarios)
python -m nomosguard.benchmark.security_main           # security corpus (5 scenarios)

Use it from an MCP client

{"command": "python", "args": ["-m", "nomosguard.mcp_stdio"]}

The server never calls a model. Agents load SKILL.md for the behavioral contract: assert claims with evidence, derive paths, read the gate decision, explain it — never decide it.

License

Apache-2.0