staleguard 0.1.1

Post-retrieval audit layer for RAG pipelines.

StaleGuard

StaleGuard is a post-retrieval audit layer for RAG pipelines.

It sits between retrieval and generation:

Retriever / Vector DB
        ->
Retrieved chunks
        ->
StaleGuard audit
        ->
Decision:
  FRESH
  STALE
  MIXED
  CONFLICTED
  UNKNOWN
        ->
LLM

StaleGuard does not replace retrieval. It checks whether retrieved chunks are trustworthy enough to send to the model.

Install

pip install staleguard

For local embedding and NLI models:

pip install "staleguard[local]"

For the repo demos:

pip install "staleguard[demo]"

What It Does

• score chunk freshness from metadata, version, and date signals
• find fresher alternatives from a corpus
• detect contradictions with rules and optional local NLI
• surface schema issues when retrieved metadata is incomplete
• return a provenance object you can use in middleware or UI

Current trust order:

metadata > rules = nli

That means:

• explicit metadata wins when it is available
• rules and NLI are secondary evidence sources
• if rules and NLI agree, confidence increases

Core API

The shortest path is the package-level audit(...) function.

from staleguard import audit

result = audit(
    query="How do I configure Redis Cluster in Redis 8?",
    retrieved=retriever_output,
    corpus=corpus,
)

print(result.verdict)
print(result.conflicts)
print(result.provenance)

If you want to reuse configuration across calls, use StaleGuard.

from staleguard import StaleGuard

guard = StaleGuard(use_nli=True, block_on_conflict=False)
result = guard.audit(
    query="How do I configure Redis Cluster in Redis 8?",
    retrieved=retriever_output,
    corpus=corpus,
)

For advanced integrations, the same API also accepts:

• already-normalized chunk dicts
• raw Chroma query results
• LangChain-style documents
• custom embedding / conflict providers

Normalized chunks

from staleguard import StaleGuard

guard = StaleGuard(use_nli=True)

result = guard.audit_chunks(
    query="How do I configure Redis Cluster in Redis 8?",
    retrieved_chunks=[
        {
            "id": "redis_6_cluster_001",
            "text": "Redis 6 uses requirepass configuration for cluster authentication.",
            "product": "redis",
            "topic": "cluster_configuration",
            "version": "6.2",
            "date_ts": 1640995200,
            "source": "redis-6.2-cluster.md",
            "metadata": {"status": "superseded", "superseded_by": "8.0"},
        }
    ],
    corpus=[...],
)

audit(...) and guard.audit(...) auto-detect:

• raw Chroma query results
• LangChain-style documents with page_content and metadata
• already-normalized chunk dicts

The lower-level helpers still exist for direct use:

• audit_retrieved(...)
• audit_chroma_result(...)
• audit_langchain_docs(...)

Verdicts

• FRESH: retrieved context looks current
• STALE: outdated chunks were found
• MIXED: stale and conflicting evidence were found
• CONFLICTED: conflicting evidence was found
• UNKNOWN: metadata is too incomplete to make a strong judgment

If you want any conflict to block generation, set:

block_on_conflict=True

That collapses MIXED into CONFLICTED.

Middleware Pattern

This is the intended integration shape:

from staleguard import StaleGuard

guard = StaleGuard(use_nli=True, block_on_conflict=False)

def audited_retrieve(query: str, retriever_output, corpus: list[dict]):
    audit_result = guard.audit(
        query=query,
        retrieved=retriever_output,
        corpus=corpus,
    )

    if audit_result.verdict == "CONFLICTED":
        return {"block": True, "audit": audit_result}

    return {"block": False, "audit": audit_result}

The application can then:

• send FRESH chunks to the LLM
• replace or warn on STALE
• block or escalate on CONFLICTED
• show provenance on MIXED

Supported Retrieval Inputs

Chroma

from staleguard import StaleGuard

guard = StaleGuard(use_nli=True)

result = guard.audit(
    query=query,
    retrieved=chroma_result,
    corpus=corpus,
)

LangChain-style documents

from staleguard import StaleGuard

guard = StaleGuard(use_nli=True)

result = guard.audit(
    query=query,
    retrieved=docs,
    corpus=corpus,
)

The repo also exposes adapter helpers:

• normalize_chroma_result(...)
• normalize_langchain_docs(...)
• normalize_chunks(...)

CLI

staleguard audit --query "How do I configure Redis Cluster in Redis 8?" --retrieved retrieved.json --corpus corpus.json
staleguard eval --corpus eval_cases/kubernetes/corpus.json --cases eval_cases/kubernetes/cases.json --use-nli

Chunk Schema

Best case input:

{
    "id": "redis_8_cluster_001",
    "text": "...",
    "product": "redis",
    "topic": "cluster_configuration",
    "version": "8.0",
    "date_ts": 1735689600,
    "source": "redis-8.0-cluster.md",
    "metadata": {
        "status": "current"
    }
}

Audit-critical fields:

• text
• product
• topic
• version
• date_ts

If some metadata is missing, StaleGuard will:

• infer a few safe fields from source
• record schema_issues
• lower confidence or return UNKNOWN when needed

Demos

Redis demo

python -m staleguard.chroma_demo

This builds a local Chroma collection and shows:

• raw Chroma retrieval
• normalized chunks
• prepared chunks
• audit result

Large engineering demo

python -m staleguard.engineering_demo

This uses a larger corpus around:

• loop engineering
• context engineering
• tool loop policy
• memory policy
• planning
• retrieval policy

The example intentionally includes:

• stale 2024/2025 chunks
• fresher 2026 replacements
• conflicting guidance that triggers NLI

License

MIT