rag-pathology 0.1.0

Diagnose RAG pipeline failures by type and location. Four Soils classification. Epistemic mismatch detection.

rag-pathology

RAGAS gives you a score. rag-pathology tells you what's wrong.

Diagnoses RAG pipeline failures by type and location. Four Soils classification. Epistemic mismatch detection. Zero dependencies.

pip install rag-pathology

The Problem

Your RAG pipeline scores 0.6 on RAGAS. Now what? Is it a retrieval problem? A generation problem? Are you retrieving facts when the user asked for procedures? RAGAS won't tell you. Neither will DeepEval, TruLens, or Promptfoo.

rag-pathology diagnoses the specific pathology at each stage of your pipeline.

Four Soils Classification

Every query is classified into one of four failure types (inspired by Mark 4:3-8):

Soil	Meaning	Fix
PATH	Total retrieval miss — relevant docs exist but weren't retrieved	Fix embeddings, chunk size, or query expansion
ROCKY	Good retrieval but generation ignores the context	Strengthen grounding prompt, add citations
THORNY	Good retrieval + generation, but noisy context corrupts output	Add reranking, reduce top-k
GOOD	Successful RAG	No action needed

Quick Start

from rag_pathology import RAGDiagnoser, RAGQuery, Chunk

diagnoser = RAGDiagnoser("my_pipeline")

query = RAGQuery(
    query="What is Ghana's GDP growth rate?",
    retrieved_chunks=[
        Chunk("Ghana GDP growth is 6.0% in 2025", score=0.95),
        Chunk("Recipe for jollof rice", score=0.1),
    ],
    generated_answer="Ghana's GDP growth rate is 6.0%.",
)

diagnosis = diagnoser.diagnose_query(query)
print(diagnosis.soil_type)        # SoilType.GOOD
print(diagnosis.failure_stage)    # FailureStage.NONE
print(diagnosis.evidence)         # "Healthy RAG: relevance=0.45, grounding=0.67..."

# Pipeline-level diagnosis
pipeline = diagnoser.pipeline_diagnosis()
print(pipeline.summary())
print(pipeline.overall_health)    # 0.67
print(pipeline.recommendations)   # ["33% of queries are THORNY..."]

Epistemic Mismatch Detection

Detects when your retrieval returns the wrong type of knowledge:

• User asks "How do I register a company?" (PROCEDURAL)
• RAG retrieves "Companies in Ghana must register with RGD" (FACTUAL)
• Structurally relevant, epistemically wrong type

License

MIT