Skip to main content

AI RAG Common Framework

A lightweight, modular, zero-bloat foundation framework for building RAG systems, Model Context Protocol (MCP) servers, autonomous multi-agent workflows, and production AI microservices.

PyPI Python Version Tests Coverage License


AI RAG Common Framework is engineered as a clean, idiomatic, and dependency-light standard library for modern AI engineering. It replaces sprawling, over-engineered stacks with composable, typed primitives that work seamlessly across asynchronous coroutines and synchronous pipelines.

🌟 Core Architectural Pillars

  1. Agnostic LLM Engine & Enterprise Resilience:

    • Unified asynchronous LLM client supporting Google Gemini, OpenAI, Anthropic, Ollama, DeepSeek, Groq, and vLLM.
    • FallbackLLMClient: Automatic circuit-breaking failover across prioritized provider chains.
    • Real-time Server-Sent Events (SSE) token streaming (complete_stream()).
    • SemanticCache: Sub-millisecond similarity caching with LRU eviction and TTL invalidation.
    • Native Pydantic Structured Outputs with schema validation.
  2. Advanced RAG & Embedded Vector Persistence:

    • RAGEngine: End-to-end orchestration of chunking, vectorization, hybrid retrieval, and generation.
    • SQLiteVectorStore: ACID-compliant, single-file embedded vector store using Python's standard library sqlite3 and FTS5 full-text indexing (no external database daemon required).
    • InMemoryVectorStore: High-performance pure-Python vector store with BLAS GEMV acceleration.
    • Hybrid Search: Native Okapi BM25 sparse matching fused with dense vector similarity via Reciprocal Rank Fusion (RRF).
    • Algorithmic Retrieval Strategies: Parallel MultiQueryStrategy expansion and Hypothetical Document Embeddings (HyDEStrategy).
    • Two-Stage Reranking: LocalCrossEncoderReranker and CohereReranker pipeline stages.
  3. Autonomous Agents & Stateful Workflow Orchestration:

    • ReActAgent: Autonomous thought-action-observation reasoning loop with iteration limits.
    • AgentSupervisor: Orchestrator-Workers hierarchy routing subtasks to specialized agents with a shared AgentBus state bus.
    • StateGraph: Stateful directed graph execution engine supporting cyclic and acyclic workflow topologies, conditional branching, execution history, and state checkpointing.
    • ConversationBufferMemory and VectorMemory for contextual recall.
    • Human-in-the-Loop (HITL) approval hooks for sensitive tool execution.
  4. Universal Tooling & Model Context Protocol (MCP 2.x):

    • Universal @tool decorator generating standard JSON schemas via Pydantic introspection.
    • FrameworkMCPServer: Full MCP 2.x server adapter supporting Tools, Resources, and Prompts over stdio and SSE transports.
    • FrameworkMCPClient: Bidirectional MCP client dynamically discovering and mounting remote tools into local agent registries.
    • Single-line export: Any RAG pipeline or interpreter can be converted into an MCP tool via as_tool().
  5. Security, Guardrails & Ephemeral Sandboxing:

    • CodeInterpreter: Secure Python code sandbox running in isolated subprocesses with ephemeral directories, secret scrubbing, and execution timeouts.
    • PromptGuardrail: Fast regex and heuristic detection of prompt injections, jailbreaks, and system prompt override attempts.
    • PIISanitizer: High-throughput redacting and masking of sensitive personal data (emails, credit cards, SSNs, API keys).
    • Strict SSRF validation and path traversal guards across all filesystem persistence.
  6. Observability, Telemetry & Quality Evaluation:

    • TokenCostTracker: Granular dollar accounting tracking prompt and completion tokens across providers.
    • PrometheusRegistry: Zero-dependency metrics router exposing /metrics in standard Prometheus text format.
    • Distributed OpenTelemetry Tracing: W3C traceparent context propagation and trace_span context managers.
    • RAGTriadEvaluator: In-process evaluation of Context Relevance, Groundedness (Faithfulness), and Answer Relevance.
    • HallucinationDetector: Sentence-level grounding confidence scoring without third-party LLM evaluation dependencies.
  7. Developer CLI & Interactive Playground:

    • Unified CLI console command: ai-rag and rag-mcp.
    • ai-rag doctor: Comprehensive diagnostic utility checking Python version, API keys, Ollama reachability, and directory permissions.
    • ai-rag repl: Interactive terminal playground for testing prompts, running tools, ingesting documents, and executing RAG queries live.
    • ai-rag init: Production microservice scaffolding generator.

📦 Installation

From PyPI (Standard Release)

pip install ai-rag-common-framework

Install with optional feature bundles:

# With FastAPI HTTP microservice gateway & Uvicorn
pip install "ai-rag-common-framework[api]"

# With Model Context Protocol (MCP) server & client
pip install "ai-rag-common-framework[mcp]"

# With PostgreSQL / pgvector & NumPy BLAS vector acceleration
pip install "ai-rag-common-framework[rag]"

# Full installation with all optional extensions
pip install "ai-rag-common-framework[all]"

Local Development Installation

Clone the repository and install in editable mode:

git clone https://github.com/buhozurdo/ai-rag-common-framework.git
cd ai-rag-common-framework
pip install -e ".[dev]"

🛠️ Quick Configuration

Create a .env file in your project root (refer to .env.example):

FRAMEWORK_ENV=development
LOG_LEVEL=INFO

# Default LLM Provider (gemini, openai, anthropic, ollama)
DEFAULT_LLM_PROVIDER=gemini
DEFAULT_LLM_MODEL=gemini-2.5-flash

# API Keys
GEMINI_API_KEY=your_gemini_api_key
OPENAI_API_KEY=your_openai_api_key
ANTHROPIC_API_KEY=your_anthropic_api_key

💻 Usage Examples

1. Unified LLM with Structured Outputs & Model Fallbacks

import asyncio
from pydantic import BaseModel, Field
from common_framework import (
    LLMFactory,
    FallbackLLMClient,
    ChatMessage,
    Role,
)

class ArticleSummary(BaseModel):
    title: str
    key_points: list[str] = Field(description="3-5 bullet points")
    sentiment: str = Field(description="positive, neutral, or negative")

async def main():
    # Primary client (OpenAI) with automatic fallback to Gemini
    primary = LLMFactory.create("openai", model="gpt-4o-mini")
    backup = LLMFactory.create("gemini", model="gemini-2.5-flash")
    client = FallbackLLMClient(candidates=[primary, backup])

    messages = [
        ChatMessage(
            role=Role.USER,
            content="Summarize: AI RAG Common Framework delivers modular RAG, MCP tools, and multi-agent workflows."
        )
    ]

    result: ArticleSummary = await client.structured_complete(
        messages=messages,
        response_model=ArticleSummary,
    )
    print(f"Title: {result.title}")
    print(f"Points: {result.key_points}")
    print(f"Sentiment: {result.sentiment}")

if __name__ == "__main__":
    asyncio.run(main())

import asyncio
from common_framework import RAGEngine, SQLiteVectorStore

async def main():
    # Single-file ACID vector store with FTS5 BM25 keyword index
    vector_store = SQLiteVectorStore(database_path="knowledge.db")

    engine = RAGEngine(vector_store=vector_store)

    # Ingest knowledge documents
    await engine.ingest_texts([
        "Authentication is handled via Bearer JWT tokens in the HTTP Authorization header.",
        "Rate limits are enforced using an in-memory token bucket algorithm with 60 req/min.",
        "Database backups execute nightly at 03:00 UTC to an isolated S3 bucket."
    ])

    # Hybrid query combining vector similarity with BM25 full-text rank fusion
    response = await engine.query(
        "How is user authentication implemented?",
        mode="hybrid",
        top_k=2,
    )
    print("Answer:\n", response.answer)

if __name__ == "__main__":
    asyncio.run(main())

3. Multi-Agent Supervisor & Delegation Pattern

import asyncio
from common_framework import AgentSupervisor, LLMFactory

async def researcher(task: str) -> str:
    return f"Research findings: Found 3 relevant CVEs regarding SSL renegotiation."

def coder(task: str) -> str:
    return "Patch generated: SSL_OP_NO_RENEGOTIATION flag enabled in context."

async def main():
    supervisor = AgentSupervisor(
        agents={
            "researcher": ("Security researcher finding vulnerabilities", researcher),
            "coder": ("Senior systems engineer writing security patches", coder),
        },
        llm_client=LLMFactory.create(),
        max_steps=4,
    )

    response = await supervisor.run("Investigate and patch SSL renegotiation vulnerability.")
    print("Final Resolution:\n", response.final_answer)
    for step in response.delegations:
        print(f"Step {step.step}: [{step.agent_name}] -> {step.result[:60]}...")

if __name__ == "__main__":
    asyncio.run(main())

4. Stateful Directed Graph Workflow (StateGraph)

import asyncio
from common_framework import StateGraph, START, END

def fetch_data(state: dict) -> dict:
    return {"query": state.get("query"), "raw_text": "Sample financial report data."}

def analyze_risk(state: dict) -> dict:
    risk_score = 75 if "financial" in state["raw_text"] else 20
    return {"risk_score": risk_score}

def router(state: dict) -> str:
    return "high_risk" if state["risk_score"] > 50 else "low_risk"

def flag_review(state: dict) -> dict:
    return {"status": "Escalated for manual compliance review"}

def auto_approve(state: dict) -> dict:
    return {"status": "Auto-approved"}

async def main():
    graph = (
        StateGraph()
        .add_node("fetch", fetch_data)
        .add_node("analyze", analyze_risk)
        .add_node("flag_review", flag_review)
        .add_node("auto_approve", auto_approve)
        .add_edge(START, "fetch")
        .add_edge("fetch", "analyze")
        .add_conditional_edges("analyze", router, {
            "high_risk": "flag_review",
            "low_risk": "auto_approve",
        })
        .add_edge("flag_review", END)
        .add_edge("auto_approve", END)
    )

    result = await graph.run({"query": "Audit Q3 numbers"})
    print("Workflow Completed:", result.completed)
    print("Final State:", result.state)

if __name__ == "__main__":
    asyncio.run(main())

5. Sandboxed Python Code Interpreter

import asyncio
from common_framework import CodeInterpreter, code_interpreter_tool

async def main():
    # Ephemeral sandbox stripping parent secrets and enforcing 5s timeout
    interpreter = CodeInterpreter(timeout=5.0)

    code = """
import statistics
data = [10.2, 14.5, 18.9, 22.1, 35.6]
mean = statistics.mean(data)
stdev = statistics.stdev(data)
print(f"Mean: {mean:.2f}, Stdev: {stdev:.2f}")
"""
    result = await interpreter.run(code)
    print("Execution Success:", result.success)
    print("Output:\n", result.stdout)

if __name__ == "__main__":
    asyncio.run(main())

6. Production FastAPI Microservice with RAG & Authenticated MCP

from common_framework import RAGEngine, create_app, create_mcp_server, tool

# 1. Initialize RAG Engine and MCP Server
rag = RAGEngine()
mcp = create_mcp_server(name="gateway-mcp")

@mcp.tool(name="status", description="Get cluster health status")
def get_status() -> dict[str, str]:
    return {"cluster": "us-east-1", "health": "green"}

# 2. Create production FastAPI service with unified endpoints
app = create_app(
    title="AI Knowledge Gateway",
    enable_rag_routes=True,     # Exposes /rag/query, /rag/retrieve, /rag/ingest, /rag/collections
    rag_engine=rag,
    mcp_server=mcp,             # Mounts MCP server over SSE (/mcp/sse, /mcp/messages/)
    mcp_prefix="/mcp",
    mcp_api_key="secret-token", # Enforces Bearer / X-API-Key / ?api_key= auth
)

# Run with: uvicorn main:app --host 0.0.0.0 --port 8000

7. Unified Framework CLI

The package provides the ai-rag command-line interface out of the box:

# Run environment diagnostics and health checks
ai-rag doctor

# Output diagnostics in JSON format for CI/CD pipelines
ai-rag doctor --json

# Launch the interactive terminal REPL playground
ai-rag repl

# Scaffold a new starter microservice project
ai-rag init --dir ./my_ai_service

# Run a standalone MCP server over stdio or SSE
ai-rag run-mcp --transport stdio
ai-rag run-mcp --transport sse --port 8080

# Check installed framework version
ai-rag version

🧪 Running Tests & Quality Verification

The test harness executes 331 tests (100% passing) with 91% global line coverage:

  • 312 unit and regression tests across 29 focused test modules in tests/
  • 19 enterprise end-to-end integration tests across 5 test suites in examples/enterprise/
# Run all unit tests
python -m pytest tests/ -v

# Run full test suite with enterprise integration suites and coverage
python -m pytest tests/ examples/ --cov=common_framework

📚 Documentation & Roadmap

  • CHANGELOG.md: Complete release history, Pull Request notes, and breaking changes.
  • RELEASE.md: Gitflow release process, tagging schemes, and tag-by-tag release notes.
  • ROADMAP.md: Release train progression and future architectural milestones.

📄 License

This project is licensed under the MIT License.

Metadata

Release files for ai-rag-common-framework 0.19.8

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for ai-rag-common-framework 0.19.8
File Size Uploaded Provenance Actions
ai_rag_common_framework-0.19.8.tar.gz 264.0 kB With provenance Details for ai_rag_common_framework-0.19.8.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for ai-rag-common-framework 0.19.8
File Interpreter ABI Platform Actions
ai_rag_common_framework-0.19.8-py3-none-any.whl Python 3 none any Details for ai_rag_common_framework-0.19.8-py3-none-any.whl

Total release size: 408.0 kB

Release files / ai_rag_common_framework-0.19.8.tar.gz

Download URL ai_rag_common_framework-0.19.8.tar.gz
Size 264.0 kB
Tags Source
SHA-256 checksum
How to use checksums
c218fc960d5ff592924f315a2a3db02a64b483284812007dad6589ee91ee4a16
BLAKE2b-256 checksum
How to use checksums
ad623e630d1a98e2862de59b2412c3c9005a7f59f50e1eaac4f3b41428b3381a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 20, 2026.

Transparency log

Release files / ai_rag_common_framework-0.19.8-py3-none-any.whl

Download URL ai_rag_common_framework-0.19.8-py3-none-any.whl
Size 144.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
1b7d3fabbdad59160b8d46fd9b52f792d570ac6512d5846338a14fe7a7579a87
BLAKE2b-256 checksum
How to use checksums
792c9a0d24d2ee6e490eb7dcc3cc7aa1a22488efe56d99078203162020f44009
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 20, 2026.

Transparency log

Security for ai-rag-common-framework 0.19.8

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for ai_rag_common_framework-0.19.8.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 20, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.19.8 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page