ragpy-core 1.0.0

A modular Retrieval-Augmented Generation (RAG) pipeline for Python.

RAGpy RAGpy is a lightweight, modular Retrieval-Augmented Generation (RAG) pipeline for Python. It provides a clear and testable architecture for document ingestion, chunking, embedding, retrieval, reranking, context compression, and grounded answer generation using Azure OpenAI and ChromaDB.

RAGpy is designed for developers who want a transparent, hackable RAG system without the complexity of large frameworks.

Features Modular ingestion pipeline for text and PDF documents

Chunking and batching utilities for efficient embedding

Azure OpenAI embeddings and chat completions

ChromaDB vector database integration

LLM-based reranking for improved retrieval quality

Context compression to reduce token usage

Fully monkeypatch-friendly design for offline testing

Clean architecture suitable for extension and customization

Installation Once published to PyPI:

Code pip install ragpy For development:

Code git clone https://github.com/yourusername/ragpy cd ragpy pip install -e . Quickstart Example python from ragpy.RAGOrchestrator import IngestFile, GenerateAnswer from ragpy.VectorDatabase import OpenDatabase

OpenDatabase("AeroDB", "./vectorDB") IngestFile("engine_vibration.pdf", "AeroDB")

answer = GenerateAnswer("What causes engine vibration?", "AeroDB") print(answer) How RAGpy Works

1. Ingestion Load text or PDF using FileLoader

Chunk text using TextChunker

Batch chunks using ChunkBatcher

Generate embeddings with Azure OpenAI

Store vectors and metadata in ChromaDB

2. Retrieval Embed the user query

Retrieve top-K candidates from the vector database

3. Reranking Use an LLM-based reranker to reorder retrieved chunks by relevance

4. Compression Summarize top chunks into a compact context block

5. Answer Generation Build a prompt using compressed context

Generate a grounded answer using Azure OpenAI

Project Structure Code ragpy/ AzureOpenAIRelay.py RAGOrchestrator.py VectorDatabase.py Reranker.py ChunkCompressor.py loaders/ FileLoader.py TextChunker.py batching/ ChunkBatcher.py tests/ docs/ Requirements Python 3.9+

ChromaDB

numpy

tiktoken

pypdf

openai (Azure OpenAI SDK)

Testing RAGpy includes a full pytest suite. All Azure calls are monkeypatch-friendly, allowing offline testing with mock LLMs.

Run tests:

Code pytest -q Contributing Contributions are welcome. Please open an issue or submit a pull request on GitHub.

Planned enhancements include:

Local embedding support (sentence-transformers)

Hybrid retrieval (vector + keyword)

Multimodal RAG (image + text)

Evaluation tools for relevance and faithfulness

Agentic RAG extensions

License RAGpy is released under the MIT License.