RAGKit

A modular Retrieval-Augmented Generation (RAG) library for Python. RAGKit ingests your documents, indexes them for both dense (vector) and keyword (BM25) search, retrieves and reranks the most relevant chunks for a question, and generates a grounded, citation-backed answer — through an API that's simple by default and fully customizable when you need it.

import os
from ragkit import RAG

rag = RAG(
    document="document.pdf",
    base_url="https://openrouter.ai/api/v1/chat/completions",
    model="google/gemma-4-31b-it:free",
    api_key=os.getenv("OPENROUTER_API_KEY"),
)

response = rag.query("What is this document about?")
print(response["choices"][0]["message"]["content"])

RAG() handles document loading, chunking, embeddings, vector indexing, retrieval, reranking, context construction, and LLM generation internally — you only need to provide a document and an LLM configuration for the basic use case. See Usage below for every supported pattern.

Features

• Simple by default, powerful when needed — one call for basic use, direct keyword arguments for tuning, full component injection for advanced customization.
• PDF, TXT, Markdown, and DOCX ingestion, with accurate per-chunk provenance (document, page).
• Hybrid retrieval — dense vector search fused with BM25 keyword search (RRF or weighted), persisted across process restarts.
• Cross-encoder reranking for more accurate result ordering.
• OpenAI-compatible LLMs — OpenRouter, Groq, Gemini's compat layer, self-hosted servers, or real OpenAI — plus a local, zero-API-cost path via Ollama.
• Local embeddings via sentence-transformers — no embedding API key needed.
• Source-aware, OpenAI-compatible responses — response["choices"][0]["message"]["content"] and typed response.sources read the same data.
• Persistent local vector index (embedded Qdrant, no server required) or a real Qdrant server for concurrent/production workloads.
• Typed errors (RAGKitError hierarchy) with actionable messages instead of raw stack traces.

Installation

Requires Python 3.10–3.12.

pip install ragkit-rag

from ragkit import RAG

The PyPI distribution is named ragkit-rag (ragkit was already taken by an unrelated project), but the importable package is ragkit. For local development:

git clone <this-repo>
cd rag
python -m venv .venv && .venv\Scripts\activate      # macOS/Linux: source .venv/bin/activate
pip install -e ".[dev]"

No separate services are required to get started: the vector store (Qdrant) runs embedded in-process, and embeddings/reranking run locally via sentence-transformers. Only LLM generation needs an API key (or Ollama, for zero API cost — see 6. Local/Ollama).

Usage

Every supported way of writing RAGKit code lives in this section.

1. Basic RAG

The absolute minimum: a document and an LLM configuration.

from ragkit import RAG

rag = RAG(
    document="document.pdf",          # also accepts file:// URIs, incl. paths with spaces
    model="google/gemma-4-31b-it:free",
    api_key="YOUR_API_KEY",
)

response = rag.query("What is this document about?")
print(response["choices"][0]["message"]["content"])

# Same data, typed:
print(response.answer)

RAG(document=...) loads, cleans, chunks, embeds, and indexes the document immediately — .query() is ready to call right after construction. document= also accepts a list of paths to ingest several files at once, or can be omitted and passed later via rag.ingest(path).

2. OpenRouter

Any OpenAI-Chat-Completions-shaped endpoint works via base_url — no adapter class needed. RAGKit normalizes the URL internally, so both the bare API base and the full .../chat/completions form some provider docs show work identically:

base_url="https://openrouter.ai/api/v1"                     # both
base_url="https://openrouter.ai/api/v1/chat/completions"     # forms work

Production-style example, reading the key from an environment variable:

import os
from ragkit import RAG

rag = RAG(
    document="document.pdf",
    base_url="https://openrouter.ai/api/v1/chat/completions",
    model="google/gemma-4-31b-it:free",   # any OpenRouter model works - see https://openrouter.ai/models
    api_key=os.getenv("OPENROUTER_API_KEY"),
)

response = rag.query("What is this document about?")
print(response["choices"][0]["message"]["content"])

Set the key before running:

export OPENROUTER_API_KEY="sk-or-..."          # macOS/Linux
$env:OPENROUTER_API_KEY = "sk-or-..."          # Windows PowerShell

Free-tier models on OpenRouter share a rate-limited pool and their catalog changes over time — a 429 means retry shortly, and a 404 means the slug no longer exists; check openrouter.ai/models for current options.

3. Query with Sources

RAG.query() returns both the generated answer and the retrieved evidence behind it:

response = rag.query("What is the refund policy?")

print(response["choices"][0]["message"]["content"])   # OpenAI-compatible

for source in response.sources:            # typed: .document, .page, .chunk_id, .retrieval_score, .text
    print(source)

response["sources"]                        # same data as plain dicts
response.metadata["latency"]               # per-stage timing: retrieval, reranking, generation, total

4. Custom Retrieval

Every stage is an interface (EmbeddingProvider, VectorStore, Retriever, Reranker, QueryRewriter, LLMProvider, Chunker) — swap one in by passing an instance to the same constructor, without subclassing RAG or touching internals:

from ragkit import RAG
from ragkit.retrieval.reranker import Reranker

class MyReranker(Reranker):
    def rerank(self, query, results):
        ...  # your logic

rag = RAG(document="document.pdf", api_key="YOUR_API_KEY", reranker=MyReranker())

5. Advanced Configuration

Common tuning knobs are direct keyword arguments — no manual component construction required:

from ragkit import RAG

rag = RAG(
    document="document.pdf",
    model="google/gemma-4-31b-it:free",
    api_key="YOUR_API_KEY",
    chunk_size=1000,        # characters per chunk
    chunk_overlap=150,      # characters of overlap carried between chunks
    top_k=10,                # chunks included in the final answer's context
    rerank=True,             # cross-encoder rerank of retrieved candidates
    retrieval="hybrid",      # retrieval strategy - see below
    persist_directory="./my_index",   # where the local vector index is stored
)

Supported retrieval modes:

retrieval="hybrid"    # dense vector + BM25 keyword search, fused (default)
retrieval="vector"    # dense vector search only
retrieval="keyword"   # BM25 keyword search only

Anything not exposed as a direct constructor argument is still reachable via settings=:

from ragkit import RAG, Settings
rag = RAG(settings=Settings(hybrid_fusion_strategy="weighted", hybrid_alpha=0.7))

6. Local/Ollama

Run generation on your own machine for zero API cost via Ollama — retrieval already runs locally (embeddings, reranking, vector store), so this makes the whole pipeline offline:

ollama pull llama3.2:3b

from ragkit import RAG, Settings

rag = RAG(settings=Settings(llm_provider="ollama", ollama_model="llama3.2:3b"))
rag.ingest("document.pdf")
response = rag.query("What is this document about?")

Or set LLM_PROVIDER=ollama / OLLAMA_MODEL=llama3.2:3b in .env and just call RAG(). If Ollama isn't running or the model isn't pulled, you get an actionable ConfigurationError, not a generic connection error.

How It Works

Document
   |
   v
Loader              .pdf / .txt / .md / .docx -> Document
   |
   v
Chunker             RecursiveCharacterChunker -> list[Chunk]
   |
   v
EmbeddingProvider    SentenceTransformer (local) -> vectors
   |
   v
VectorStore          Qdrant (embedded local, or a real server)
   |
   +----------------------+
   |                       |
   v                       v
VectorRetriever        BM25Retriever
   |                       |
   +----------+------------+
              v
       HybridRetriever (RRF or weighted fusion)
              |
              v
       Reranker (optional, cross-encoder)
              |
              v
       ContextBuilder (dedup, limit, cite)
              |
              v
       LLMProvider          OpenAI-compatible (OpenRouter/Groq/self-hosted) or Ollama
              |
              v
       RAGResponse (answer + sources + metadata; OpenAI-compatible dict access)

RAG is the only module that knows about every other module — it wires concrete implementations together from Settings and constructor arguments, and exposes ingest()/query() as the public surface. Hybrid retrieval, reranking, and query rewriting are configuration (feature flags), not forked code paths.

Vector storage runs embedded by default — no separate service to start, and both the vector index and the BM25 keyword index survive a process restart when pointed at the same persist_directory. For concurrent multi-process access or larger-scale production use, point at a real Qdrant server instead via VECTOR_DB_HOST — no other code changes needed.

Supported Documents

Supported and tested: .pdf, .txt, .md, .docx.

Not implemented: any other format (.pptx, .html, .csv, images/OCR, etc.) raises UnsupportedFileTypeError rather than silently failing. Adding a format is one loader function in ingestion/loaders.py — nothing else in the pipeline changes.

Both plain paths and file:// URIs are accepted, including paths with spaces:

document="C:/Users/name/My Documents/report.pdf"
document="file:///C:/Users/name/My Documents/report.pdf"

Supported Providers

Any server that speaks the OpenAI Chat Completions API works via base_url — verified against:

Provider	base_url
OpenAI	(leave unset)
OpenRouter	https://openrouter.ai/api/v1
Groq	https://api.groq.com/openai/v1
Google Gemini (compat layer)	https://generativelanguage.googleapis.com/v1beta/openai/
Self-hosted (vLLM, LM Studio, ...)	your server's /v1 URL
Ollama (native, no API key)	ollama_base_url via Settings/.env, not base_url

A provider's native, non-OpenAI-shaped API (e.g. Anthropic's Messages API) isn't covered by this path — that would need a dedicated LLMProvider implementation (see 4. Custom Retrieval for the same extension pattern).

Configuration

Everything is a RAG(...) keyword argument or an environment variable / .env entry — nothing is hardcoded. An explicit constructor argument always overrides .env for that instance.

Variable	RAG(...) argument	Default	Purpose
API_KEY	api_key	(none)	LLM API key. Needed only for .query(), not .ingest().
MODEL	model	gpt-4o-mini	Chat model name.
OPENAI_BASE_URL	base_url	(none)	Any OpenAI-compatible endpoint (OpenRouter, Groq, self-hosted, ...).
LLM_PROVIDER	(via settings=)	openai	openai or ollama.
OLLAMA_BASE_URL / OLLAMA_MODEL	(via settings=)	http://localhost:11434 / llama3.2:3b	Used when LLM_PROVIDER=ollama.
EMBEDDING_MODEL	embedding_provider (instance)	sentence-transformers/all-MiniLM-L6-v2	Local embedding model.
RERANKER_MODEL	(via settings=)	cross-encoder/ms-marco-MiniLM-L-6-v2	Local cross-encoder model.
VECTOR_DB_PATH	persist_directory	./.ragkit_data/qdrant	Embedded local index location.
VECTOR_DB_HOST / VECTOR_DB_PORT	(via settings=)	(none) / 6333	Set VECTOR_DB_HOST to use a real Qdrant server.
CHUNK_SIZE / CHUNK_OVERLAP	chunk_size / chunk_overlap	500 / 100	Chunking parameters, in characters.
TOP_K	top_k	5	Chunks included in the final answer's context.
ENABLE_HYBRID_RETRIEVAL	retrieval / hybrid_retrieval	true	Combine vector + BM25 retrieval.
ENABLE_RERANKING	rerank / reranker	true	Cross-encoder rerank of candidates.
ENABLE_QUERY_REWRITING	query_rewriter	false	LLM-based query rewriting before retrieval.
LOG_LEVEL	(via settings=)	INFO	Logging level for the ragkit logger namespace.

Copy .env.example to .env for local development; .env is git-ignored. Errors are all subclasses of RAGKitError (ConfigurationError, IngestionError, UnsupportedFileTypeError, EmbeddingError, VectorStoreError, RetrievalError, GenerationError) — catch the base class to handle any of them, or a specific one to react differently.

Troubleshooting

• ConfigurationError: No API_KEY configured — api_key= (or API_KEY in .env) is only needed for .query(), not .ingest(). Set it before calling .query().
• UnsupportedFileTypeError — the extension isn't in Supported Documents. The error message lists exactly what's registered.
• A confident-sounding wrong answer — check response.sources: if they don't actually support the answer, that's a retrieval problem (try rerank=True, a larger top_k, or retrieval="hybrid"), not a generation problem. The system prompt instructs the model to say "not in the provided context" rather than guess, but retrieving the wrong chunks still produces an ungrounded answer.
• VectorStoreError opening the same path twice — embedded Qdrant is single-process; a second RAG instance pointed at the same persist_directory within the same process raises this. Use separate paths, or a real Qdrant server (VECTOR_DB_HOST) for concurrent access. A later, separate process reusing the same path is fine — both the vector and BM25 indexes persist.
• OpenRouter free-tier 429 — free models on OpenRouter share a rate-limited pool; this means retry shortly, not that anything is misconfigured.

Contributing

pip install -e ".[dev]"

pytest tests/unit -q          # fast, no external services
pytest tests/integration -q   # full pipeline wiring (fakes) + optional live tests

RAGKIT_RUN_LIVE_INTEGRATION_TESTS=1 pytest tests/integration -q   # needs a real API_KEY
RAGKIT_RUN_LIVE_OLLAMA_TESTS=1 pytest tests/integration -q        # needs Ollama running

pytest --cov=ragkit --cov-report=term-missing

Runnable examples matching each part of Usage live in examples/ (basic.py, openrouter.py, sources.py, advanced.py). A dataset-driven evaluation harness (evaluation/evaluate.py) compares retrieval configurations against a fixed corpus and question set — see the script's docstring for usage.

To build and check the package before publishing:

pip install build twine
python -m build
python -m twine check dist/*

License

MIT