Skip to main content

RAGBench

A layered, open-source auto-RAG framework with complete retrieve → rerank → generate pipeline. Every layer is independent, connectable through a tiny data contract, and chainable with one call.

ingestion  ->  chunking  ->  embedding  ->  vector store  ->  retrieval  ->  reranking  ->  generation
 Document       Chunk        EmbeddedChunk      IndexReport      RetrievalResult   RerankedResult   GeneratedAnswer
Layer Package Units (one folder each) Output
Ingestion core/, connectors/, ingestion/ 82 connectors Document
Chunking chunking/ 106 methods Chunk
Embedding embedding/ 61 methods EmbeddedChunk
Vector store vectorstore/ 41 storage backends IndexReport / SearchResult
Retrieval retrieval/ 28 methods RetrievalResult
Reranking reranking/ 21 methods RerankedResult
Generation generation/ 9 LLM providers GeneratedAnswer
Auto mode (glue) pipeline/ routing + ask() any of the above

Quick Start

from ragbench.pipeline import ask, auto

# 1. Index your documents
auto("document.pdf", embed=True, index=True, 
     embedder="hash_baseline", collection="docs")

# 2. Ask questions - complete RAG pipeline
answer = ask(
    "What is quantum computing?",
    collection="docs",
    embedder="hash_baseline",
    generation_method="ollama"  # local LLM
)

print(answer.answer)

Layer Independence

Each layer works standalone:

from ragbench.ingestion import ingest           # layer 1 alone
from ragbench.chunking import chunk             # layer 2 alone  
from ragbench.embedding import embed            # layer 3 alone
from ragbench.retrieval import retrieve         # layer 5 alone
from ragbench.reranking import rerank           # layer 6 alone
from ragbench.generation import Generator       # layer 7 alone
from ragbench.pipeline import auto, ask         # all layers, auto-routed

# Use layers independently
docs = ingest("report.pdf")
chunks = chunk(docs, method="auto")
vectors = embed(chunks, method="openai")
hits = retrieve("what is chunking?", corpus=chunks, method="bm25")
reranked = rerank("what is chunking?", hits, method="cohere")

# Generate answer from retrieved context
generator = Generator(method="anthropic")
answer = generator.generate(
    query="what is chunking?", 
    context=reranked
)
print(answer.answer)

# Or use the pipeline
chunks = auto("report.pdf")                    # file -> chunks
vectors = auto("report.pdf", embed=True)       # file -> embeddings
hits = auto("report.pdf", query="...", 
            store="sqlite", embedder="hash_baseline")  # file -> ranked hits

# Complete RAG in one call
answer = ask(
    query="what is chunking?",
    collection="docs",
    rerank=True,
    generation_method="openai"
)

Layer independence is enforced by tests. Layers never import each other; only pipeline/ knows more than one.

Features

✅ Complete RAG Pipeline

  • Retrieve → Rerank → Generate in one call
  • 21 reranking methods (BM25, Cohere, BGE, ColBERT, etc.)
  • 9 LLM providers (Ollama, OpenAI, Anthropic, Cohere, etc.)
  • Streaming generation support
  • Multi-turn conversations with history
  • Citation extraction and tracking

✅ Modular Architecture

  • 82 connectors (PDF, Office, databases, cloud storage, APIs)
  • 106 chunking methods (semantic, fixed, sliding, hierarchical)
  • 61 embedding methods (OpenAI, Cohere, local models)
  • 41 vector stores (Pinecone, Qdrant, local SQLite/DuckDB)
  • 28 retrieval methods (dense, sparse, hybrid, fusion)

✅ Smart Defaults

  • Auto-routing picks optimal methods for your data
  • Works offline with hash_baseline embedder + BM25
  • Fallback to local models when APIs unavailable
  • No hardcoded API keys anywhere

Documentation

Installation

# Base install (standard library only)
pip install -e .

# With local embedders
pip install -e ".[embed-local]"

# Run tests
python -B -m unittest discover -s tests

Configuration

Setup for API keys: copy .env.example to .env (never commit it).

# Local setup (no API keys needed)
RAGBENCH_EMBEDDER=hash_baseline
RAGBENCH_VECTORSTORE=sqlite

# Cloud setup
OPENAI_API_KEY=sk-...
COHERE_API_KEY=...
ANTHROPIC_API_KEY=...

Note: The default embedder (hash_baseline) is an offline, non-semantic baseline for development. Configure a real embedding model (e.g., openai, cohere) for production use.

Metadata

Release files for ragbench-ai 0.4.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for ragbench-ai 0.4.0
File Size Uploaded Provenance Actions
ragbench_ai-0.4.0.tar.gz 495.3 kB Without provenance Details for ragbench_ai-0.4.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for ragbench-ai 0.4.0
File Interpreter ABI Platform Actions
ragbench_ai-0.4.0-py3-none-any.whl Python 3 none any Details for ragbench_ai-0.4.0-py3-none-any.whl

Total release size: 1.3 MB

Release files / ragbench_ai-0.4.0.tar.gz

Download URL ragbench_ai-0.4.0.tar.gz
Size 495.3 kB
Tags Source
SHA-256 checksum
How to use checksums
0b83ca4c2b90e6eea732cab53918fffbf02a514a7acf86bb09a8f3c16a5a2df5
BLAKE2b-256 checksum
How to use checksums
8fac13ee94b39011b21aa6b6a0015b8af8f8002283647612ec1f05c6a568e61e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.10.11

Release files / ragbench_ai-0.4.0-py3-none-any.whl

Download URL ragbench_ai-0.4.0-py3-none-any.whl
Size 826.8 kB
Tags Python 3
SHA-256 checksum
How to use checksums
b7ae923c818ebc5c2d8c11df62fe04a1f6fdfac66f8bcfbbe94fe6341bada6fb
BLAKE2b-256 checksum
How to use checksums
436e1e5048c416788247f77876d126b2b356b4b556bc2b022fe203d0bd44a43d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.10.11

Security for ragbench-ai 0.4.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.4.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page