ragpick
Pick the right RAG architecture for your system, define it as code, and prove it works on your data.
Why ragpick
Building a RAG pipeline is easy. Choosing the right one is the hard part. Plain vector search misses the product codes and IDs your customers type. Internal documents leak into answers for the wrong users. A "better" pipeline ships without evidence that it is better.
ragpick makes the architecture an explicit, testable decision:
- Recommend. Describe your requirements and get a blueprint plus the reasons for it, the alternatives, and warnings.
- Build. The pipeline is a validated
rag.yaml(architecture as code), with document-level access control built in. - Evaluate. Compare blueprints on your own questions: hit rate, MRR, faithfulness, latency and cost, side by side.
ragpick works alongside LangChain and LlamaIndex rather than replacing them. It owns the interfaces, config, selector and evaluation; anything else plugs in as a component.
60-second quickstart (offline, no API key)
pip install ragpick
git clone https://github.com/kavishkakalhara1/ragpick && cd ragpick # for the sample data
ragpick init --blueprint naive --yes # writes rag.yaml with fake, offline models
ragpick ingest examples/data # load → chunk → embed → index
ragpick query "How do I reset my password?" # answer with citations
╭──────────────────────────────── Answer ────────────────────────────────╮
│ On the sign-in screen of the Lumen app, tap "Forgot password" and │
│ enter the email address on your account. [1] You will receive a reset │
│ link that is valid for 30 minutes. [1] │
╰────────────────────────────────────────────────────────────────────────╯
Citations: account-and-password.md#0
Ask which architecture fits, then compare two blueprints on a question set:
ragpick recommend --config examples/hybrid.yaml
ragpick eval examples/eval.jsonl --config examples/naive.yaml --config examples/hybrid.yaml
Or let ragpick bench try every stable blueprint on your data and pick the winner:
ragpick bench examples/eval.jsonl --data examples/data --write best.yaml
Evaluation results
┏━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┓
┃ Metric ┃ naive ┃ hybrid ┃
┡━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━┩
│ hit_rate@k │ 0.929 │ 1.000 │
│ mrr │ 0.774 │ 0.893 │
│ precision@k │ 0.357 │ 0.381 │
│ ... │ │ │
The fake components are deterministic stand-ins for tests and demos: the embedder is a hashing trick and the generator quotes the best-matching sentences.
Quickstart with OpenAI
pip install "ragpick[openai]"
export OPENAI_API_KEY=sk-...
ragpick init --blueprint hybrid --provider openai --yes
ragpick ingest ./docs
ragpick query "How do I reset my password?" --groups support --show-contexts
Or in Python:
from ragpick import Pipeline
p = Pipeline.from_yaml("rag.yaml")
p.ingest("./docs")
ans = p.query("How do I reset my password?", groups=["support"])
print(ans.text, ans.citations)
A rag.yaml names a blueprint and overrides only what differs:
version: 1
name: my-support-bot
blueprint: hybrid
requirements: {corpus_size: medium, keyword_heavy: true, access_control: true}
components:
embedder: {type: openai, params: {model: text-embedding-3-small}}
generator: {type: openai, params: {model: gpt-4o-mini, temperature: 0}}
query: {top_k: 5}
Restrict a document to groups with front matter. Queries without matching --groups never see it:
---
allowed_groups: [support]
---
Blueprints
| Blueprint | Status | Best for |
|---|---|---|
naive |
✅ stable | prototypes, small corpora |
hybrid |
✅ stable | default for commercial systems; keyword-heavy domains (BM25 + vector + RRF, optional cross-encoder) |
parent_child |
✅ stable | long structured documents: search small chunks, answer from whole sections |
sql |
✅ stable | structured data (CSV, SQLite, PostgreSQL): read-only text-to-SQL with table-level access control |
graph |
✅ stable | multi-hop and "main themes" questions: knowledge graph + community summaries (GraphRAG) |
agentic |
✅ stable | multi-part and comparison questions: plans several searches, fuses the results |
multimodal |
🗺 planned | images, charts and scanned pages inside documents |
corrective |
✅ stable | grades retrieval, retries, and says "I don't know" instead of guessing |
CLI
| Command | What it does |
|---|---|
ragpick init |
interactive questions → rag.yaml (--blueprint NAME --yes for scripts) |
ragpick recommend |
blueprint, reasons, alternatives, warnings |
ragpick blueprints |
catalogue and roadmap |
ragpick ingest PATH |
load, chunk, embed, index, persist |
ragpick query "…" |
answer with citations (--groups, --show-contexts, --trace, --stream) |
ragpick eval DATASET -c a.yaml -c b.yaml |
comparison table (--output results.json/.csv) |
ragpick bench DATASET --data DOCS |
try every stable blueprint, pick the winner (--write rag.yaml) |
ragpick validate |
print the merged config (secrets masked) |
ragpick cache stats / clear |
inspect or clear the embedding/LLM cache |
ragpick serve |
JSON HTTP API with API keys mapped to groups |
Optional extras
| Extra | Adds |
|---|---|
openai |
OpenAI embeddings and chat models (and OpenAI-compatible servers) |
local |
sentence-transformers embeddings |
rerank |
cross-encoder reranker |
qdrant |
Qdrant vector store |
pdf |
PDF loader |
all |
everything above |
The Ollama generator needs no extra.
Routing and serving
Answer from documents and databases: a router sends each question to the right
pipeline, and ragpick serve puts it behind an HTTP API where API keys decide which
groups a caller may act as:
# router.yaml
router:
routes:
- {name: docs, config: hybrid.yaml, description: Manuals, troubleshooting, policies}
- {name: analytics, config: sql.yaml, description: Orders, sales and prices,
keywords: [how many, total, revenue], groups: [staff]}
serve:
api_keys: [{key: "${PORTAL_KEY}", name: support-portal, groups: [support]}]
ragpick serve -c router.yaml
curl -s localhost:8000/query -H "Authorization: Bearer $PORTAL_KEY" \
-d '{"question": "What does error E-1042 mean?"}'
Answers can stream token by token: ragpick query "…" --stream, pipeline.stream(...),
or POST /query {"stream": true} (server-sent events).
Serving many customers? tenancy: {enabled: true} gives every tenant an isolated
index, and API keys can be bound to a tenant. See docs/tenancy.md.
See docs/routing.md and docs/serving.md.
Tracing and caching
See where time goes, and stop paying twice for the same work:
ragpick query "What does error E-1042 mean?" --trace
query 2.4 ms
├── retrieve 0.7 ms retriever=corrective attempts=1 kept=2
│ ├── vector_search 0.5 ms results=5
│ ├── keyword_search 0.0 ms results=2
│ └── grade 0.2 ms grader=lexical contexts=5 relevant=2
└── generate 0.1 ms prompt_tokens=516 completion_tokens=33 cached=False
tracing: {enabled: true, exporters: [{type: jsonl, params: {path: .ragpick/traces.jsonl}}]}
cache: {enabled: true} # embeddings, answers and LLM calls, keyed by model fingerprint
Traces can also go to OpenTelemetry ({type: otel} exporter). See docs/operations.md.
LangChain and LlamaIndex
Use their loaders, embedding models, retrievers and LLMs as ragpick components, with no extra dependency in ragpick:
components:
embedder:
type: langchain
params: {class_path: langchain_openai.OpenAIEmbeddings, kwargs: {model: text-embedding-3-small}}
generator:
type: llamaindex
params: {class_path: llama_index.llms.openai.OpenAI, kwargs: {model: gpt-4o-mini}}
See docs/adapters.md.
Documentation
Concepts · Blueprints · Config reference · Writing a plugin · Evaluation · Routing · Serving · Multi-tenancy · Tracing & caching · LangChain & LlamaIndex · Design decisions
Roadmap
- v0.2 ✅:
parent_childblueprint,ragpick bench, LangChain and LlamaIndex adapters - v0.3 ✅:
sqlandcorrectiveblueprints, tracing, caching - v0.4 ✅: query routing between
sqland document pipelines,ragpick serveHTTP API, OpenTelemetry trace exporter - v0.5 ✅:
graphblueprint (GraphRAG local search), streaming answers, PostgreSQL backend forsql - v0.6 ✅: GraphRAG global search (community summaries),
agenticblueprint, per-tenant indexes - v0.7:
multimodalblueprint, pgvector store, evaluation datasets generated from your documents - Later: TypeScript SDK, more vector stores (Chroma, LanceDB)
Contributing
Contributions are very welcome, especially new loaders, stores and blueprints. See
CONTRIBUTING.md to get set up in two minutes with uv.
Author
Created and maintained by Kavishka Kalhara: kavishka-kalhara.me · GitHub @kavishkakalhara1
License
Copyright 2026 Kavishka Kalhara. Licensed under Apache-2.0.
Metadata
Release files for ragpick 0.6.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| ragpick-0.6.0.tar.gz | 197.9 kB | Without provenance | Details for ragpick-0.6.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| ragpick-0.6.0-py3-none-any.whl | Python 3 | none | any | Details for ragpick-0.6.0-py3-none-any.whl |
Total release size: 332.5 kB
Release files / ragpick-0.6.0.tar.gz
| Download URL | ragpick-0.6.0.tar.gz |
|---|---|
| Size | 197.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
fa8b0815d5c205e2a28c5070bf6d6717356b0af0f9ba30efdc117378a9343e8d
|
|
BLAKE2b-256 checksum How to use checksums |
621855d60d3deba655ee7b01b217f961522b939effa3efee022c3466ab85712c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.13.9 {"installer":{"name":"uv","version":"0.13.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / ragpick-0.6.0-py3-none-any.whl
| Download URL | ragpick-0.6.0-py3-none-any.whl |
|---|---|
| Size | 134.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
b25cb15464567b37497529883346e9becb026f59e9137b2ee84dff4bc8e2f1dc
|
|
BLAKE2b-256 checksum How to use checksums |
200b81c0299391e5aaf48f2f3cf3a5161964cde6aea4c8c182d41758c895ff91
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.13.9 {"installer":{"name":"uv","version":"0.13.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Security for ragpick 0.6.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.