Skip to main content

ragpick

Pick the right RAG architecture for your system, define it as code, and prove it works on your data.

CI PyPI Python License

Why ragpick

Building a RAG pipeline is easy. Choosing the right one is the hard part. Plain vector search misses the product codes and IDs your customers type. Internal documents leak into answers for the wrong users. A "better" pipeline ships without evidence that it is better.

ragpick makes the architecture an explicit, testable decision:

  • Recommend. Describe your requirements and get a blueprint plus the reasons for it, the alternatives, and warnings.
  • Build. The pipeline is a validated rag.yaml (architecture as code), with document-level access control built in.
  • Evaluate. Compare blueprints on your own questions: hit rate, MRR, faithfulness, latency and cost, side by side.

ragpick works alongside LangChain and LlamaIndex rather than replacing them. It owns the interfaces, config, selector and evaluation; anything else plugs in as a component.

60-second quickstart (offline, no API key)

pip install ragpick
git clone https://github.com/kavishkakalhara1/ragpick && cd ragpick   # for the sample data

ragpick init --blueprint naive --yes          # writes rag.yaml with fake, offline models
ragpick ingest examples/data                  # load → chunk → embed → index
ragpick query "How do I reset my password?"   # answer with citations
╭──────────────────────────────── Answer ────────────────────────────────╮
│ On the sign-in screen of the Lumen app, tap "Forgot password" and      │
│ enter the email address on your account. [1] You will receive a reset  │
│ link that is valid for 30 minutes. [1]                                 │
╰────────────────────────────────────────────────────────────────────────╯
Citations: account-and-password.md#0

Ask which architecture fits, then compare two blueprints on a question set:

ragpick recommend --config examples/hybrid.yaml
ragpick eval examples/eval.jsonl --config examples/naive.yaml --config examples/hybrid.yaml

Or let ragpick bench try every stable blueprint on your data and pick the winner:

ragpick bench examples/eval.jsonl --data examples/data --write best.yaml
           Evaluation results
┏━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┓
┃ Metric           ┃   naive ┃   hybrid ┃
┡━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━┩
│ hit_rate@k       │   0.929 │    1.000 │
│ mrr              │   0.774 │    0.893 │
│ precision@k      │   0.357 │    0.381 │
│ ...              │         │          │

The fake components are deterministic stand-ins for tests and demos: the embedder is a hashing trick and the generator quotes the best-matching sentences.

Quickstart with OpenAI

pip install "ragpick[openai]"
export OPENAI_API_KEY=sk-...

ragpick init --blueprint hybrid --provider openai --yes
ragpick ingest ./docs
ragpick query "How do I reset my password?" --groups support --show-contexts

Or in Python:

from ragpick import Pipeline

p = Pipeline.from_yaml("rag.yaml")
p.ingest("./docs")
ans = p.query("How do I reset my password?", groups=["support"])
print(ans.text, ans.citations)

A rag.yaml names a blueprint and overrides only what differs:

version: 1
name: my-support-bot
blueprint: hybrid
requirements: {corpus_size: medium, keyword_heavy: true, access_control: true}
components:
  embedder:  {type: openai, params: {model: text-embedding-3-small}}
  generator: {type: openai, params: {model: gpt-4o-mini, temperature: 0}}
query: {top_k: 5}

Restrict a document to groups with front matter. Queries without matching --groups never see it:

---
allowed_groups: [support]
---

Blueprints

Blueprint Status Best for
naive ✅ stable prototypes, small corpora
hybrid ✅ stable default for commercial systems; keyword-heavy domains (BM25 + vector + RRF, optional cross-encoder)
parent_child ✅ stable long structured documents: search small chunks, answer from whole sections
sql ✅ stable structured data (CSV, SQLite): read-only text-to-SQL with table-level access control
graph 🗺 planned multi-hop and global questions (GraphRAG)
agentic 🗺 planned multi-step questions and tool use
corrective ✅ stable grades retrieval, retries, and says "I don't know" instead of guessing

CLI

Command What it does
ragpick init interactive questions → rag.yaml (--blueprint NAME --yes for scripts)
ragpick recommend blueprint, reasons, alternatives, warnings
ragpick blueprints catalogue and roadmap
ragpick ingest PATH load, chunk, embed, index, persist
ragpick query "…" answer with citations (--groups, --show-contexts, --trace)
ragpick eval DATASET -c a.yaml -c b.yaml comparison table (--output results.json/.csv)
ragpick bench DATASET --data DOCS try every stable blueprint, pick the winner (--write rag.yaml)
ragpick validate print the merged config (secrets masked)
ragpick cache stats / clear inspect or clear the embedding/LLM cache
ragpick serve JSON HTTP API with API keys mapped to groups

Optional extras

Extra Adds
openai OpenAI embeddings and chat models (and OpenAI-compatible servers)
local sentence-transformers embeddings
rerank cross-encoder reranker
qdrant Qdrant vector store
pdf PDF loader
all everything above

The Ollama generator needs no extra.

Routing and serving

Answer from documents and databases: a router sends each question to the right pipeline, and ragpick serve puts it behind an HTTP API where API keys decide which groups a caller may act as:

# router.yaml
router:
  routes:
    - {name: docs, config: hybrid.yaml, description: Manuals, troubleshooting, policies}
    - {name: analytics, config: sql.yaml, description: Orders, sales and prices,
       keywords: [how many, total, revenue], groups: [staff]}
serve:
  api_keys: [{key: "${PORTAL_KEY}", name: support-portal, groups: [support]}]
ragpick serve -c router.yaml
curl -s localhost:8000/query -H "Authorization: Bearer $PORTAL_KEY" \
  -d '{"question": "What does error E-1042 mean?"}'

See docs/routing.md and docs/serving.md.

Tracing and caching

See where time goes, and stop paying twice for the same work:

ragpick query "What does error E-1042 mean?" --trace
query 2.4 ms
├── retrieve 0.7 ms  retriever=corrective attempts=1 kept=2
│   ├── vector_search 0.5 ms  results=5
│   ├── keyword_search 0.0 ms  results=2
│   └── grade 0.2 ms  grader=lexical contexts=5 relevant=2
└── generate 0.1 ms  prompt_tokens=516 completion_tokens=33 cached=False
tracing: {enabled: true, exporters: [{type: jsonl, params: {path: .ragpick/traces.jsonl}}]}
cache: {enabled: true}     # embeddings, answers and LLM calls, keyed by model fingerprint

Traces can also go to OpenTelemetry ({type: otel} exporter). See docs/operations.md.

LangChain and LlamaIndex

Use their loaders, embedding models, retrievers and LLMs as ragpick components, with no extra dependency in ragpick:

components:
  embedder:
    type: langchain
    params: {class_path: langchain_openai.OpenAIEmbeddings, kwargs: {model: text-embedding-3-small}}
  generator:
    type: llamaindex
    params: {class_path: llama_index.llms.openai.OpenAI, kwargs: {model: gpt-4o-mini}}

See docs/adapters.md.

Documentation

Concepts · Blueprints · Config reference · Writing a plugin · Evaluation · Routing · Serving · Tracing & caching · LangChain & LlamaIndex · Design decisions

Roadmap

  • v0.2 ✅: parent_child blueprint, ragpick bench, LangChain and LlamaIndex adapters
  • v0.3 ✅: sql and corrective blueprints, tracing, caching
  • v0.4 ✅: query routing between sql and document pipelines, ragpick serve HTTP API, OpenTelemetry trace exporter
  • v0.5: graph blueprint (GraphRAG), streaming answers (/query server-sent events), Postgres backend for sql
  • Later: agentic blueprint, multi-tenancy, TypeScript SDK

Contributing

Contributions are very welcome, especially new loaders, stores and blueprints. See CONTRIBUTING.md to get set up in two minutes with uv.

Author

Created and maintained by Kavishka Kalhara: kavishka-kalhara.me · GitHub @kavishkakalhara1

License

Copyright 2026 Kavishka Kalhara. Licensed under Apache-2.0.

Metadata

Release files for ragpick 0.4.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for ragpick 0.4.0
File Size Uploaded Provenance Actions
ragpick-0.4.0.tar.gz 161.6 kB Without provenance Details for ragpick-0.4.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for ragpick 0.4.0
File Interpreter ABI Platform Actions
ragpick-0.4.0-py3-none-any.whl Python 3 none any Details for ragpick-0.4.0-py3-none-any.whl

Total release size: 274.2 kB

Release files / ragpick-0.4.0.tar.gz

Download URL ragpick-0.4.0.tar.gz
Size 161.6 kB
Tags Source
SHA-256 checksum
How to use checksums
8a53314e3f092b1fa5493cf8b8dd03d11732437c66e90257881a23a268f86270
BLAKE2b-256 checksum
How to use checksums
86e99e310baab832baed11e1198e5b81c034ed449a41ba80ec26bada42149a03
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.9 {"installer":{"name":"uv","version":"0.13.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / ragpick-0.4.0-py3-none-any.whl

Download URL ragpick-0.4.0-py3-none-any.whl
Size 112.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
338536a6a2451e9bee841ae0838d8027253ec115dbc51c741ac837fe8c32bd20
BLAKE2b-256 checksum
How to use checksums
c62d6a453b28ae304dc1c8d3d58667973b6d826af0d6e6cff010c24b9e34d582
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.9 {"installer":{"name":"uv","version":"0.13.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for ragpick 0.4.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page