rag-titan 0.8.2

📖 Documentation · Getting started · Supported RAG types · API reference · PyPI

Why rag-titan?

• Many RAG types, one API. Two today (simple, adaptive), with more on the way. Switch with one word: rag_type="adaptive".
• Runs 100% locally with Ollama, so your documents never leave your machine.
• Grounded answers. The model answers only from your files and cites the chunks it used.
• Chunking your way. Eight chunkers: by words, characters or model tokens (tiktoken or your own tokenizer), recursive separators, and structure-aware chunkers that keep sentences, paragraphs, heading sections or pages whole. Try them live in the chunk playground →
• Watch every step, in your browser. The RAG playground → runs the real package (Pyodide) side by side for any chunker and RAG type, and lights up each step on a live flow diagram: click a step to see the chunks it found. Recorded answers work instantly; a live mode with your own Ollama is work in progress.
• PDF, Word, Markdown and text files, with paragraphs, headings and pages kept for the chunkers.
• Measure, don't guess. Compare any chunker with any RAG type on your own questions in one command (rag-titan evaluate): recall, MRR, nDCG, answered rate and p95 time, plus every step of a run with --trace. Evaluating RAG →
• Find the stage that fails, in any RAG system. Pass the logged outputs of your pipeline (rag-titan or not) and get 70+ metrics for chunking, embedding, retrieval, reranking, context, answers and end to end, a 0 to 100 scorecard, root-cause hints and regression checks. No LLM judge. Evaluating every stage →
• Easy to read, easy to change. Short files, plain Python, and a docstring on every class. Swap any part by inheriting one base class.

Install

pip install rag-titan
ollama pull gemma3:4b && ollama pull nomic-embed-text

For token-based chunking, add the extra: pip install "rag-titan[tokens]".

Needs Python 3.10+ and Ollama. → Full setup guide

Quick start

from rag_titan import RAGTitan

rag = RAGTitan(rag_type="simple")            # or "adaptive"
rag.index(["handbook.pdf", "policy.docx"])

print(rag.ask("How many leave days do I get?"))
# You get 20 paid leave days every year [chunk_0].

print(rag.run("How many leave days do I get?")["log"])   # every step it took
rag.evaluate("questions.json")                           # score it on your own test questions

Or from the terminal:

rag-titan ask --files handbook.pdf --question "How many leave days do I get?"
rag-titan ask --files handbook.pdf --question "How many leave days do I get?" --trace

Evaluate on your own questions

Which chunker and which RAG type work best for your documents? Write a few test questions, each naming the file (or page) that holds the answer, and compare every combination in one command:

rag-titan evaluate --files docs/*.md --questions questions.json \
    --rag-type simple adaptive --chunker word paragraph

setup                   Recall       MRR      nDCG  Answered     Steps     p50 s     p95 s
simple + word            1.000     1.000     1.000      100%       3.0      0.92      1.26
adaptive + word          1.000     1.000     1.000      100%       5.0      1.67      2.02
simple + paragraph       0.917     1.000     0.936      100%       3.0      0.64      0.81
adaptive + paragraph     1.000     1.000     0.987      100%       6.3      0.96      2.91

Questions to look at:
  simple + paragraph: q_006  recall 0.50  found ['remote_work.md']

Then see why one question went wrong. Here search found the right file, so the problem is the answer step, not the chunker:

$ rag-titan ask --files docs/*.md --question "Can I work from Spain for a month?" --trace
1. retrieve     7.47 ms  -> 3 chunks: chunk_2 remote_work.md (0.615), chunk_0 expenses.md (0.540), ...
2. rerank       0.04 ms  -> 2 chunks: chunk_2 remote_work.md (rerank 0.815), chunk_0 expenses.md (rerank 0.690)
3. generate   567.39 ms  -> "I don't know based on the available documents."

Real runs with gemma3:4b on the sample policies in examples/sample_docs/. Search-only scoring (--retrieval-only, no LLM) and the Python API are in Evaluating RAG →

Evaluate every stage of any RAG system

A wrong answer does not say which part failed. Pass what each stage of your pipeline produced, plus the right answers, and every stage is scored on its own. It works for any RAG system, uses no LLM judge and needs no extra dependencies:

Already have a RAG and only want to evaluate it? Follow the 6 steps in the Evaluation cookbook →

from rag_titan.evaluation import RAGEvaluator

report = RAGEvaluator(k=5).evaluate(
    chunks=chunks,        # what your chunker produced
    queries=queries,      # per question: gold labels + retrieved ids, context, answer, latency
    document_text=text,   # optional: more metrics when given
)
print(report["scorecard"]["overall"]["score"])          # 0 to 100
print(report["root_causes"]["first_failing_stage"])     # e.g. "retrieval"

rag-titan evaluate --input data.json --baseline last_report.json --fail-on-regression

| Pipeline stage       | Score / 100 | Rating    | Measurement coverage |
|----------------------|------------:|-----------|---------------------:|
| Document preparation |        97.6 | Excellent |                  74% |
| Semantic indexing    |        98.4 | Excellent |                 100% |
| Information search   |        79.4 | Good      |                 100% |
| Search ordering      |        89.7 | Very Good |                 100% |
| Answer context       |        92.3 | Excellent |                 100% |
| Answer quality       |       100.0 | Excellent |                 100% |
| User experience      |       100.0 | Excellent |                 100% |

Using rag-titan itself? rag.evaluate_stages(files, "golden.json") collects the data for you. Your own RAG? rag-titan golden drafts your golden questions and build_dataset() turns your logs into the data file, see Evaluate your own RAG →. Every formula is on the Evaluation metrics → page, and the guide is Evaluating every stage →

Supported RAG types

rag-titan supports 2 RAG types. Pick one with rag_type. Every type has answer(), and run() that also returns every step it took (see the steps):

from rag_titan import RAG_TYPES
print(RAG_TYPES)        # ['simple', 'adaptive']

Coming next (not available yet): Corrective RAG · Multi-query RAG · HyDE · Hybrid RAG · Self-RAG · Parent-document RAG · Graph RAG · Agentic RAG. → Roadmap

Also included: a tool-calling agent loop. It's a separate feature, not a rag_type.

Explore the docs

Click any picture to open its page.

RAG playground Run the RAG types in your browser and watch every step light up.	Chunk playground Paste your text and watch the chunkers cut it, live in the browser.
Evaluating RAG Compare any chunker with any RAG type on your own questions.	Chunking methods All 8 chunkers explained, with formulas and animated diagrams.
Building blocks Load → chunk → embed → store, with the data at every step.	Extending rag-titan Add your own LLM, vector store, chunker or RAG type.
Agents Let the LLM call your Python functions as tools.	Command line rag-titan ask (with --trace), chat and evaluate.

Contributing

Contributions are welcome! Read CONTRIBUTING.md to get set up. The tests run without Ollama.

License

MIT © Kumar Nishant