raglite-toolkit

Build semantic search, multi-provider question answering, and REST APIs over your documents, directories, or web URLs in a few lines of Python.

raglite-toolkit is a Python port of the raglite-toolkit TypeScript package with full 1:1 feature parity.

Features

• 📄 PDF, TXT, JSON, Markdown, DOCX loaders out of the box
• 📁 Multi-document, directory, & URL ingestion — index folders, glob patterns, or web URLs with DocumentCollection
• 🤖 Multi-provider LLMs — OpenAI, Anthropic (Claude), Google (Gemini), Mistral, Cohere, Groq, xAI, Ollama
• 🔢 Multi-provider embeddings — OpenAI, Google, Mistral, Cohere, Voyage, Ollama, or a local offline sentence-transformer (no API key needed)
• 📐 Cosine similarity scoring with L2-normalized vectors
• 🔎 Hybrid search — BM25 keyword search fused with vector search, for exact terms like error codes and SKUs
• 🌏 Any language — Chinese, Japanese, Korean and Thai text is chunked and keyword-indexed correctly
• ♻️ Content-hash cache — reindexes only when the file actually changes
• 🗂 Per-document namespacing — indexes are isolated, two documents never collide
• 🌐 REST API via FastAPI with optional bearer-token auth
• ⚡ Streaming answers
• 🐍 Python-native — Pydantic models, type-annotated, fully testable

Install

Requires Python 3.11 or later.

The core install is small. Add an extra for each provider you use; the Supported Providers tables list the extra for each one:

pip install 'raglite-toolkit[openai,anthropic]'   # OpenAI embeddings + Claude answers
pip install 'raglite-toolkit[local]'              # local offline embeddings (the default), no API key
pip install 'raglite-toolkit[all]'                # every provider

If an extra is missing, RAGLite raises a ConfigError that names the pip install command to run. The local extra installs sentence-transformers (with PyTorch); the all-MiniLM-L6-v2 model (~90 MB) is downloaded on first use.

Quick Start

from raglite import Document

doc = Document("./policy.pdf", {
    "embeddings": {"provider": "openai", "apiKey": "sk-..."},
    "llm":        {"provider": "anthropic", "apiKey": "sk-ant-..."},
})

doc.build()                              # chunk → embed → persist

hits = doc.search("refund policy", top_k=3)

answer = doc.ask("What is the refund policy?")
print(answer.text)

Index Text You Already Have

No file needed: index a database row, an upload or CMS content with Document.from_text(id, text). The id names the index (reuse it to reuse the index) and becomes each chunk's source. Changed text is re-indexed on the next build().

doc = Document.from_text("faq-42", faq_text, {"embeddings": {"provider": "openai"}})
doc.build()

Multi-Document & Directory Ingestion (DocumentCollection)

Index entire directories (./docs), glob patterns, web URLs, or mixed file arrays:

from raglite import DocumentCollection

collection = DocumentCollection(["./docs", "https://example.com"], {
    "embeddings": {"provider": "local"},
    "llm": {"provider": "openai", "apiKey": "sk-..."},
})

# Index all documents concurrently
result = collection.build()
print(f"Indexed {result.totalDocuments} document(s), {result.totalChunks} chunk(s).")

# Search across all collection documents simultaneously
hits = collection.search("refund policy", top_k=5)

# Contextual Q&A across the entire collection
answer = collection.ask("What is the refund policy?")
print(answer.text)

Hybrid Search (Keyword + Vector)

Vector search matches meaning, but it can miss exact terms such as error codes, SKUs, function names or rare product names. Keyword search (BM25) finds those exactly. hybrid runs both and merges the results with Reciprocal Rank Fusion.

doc.search("ERR_4021", mode="hybrid")  # "vector" (default) | "keyword" | "hybrid"
doc.ask("What does ERR_4021 mean?", {"mode": "hybrid"})

# Or set a default once; it applies to search(), ask() and ask_stream().
Document("./runbook.md", {
    "retrieval": {
        "mode": "hybrid",
        "hybrid": {"rrfK": 60, "candidates": 50, "weights": {"vector": 1, "keyword": 1}},
    },
})

• In keyword and hybrid modes, score is the fused rank score scaled to 0..1 (1 means ranked first by every retriever). Each result also has scores with vector, keyword and fused. Vector mode results are unchanged.
• scoreThreshold is still a cosine similarity. In hybrid mode it filters the vector results before fusion; keyword matches are not filtered by it.
• The keyword index is built by build() from the chunk texts, with no extra embedding calls, and saved as <storeDir>/<namespace>/keyword.json. It works with every vector store. With Qdrant or Pinecone, keep storeDir on persistent disk.
• Indexes built before 1.3: the memory and Qdrant stores build the keyword index from the stored chunks on the first keyword or hybrid search. Other stores log a warning and use vector search until you run build(rebuild=True).
• The same options work over HTTP ("mode" on /search and /ask) and in the CLI (--mode hybrid).

The tokenizer is the same in the Python and TypeScript SDKs:

There is no stemming or stopword list, because both are language-specific: in keyword mode refund does not match refunds. Hybrid mode's vector side covers those cases.

Fully Offline — No API Key Needed

Install pip install 'raglite-toolkit[local,ollama]', then:

from raglite import Document

doc = Document("./manual.txt", {
    "embeddings": {"provider": "local"},
    "llm":        {"provider": "ollama", "model": "llama3.2"},
})

doc.build()
print(doc.ask("How do I reset the device?").text)

Choose Any LLM at Ask-Time

# Pass an inline LLM override to ask()
gpt4 = doc.ask("Summarise this document", options={
    "llm": {"provider": "openai", "model": "gpt-4o", "apiKey": "sk-..."}
})

claude = doc.ask("Summarise this document", options={
    "llm": {"provider": "anthropic", "model": "claude-sonnet-5-5", "apiKey": "sk-ant-..."}
})

Streaming Responses

for chunk in doc.ask_stream("Explain section 3 in detail"):
    print(chunk, end="", flush=True)
print()

Pluggable Vector Databases

raglite supports pluggable vector stores (Memory, Qdrant, Pinecone, or custom subclasses; LanceDB is currently TypeScript-only):

Memory Store (Default)

doc = Document("./policy.pdf", {
    "vectorStore": {"provider": "memory", "storeDir": ".raglite"}
})

Qdrant Store

By default each document gets its own collection (raglite_<namespace>). Set indexName to keep several documents in one shared collection; each document's points are tagged with its namespace, so rebuilding one document never affects the others.

doc = Document("./policy.pdf", {
    "vectorStore": {
        "provider": "qdrant",
        "url": "http://localhost:6333",
        "apiKey": "your-key",
        "indexName": "my_collection"
    }
})

Pinecone Store

doc = Document("./policy.pdf", {
    "vectorStore": {
        "provider": "pinecone",
        "url": "https://my-index.svc.pinecone.io",
        "apiKey": "your-key"
    }
})

REST API

from raglite import Document

doc = Document("./policy.pdf", {
    "embeddings": {"provider": "local"},
    "llm":        {"provider": "openai", "apiKey": "sk-..."},
})
doc.build()

# Start background FastAPI server on port 8085
doc.serve(port=8085, bearer_token="secret-token")

Endpoints:

Example curl calls

# Health check (no auth)
curl http://127.0.0.1:8085/health

# Search
curl -X POST http://127.0.0.1:8085/search \
  -H 'Authorization: Bearer secret-token' \
  -H 'Content-Type: application/json' \
  -d '{"query": "refund policy", "topK": 3}'

# Ask (non-streaming)
curl -X POST http://127.0.0.1:8085/ask \
  -H 'Authorization: Bearer secret-token' \
  -H 'Content-Type: application/json' \
  -d '{"question": "What is the refund policy?"}'

# Ask (streaming)
curl -X POST http://127.0.0.1:8085/ask \
  -H 'Authorization: Bearer secret-token' \
  -H 'Content-Type: application/json' \
  -d '{"question": "Summarize the document", "stream": true}'

CLI

# Index a document, directory, or URL
raglite index ./policy.pdf --embed-provider local

# Semantic search
raglite search ./docs "refund policy" --top-k 5
raglite search ./docs "ERR_4021" --mode hybrid

# Ask a question (streaming)
raglite ask ./docs "What is the refund policy?" \
  --llm-provider anthropic --llm-key $ANTHROPIC_API_KEY --stream

# Serve a REST API
raglite serve https://example.com \
  --llm-provider openai --llm-key $OPENAI_API_KEY \
  --port 8085 --token $RAGLITE_TOKEN

Supported Providers

LLMs

Anthropic requests send no temperature unless you set one, because Claude 5 models reject non-default sampling values.

Embeddings

The Qdrant and Pinecone stores need no extra.

Configuration Reference

Document("./policy.pdf", {
    # Chunking
    "chunkSize":      500,          # words per chunk; characters for Chinese, Japanese, Thai, ... (default: 500)
    "overlap":        50,           # overlapping words between chunks (default: 50)

    # Retrieval
    "topK":           5,            # default results returned (default: 5)
    "scoreThreshold": 0.0,          # minimum cosine similarity (0..1, default: 0)
    "retrieval":      {"mode": "vector"},  # "vector" | "keyword" | "hybrid" (default: vector)

    # Storage
    "storeDir":       ".raglite",   # where indexes are persisted (default: .raglite)

    # Providers
    "embeddings": {"provider": "local"},
    "llm":        {"provider": "openai", "model": "gpt-4o-mini", "apiKey": "sk-..."},

    # Logging
    "logLevel":   "info",           # "silent" | "info" | "debug" (default: info)
})

How Caching Works

Every build() call fingerprints the source (file bytes, or the fetched text for URLs) with a SHA-256 content hash and persists it alongside the vectors. The cached index is reused only if all of the following match the stored index:

Pass rebuild=True to build() to force a fresh index regardless.

Each document is stored under .raglite/<sha256-prefix>/, so multiple documents in the same project never overwrite each other.

Advanced Usage

Custom Vector Store

from raglite.vectordb.base import VectorStore

class MyVectorStore(VectorStore):
    # Implement: namespace (property), load, reset, add, search, count,
    #            save_index_metadata, read_index_metadata
    # Optional:  list_chunks() lets hybrid search rebuild a missing keyword index
    #            from your store; keyword_search(query, top_k) replaces the
    #            built-in BM25 index with your own.
    ...

Custom Loader

from raglite.loaders.base import BaseLoader
from raglite.loaders import get_loader

class CsvLoader(BaseLoader):
    def load(self) -> str:
        # read CSV, return string
        ...

Custom Chunker

from raglite.chunking.base import BaseChunker

class SentenceChunker(BaseChunker):
    def split(self, text: str) -> list[str]:
        ...

Direct Embedder Access

from raglite import create_embedder

embedder = create_embedder({"provider": "openai", "apiKey": "sk-..."})
vectors = embedder.embed_documents(["chunk one", "chunk two"])
query_vec = embedder.embed_query("refund policy")

Development

# Clone and set up
git clone https://github.com/creatorpiyush/raglite-py.git
cd raglite-py

# Create virtual environment
python3.12 -m venv .venv
source .venv/bin/activate    # Windows: .venv\Scripts\activate

# Install in editable mode with dev dependencies
pip install -e ".[dev]"

# Run test suite
pytest

# Run with coverage
pytest --cov=raglite --cov-report=term-missing

# Run examples
python examples/basic.py
python examples/serve.py

Test Structure

tests/
├── unit/
│   ├── test_chunking.py           # RecursiveChunker algorithm
│   ├── test_vectordb.py           # MemoryVectorStore (cosine, persistence, isolation)
│   ├── test_loaders.py            # TxtLoader, MarkdownLoader, JsonLoader
│   ├── test_directory_loader.py   # DirectoryLoader (recursive scanning, glob filtering)
│   ├── test_web_loader.py         # WebLoader (HTML parsing, tag stripping)
│   ├── test_prompt.py             # system/user prompt builders
│   ├── test_errors.py             # exception hierarchy
│   ├── test_config.py             # config defaults and overrides
│   ├── test_hash.py               # SHA-256 file hashing + namespace generation
│   ├── test_retriever.py          # Retriever with mocked embedder
│   └── test_cli.py                # CLI commands and argument parsing
└── integration/
    ├── test_document.py           # build/cache/search lifecycle (mocked embeddings)
    ├── test_collection.py         # DocumentCollection multi-document indexing & FastAPI server
    ├── test_ask.py                # ask/stream with mocked LLM generation
    └── test_api.py                # FastAPI endpoints via TestClient

License

MIT © Piyush Anand