RAGMill

A lightweight, zero-config local pipeline engine for AI data ingestion, semantic chunking, embeddings, vector search, and retrieval-augmented chat — fully offline by default (no API keys, ever), with optional cloud backends (Pinecone, Qdrant), a REST API, a standalone setup UI, and Docker support.

RAGMill takes a folder of documents and turns it into a searchable, question-answerable knowledge base in a few lines of Python — or entirely from the command line. It handles the full RAG pipeline end to end: parsing files, splitting them into overlapping semantic chunks, generating embeddings, storing vectors, retrieving the most relevant passages for a query, and generating grounded, cited answers on top of them. Everything runs on your machine out of the box; the cloud and hosted-LLM integrations are strictly opt-in.

The core install has zero dependencies and works with .txt/.md files straight away. Every heavier capability — PDF/DOCX parsing, local ONNX embeddings, the local LLM, the REST server, and cloud backends — ships as an optional extra, so you only install what you actually use.

Highlights

• Offline-first, no keys required — local embeddings (ONNX) and a local LLM (Qwen2.5-1.5B via llama-cpp-python) run entirely on your machine.
• Full RAG pipeline — ingest → chunk (configurable size/overlap) → embed → store → semantic search → grounded chat, all in one package.
• Multiple file formats — text (.txt, .md, .log, .rst), data (.csv, .tsv), documents (.pdf, .docx, .rtf, .html), office (.xlsx, .pptx), and images (.png, .jpg, .tiff, …) plus scanned PDFs via OCR.
• Pluggable vector stores — local SQLite by default; switch to Pinecone or Qdrant with a couple of env vars, no code changes.
• Swappable chat backends — local LLM, Gemini, or OpenAI, selected at runtime via RAGMILL_CHAT_BACKEND.
• Incremental sync — keep a store in step with a folder, adding, updating, and deleting only what changed.
• Backend migration — export a local store to JSONL and import it into a cloud backend (or vice versa).
• Multiple interfaces — a Python API, a ragmill CLI, a FastAPI REST server (with a browser chatbox), and a standalone setup UI that writes your .env for you.
• Docker-ready — compose profiles for both SQLite and Qdrant.

Install

pip install ragmill                          # core only (txt/md), zero dependencies
pip install ragmill[all]                     # everything installable from wheels (PDF, DOCX, embeddings, server, cloud backends)
pip install ragmill[embeddings]              # + local ONNX embeddings
ragmill setup-chat                           # + local LLM for retrieval-augmented answers (no API key) — run after installing
pip install ragmill[chat-gemini]             # + Gemini as the chat backend (needs GEMINI_API_KEY)
pip install ragmill[chat-openai]             # + ChatGPT as the chat backend (needs OPENAI_API_KEY)
pip install ragmill[pinecone]                # + Pinecone cloud backend
pip install ragmill[qdrant]                  # + Qdrant cloud backend
pip install ragmill[server]                  # + FastAPI REST API
pip install ragmill[config-ui]               # + standalone setup UI (writes .env)

[all] does not include the local LLM. llama-cpp-python publishes no PyPI wheels for recent versions, so pip builds it from a 70 MB+ source archive that vendors llama.cpp — which needs a C++ toolchain, and on Windows overruns the 260-character MAX_PATH limit while unpacking:

ERROR: Could not install packages due to an OSError: [Errno 2]
No such file or directory: 'C:\\Users\\...\\vendor\\llama.cpp\\tools\\ui\\...'

Keeping it out of [all] means pip install ragmill[all] installs from wheels alone on every platform. To add the local LLM afterwards:

pip install ragmill[all]
ragmill setup-chat

setup-chat shows what it will install and from where, asks for confirmation, then fetches a prebuilt wheel — no compiler, no long-path problem. To do it by hand:

pip install llama-cpp-python \
  --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu \
  --only-binary llama-cpp-python

The --only-binary flag is required: without it pip resolves the newer sdist-only release from PyPI and tries to compile it, which is the failure this avoids.

ragmill[chat] still works if you have a working C++ toolchain (and, on Windows, long paths enabled).

Quick start

Ingest + chunk + embed + search (local SQLite)

Note: This example requires the embeddings extra: pip install ragmill[embeddings]

from ragmill import RAGEngine
from ragmill.embeddings import EmbeddingModel
from ragmill.vector_store import VectorStore

chunks = RAGEngine().execute_pipeline("./my_documents")

model = EmbeddingModel()
vectors = model.embed([c["content"] for c in chunks])

store = VectorStore("my_store.db")
store.add(chunks, vectors)

query = model.embed(["how does the overlap work?"])[0]
for r in store.search(query, top_k=3):
    print(r["score"], r["metadata"]["filename"], "->", r["content"][:80])

Keep a store in sync with a folder

from ragmill import RAGEngine
from ragmill.embeddings import EmbeddingModel
from ragmill.vector_store import VectorStore
from ragmill.sync import sync_directory

engine = RAGEngine()
model = EmbeddingModel()
store = VectorStore("my_store.db")

result = sync_directory("./my_documents", engine, model, store)
print(result)  # {"added": 2, "updated": 1, "skipped": 40, "deleted": 1}

Ask questions, get grounded answers

Retrieval-augmented answer generation, with a choice of three backends — selected via RAGMILL_CHAT_BACKEND (default local):

ragmill setup-chat                  # local model (default, no key) — prompts before installing
ragmill chat                        # interactive terminal Q&A over your ingested docs

# Switch to Gemini
pip install ragmill[chat-gemini]
export RAGMILL_CHAT_BACKEND=gemini
export GEMINI_API_KEY=xxxxxxxx
ragmill chat

# Or ChatGPT
pip install ragmill[chat-openai]
export RAGMILL_CHAT_BACKEND=openai
export OPENAI_API_KEY=xxxxxxxx
ragmill chat

from ragmill.chat import generate_answer
answer = generate_answer("what does the overlap parameter do?", results)

All three backends share the same generate_answer(query, chunks) call — the backend is picked at call time from RAGMILL_CHAT_BACKEND, so switching is just an env var change, no code change. The setup UI below lets you pick a backend and enter its key without touching the environment by hand.

Per-backend overrides:

• Local: RAGMILL_CHAT_MODEL_REPO, RAGMILL_CHAT_MODEL_FILE, RAGMILL_CHAT_N_CTX
• Gemini: RAGMILL_GEMINI_MODEL (default gemini-flash-latest)
• OpenAI: RAGMILL_OPENAI_MODEL (default gpt-4o-mini)

Use a cloud vector store

Set environment variables and the store backend switches automatically:

export RAGMILL_STORE_TYPE=pinecone
export RAGMILL_PINECONE_API_KEY=xxxxxxxx
export RAGMILL_PINECONE_ENVIRONMENT=us-west1-gcp
export RAGMILL_PINECONE_INDEX_NAME=ragmill

from ragmill.vector_store import store_from_config
from ragmill.config import RAGMillConfig

config = RAGMillConfig.from_env()
store = store_from_config(config)   # returns a PineconeVectorStore

Or for Qdrant (local via Docker, or a managed Qdrant Cloud cluster):

export RAGMILL_STORE_TYPE=qdrant
export RAGMILL_QDRANT_URL=http://localhost:6333          # or your cloud cluster URL
export RAGMILL_QDRANT_API_KEY=xxxxxxxx                    # required for Qdrant Cloud
export RAGMILL_QDRANT_COLLECTION_NAME=ragmill

Payload indexes (filename, source_file) are created automatically the first time a collection is set up — Qdrant Cloud rejects filtered search/delete/sync operations without them, while local/self-hosted Qdrant is more lenient about it.

Migrate between backends

Export your local SQLite store to JSONL, then import into a cloud store:

# 1. Export from SQLite
ragmill export ./backup.jsonl

# 2. Switch env to point at Pinecone
export RAGMILL_STORE_TYPE=pinecone
export RAGMILL_PINECONE_API_KEY=xxx

# 3. Import into Pinecone
ragmill import ./backup.jsonl

REST API

pip install ragmill[server]
ragmill serve
# or: uvicorn ragmill.server:app --host 0.0.0.0 --port 8000

Standalone setup UI

A separate, minimal web UI for filling in optional config — cloud vector store credentials, which chat backend to use (local/Gemini/ChatGPT) and its key, or an override for the local chat model — without hand-editing anything. It runs as its own server/process (a different port than ragmill serve), so it's clearly a one-time setup tool independent of wherever RAGMill actually runs as a dependency.

pip install ragmill[config-ui]
ragmill configure   # http://127.0.0.1:8090 by default — binds to localhost only

Fill in what you need and click "Save configuration" — it writes only the fields you filled in to a local .env file (via python-dotenv, preserving any unrelated lines already there). RAGMill loads that .env automatically on the next run. Remember to add .env to .gitignore — nothing is ever written into source code.

Docker

# SQLite backend
docker compose --profile sqlite up

# Qdrant backend (spins up a Qdrant container too)
docker compose --profile qdrant up

CLI

ragmill ingest ./docs       # Ingest + embed files
ragmill sync ./docs         # Incremental sync
ragmill search "query"      # Search
ragmill setup-chat          # Install the local LLM (once, prompts first)
ragmill chat                # Interactive Q&A over stored chunks (local LLM)
ragmill count                # Chunk count
ragmill serve               # Start API
ragmill export ./out.jsonl  # Export
ragmill import ./in.jsonl   # Import
ragmill configure           # Standalone setup UI (writes .env)
ragmill --version           # Print the installed version

Configuration

All settings are controlled via environment variables (or a .env file — see the setup UI above):

Supported file types

OCR requires the system tesseract binary (and pdftoppm/poppler for scanned PDFs); it is English-only by default. Scanned PDFs with no text layer fall back to OCR automatically when the ocr extra is installed.

Project structure

ragmill/
├── src/ragmill/
│   ├── __init__.py            # Public API exports
│   ├── engine.py              # RAGEngine: ingestion + chunking
│   ├── parsers.py             # PDF/DOCX text extractors
│   ├── embeddings.py          # Local ONNX embedding model
│   ├── chat.py                # Local LLM answer generation (llama-cpp-python)
│   ├── vector_store.py        # BaseVectorStore ABC + SQLiteVectorStore
│   ├── pinecone_store.py      # Pinecone backend (optional)
│   ├── qdrant_store.py        # Qdrant backend (optional)
│   ├── config.py              # RAGMillConfig: env/.env-based configuration
│   ├── sync.py                # Incremental directory sync
│   ├── export.py              # JSONL export/import for migration
│   ├── server.py              # FastAPI REST API + chat UI (optional)
│   ├── static/                # Terminal-style chatbox HTML for server.py's `/`
│   ├── config_ui.py           # Standalone setup UI, separate server (optional)
│   ├── config_ui_static/      # Setup UI's HTML form
│   └── __main__.py            # CLI entry point
├── tests/
├── docs/                      # MkDocs documentation source
├── Dockerfile
└── docker-compose.yml

Documentation

Full documentation is published at https://abdullahbinaqeel.github.io/RAGMill/.

• Quickstart · Installation · Configuration
• CLI · REST API · Chat
• Vector stores · Migration
• API reference · FAQ

Contributing

Contributions are welcome! Please read CONTRIBUTING.md to get started, and note our Code of Conduct. Found a security issue? Please report it privately via SECURITY.md — not a public issue. Maintainers: see RELEASING.md for the release process.

License

MIT © Abdullah Bin Aqeel