📄 DocRAG LLM

Docling → Chroma → Ollama: Simple Local RAG Pipeline

🔎 What is DocRAG LLM?

DocRAG LLM is a local-first Retrieval-Augmented Generation (RAG) pipeline.
It connects Docling for parsing → ChromaDB for vector storage → Ollama for local LLM inference.

No cloud lock-in. No API costs. Just local docs → local vectors → local LLMs.

✨ Features

• 🔍 Parse documents with Docling (PDF, DOCX, PPTX, HTML, etc.)
• 📑 Intelligent chunking for retrieval
• 🧠 Store embeddings in ChromaDB
• 🤖 Answer questions using Ollama (default: llama3.2:1b)
• 🛡️ Privacy-first → all local execution
• 🖥️ Use as a CLI tool or Python library

📦 Installation

pip install docrag-llm

Requirements:

• Python 3.10+
• Ollama installed & running
• Local models:

  ollama pull llama3.2:1b -- or any other model
  ollama pull nomic-embed-text
  

🚀 Quickstart

CLI – Ingest and Ask

# Ingest a document (default collection: demo)
python -m docrag.cli ingest https://arxiv.org/pdf/2508.20755

# Ask a question (default LLM: llama3.2:1b) you can always add the tag `-llm gpt-oss:20b` for better response assuming you have the computing power for it. 
python -m docrag.cli ask "Summarize in 1 paragraph with 5 bullet points" 
python -m docrag.cli ask "Summarize in 1 paragraph with 5 bullet points" -llm gpt-oss:20b

Python API - llama3.2:1b is just for testing, reccomend if you have the computing power to ues gpt-oss:20b you will get better results. change as needed, pull from ollama first.

from docrag import DocragSettings, RAGPipeline

cfg = DocragSettings(
    persist_path="./.chroma",
    collection="demo",
    embed_model="nomic-embed-text",
    llm_model="llama3.2:1b",
)

pipeline = RAGPipeline(cfg)

# Ingest
n_chunks = pipeline.ingest("https://arxiv.org/pdf/2508.20755")
print(f"Ingested {n_chunks} chunks")

# Ask
answer = pipeline.ask("Give a concise bullet summary of the paper's contributions.")
print(answer)

⚙️ Configuration

Both CLI & Python API let you customize:

• persist_path → where ChromaDB stores vectors
• collection → logical collection name
• embed_model → embedding model (Ollama tag)
• llm_model → LLM model (default: llama3.2:1b)
• chunk_chars / chunk_overlap → chunking granularity

📊 Roadmap

• model-check CLI → list installed Ollama models
• Support multiple backends (Weaviate, Milvus)
• Streaming output for long answers
• Expanded test suite (large document regression cases)
• Example notebooks & Hugging Face demo

🤝 Contributing

PRs and issues welcome!

pip install "docrag-llm[dev]"
ruff check .
pytest

📜 License

MIT License © 2025 Armando Medina