DeepDive

A local-first AI research agent. Asks a question, searches the web (or your own documents), reads the sources, and writes a cited report. Every claim is checked against a verbatim excerpt from its source — claims the validator can't verify are dropped.

pip install deepdive-research
ollama pull llama3.2
deepdive research "When did Apollo 11 land on the Moon?" -o report.md

No API key required. Runs on Ollama by default.

Features

Span-grounded citations

The claim extractor returns each claim alongside the verbatim excerpt that supports it. A post-extraction validator re-checks every excerpt against the fetched source page via substring match. Claims that don't ground get dropped from the report.

The Markdown report ends with an Evidence appendix listing every claim and its verbatim source excerpt:

**1.** Apollo 11 landed on the Moon on July 20, 1969.
   - [✓] [Apollo 11 - Wikipedia](https://en.wikipedia.org/wiki/Apollo_11)
     > Apollo 11 (July 16-24, 1969) was the American spaceflight that
     > first landed humans on the Moon, and the fifth crewed mission of
     > NASA's Apollo program.

Trace and replay

Every research run can record a trace of every LLM call, search, and scrape. Replay reconstructs the report offline, byte-for-byte.

deepdive research "..." -o report.md         # auto-records report.md.trace.jsonl
deepdive replay report.md.trace.jsonl        # produces an identical report, no network
deepdive trace verify report.md.trace.jsonl  # re-validates every excerpt
deepdive inspect report.md.trace.jsonl       # event counts

Source restriction

Restrict search to your own allow/block lists of hostnames. Subdomains match; wildcards like *.gov are supported.

deepdive research "..." --allow-domains arxiv.org,nih.gov
deepdive research "..." --block-domains medium.com,*.substack.com

Local corpus and offline mode

Index your own documents into a sqlite-vec database, then research them with no outbound network calls.

deepdive index ~/Documents/papers -o ~/.local/share/deepdive/index.db
deepdive research "..." --corpus ~/.local/share/deepdive/index.db --offline

In --offline mode the LLM endpoint must be loopback (Ollama is fine; cloud APIs raise) and non-loopback URLs are dropped at the scraper level.

Multi-format export

deepdive research "..." -o report.md                      # markdown (default)
deepdive research "..." -o paper.tex --export latex       # +references.bib
deepdive research "..." -o data.json --export json
deepdive research "..." -o note.md --export obsidian      # YAML + [[wikilinks]]
deepdive research "..." -o page.md --export notion
deepdive research "..." -o refs.bib --export bibtex

Use as an MCP server

Expose DeepDive's research tool to any MCP-compatible client.

deepdive serve-mcp                         # stdio
deepdive serve-mcp --http --port 8765      # Streamable HTTP

Install

pip install "deepdive-research[all]"     # everything: MCP server, corpus indexing
pip install "deepdive-research[mcp]"     # MCP server only
pip install "deepdive-research[corpus]"  # local corpus indexing only

ollama pull llama3.2           # tool-capable LLM (default)
ollama pull nomic-embed-text   # embeddings (only needed for --corpus)

Architecture

DeepDive is built on actants. The pipeline:

question
   │
   ▼  query generation              (LLM)
search backend                       (DuckDuckGo / SearxNG / local corpus)
   │                                 │  optional: --allow / --block filter
   ▼  scraper                        (web / corpus / offline)
claim extractor                      (LLM, returns excerpts)
   │
   ▼  span-grounding validator       (drops claims without verbatim excerpts)
cross-reference
   │
   ▼  report builder                 (LLM)
Markdown / LaTeX / BibTeX / JSON / Obsidian / Notion

Every external call is wrapped in optional recorders, which is how trace and replay work.

What's not in scope

• Hosted SaaS or paid tier — DeepDive is open-source and runs on your hardware
• Multi-modal output (charts, infographics)
• Conversational follow-up on a single report — single-shot research only
• Browser automation
• Code execution as a research step

Status

• Version: 0.3.1
• License: MIT
• Python: 3.12+
• Framework: actants ≥ 0.5.0

Part of Open Intelligence Labs.