git-rag

Retrieval-augmented search over an Obsidian vault that is managed as a git repository. Postgres + pgvector store the index; git commits drive the updates; a Claude Code skill and MCP server give agents a query surface.

Source: dogfoodlab-io/git-rag

Agent instructions (copy-paste)

Paste the block below into Cursor, Claude Code, ChatGPT, or any coding agent. Fill in the two paths first.

You are setting up git-rag (repo: https://github.com/dogfoodlab-io/git-rag) so I can
search my Obsidian vault with hybrid RAG (Postgres + pgvector) and expose it to agents
via MCP.

My absolute vault path (git repo, preferably branch main):
  VAULT_PATH=<<<ABS_PATH_TO_OBSIDIAN_VAULT>>>

Preferred install location for this tool:
  INSTALL_DIR=<<<ABS_PATH_E.G._~/dev/git-rag>>>

Do the following end-to-end without asking me to run commands unless something needs a
secret or GUI approval (Docker Desktop, API keys):

1) Prerequisites: ensure `git`, `uv` (https://docs.astral.sh/uv/), and Docker are available.
   If Docker is unavailable, use a local Postgres with the `vector` and `pg_trgm` extensions
   and set DATABASE_URL accordingly (default Docker URL uses host port 5433).

2) Clone and install:
   git clone https://github.com/dogfoodlab-io/git-rag.git "$INSTALL_DIR"
   cd "$INSTALL_DIR"
   cp .env.example .env
   Set in .env:
     VAULT_PATH=<my vault path>
     VAULT_BRANCH=main
     DATABASE_URL=postgresql://vault:vault@localhost:5433/git_rag
     EMBED_PROVIDER=local
     EMBED_MODEL=sentence-transformers/all-MiniLM-L6-v2
     EMBED_DIM=384
   (Optional API embeds instead of local:)
     EMBED_PROVIDER=openrouter
     EMBED_MODEL=openai/text-embedding-3-small
     EMBED_DIM=1536
     OPENROUTER_API_KEY=<ask me>
   uv sync
   docker compose up -d
   uv run git-rag init
   uv run git-rag index
   uv run git-rag hooks install
   uv run git-rag status   # confirm current=true

3) If the vault is not already a git repo on main:
   cd "$VAULT_PATH" && git init -b main && git add -A && git commit -m "initial vault"

4) Wire MCP for agents (stdio). Prefer Cursor project or user MCP config.

   Cursor — merge into ~/.cursor/mcp.json or the project's .cursor/mcp.json:
   {
     "mcpServers": {
       "git-rag": {
         "command": "uv",
         "args": ["run", "--directory", "INSTALL_DIR", "git-rag", "mcp"],
         "env": {
           "GIT_RAG_ENV_FILE": "INSTALL_DIR/.env"
         }
       }
     }
   }
   Replace INSTALL_DIR with the absolute path from step 2.

   Claude Desktop — same server block under mcpServers in
   ~/Library/Application Support/Claude/claude_desktop_config.json (macOS).

   Claude Code — register the same stdio server / or rely on CLI:
     uv run --directory INSTALL_DIR git-rag context "QUESTION"

5) After MCP is connected, use tools (never invent vault facts):
   - vault_context: primary — token-budgeted pack with citations
   - vault_search: ranked hits / filters
   - vault_note: full note by path
   - vault_related, vault_tags, vault_status
   Citation rules: every claim cites path + line range; if retrieval is empty, say so and
   do not answer from general knowledge as if it came from my notes.
   If vault_status shows current=false, run: uv run git-rag index

6) Verify with:
   uv run git-rag query "test" -k 3
   uv run git-rag status --json
   Confirm the MCP server starts: uv run git-rag mcp (stdio; leave running under the client)

Report back: install path, whether index is current, document/chunk counts, and the exact
MCP JSON you wrote (with absolute paths).

MCP-only (already installed)

If the tool is already cloned and indexed, paste this instead:

Configure git-rag as an MCP stdio server and use it for questions about my notes.

Install dir: <<<ABS_PATH_TO_git-rag_CLONE>>>
Env file:    <<<ABS_PATH_TO_git-rag_CLONE>>>/.env

1) Add MCP server "git-rag":
   command: uv
   args: ["run", "--directory", "<install dir>", "git-rag", "mcp"]
   env: { "GIT_RAG_ENV_FILE": "<install dir>/.env" }

   Cursor: ~/.cursor/mcp.json or .cursor/mcp.json
   Claude Desktop: claude_desktop_config.json → mcpServers

2) Restart the MCP host / reload servers.

3) For note questions, call vault_context first (then vault_search / vault_note as needed).
   Cite path + lines on every claim. If empty, say nothing was found — do not invent.
   If results look stale, check vault_status and run git-rag index.

How it works

commit on main ──> git hook ──> git diff --name-status -M <last_indexed>..HEAD
                                          │
                        ┌─────────────────┴─────────────────┐
                        │  A  new note        → chunk+embed │
                        │  M  edited note     → re-chunk    │
                        │  R  renamed/moved   → move row    │
                        │  D  deleted note    → cascade del │
                        └─────────────────┬─────────────────┘
                                          v
                        chunk hash lookup in embedding_cache
                                          v
                        embed only the chunks never seen before
                                          v
                     documents + chunks (HNSW vector + GIN tsvector)
                                          v
                   git-rag query / context  ──> vault-search skill / MCP

GitHub Actions can run the same git-rag index against a reachable DATABASE_URL when the vault repo is pushed.

The index always corresponds to a real commit. Content is read with git show <sha>:<path> rather than from the working tree, so an unsaved or half-written note never enters the index.

Why this design

Git is the change feed. git diff --name-status -M is the only thing that reliably reports renames as renames. Comparing filesystem mtimes cannot distinguish "moved" from "deleted and created", which would throw away and recompute every embedding in the note. The last indexed commit is stored in index_state, so the delta is exact and resumable.

Embeddings are cached by content hash, not by path. embedding_cache is keyed on (model, sha256(title + heading + chunk text)) and deliberately outlives the documents that reference it. Consequences:

• Moving a note between folders costs zero embeddings.
• Renaming a note costs zero embeddings unless the rename changes the note's derived title.
• Reorganising one note's headings only re-embeds the chunks that actually changed.
• Deleting and later restoring a note is free.

The hash includes the title and heading breadcrumb because both are part of the embedded text. Keying on body alone would serve a stale vector after a title change.

Retrieval is hybrid. A vector KNN and a Postgres full-text query run in parallel and are fused with Reciprocal Rank Fusion. Vector search alone misses exact identifiers (ticket numbers, people, acronyms); full-text alone misses paraphrase. Personal notes need both, because you rarely remember the words you originally wrote.

The filename is searchable, separately from the title. Every chunk stores a context breadcrumb of filename > title > heading, which feeds both the embedding and the tsvector. This matters because the two routinely disagree: a note named 2026-06-24 - RoA Doc Mgmt Review.md may have an H1 of Resourcing for Extension. Indexing only the H1 makes the note unfindable by the name its author actually remembers — and in Obsidian the filename is the primary identifier, since that is what wikilinks resolve against.

Every note is retrievable. Obsidian vaults are full of stubs whose entire content is a title or a single heading. Those produce no body chunks, so the chunker emits a synthetic filename+title+headings chunk. Without it, roughly 30% of a real vault is invisible to search.

Setup

Requires Docker and uv.

Install from PyPI

python -m pip install git-rag
git-rag --help

The distribution installs the git_rag Python package and the git-rag and git-rag-mcp console commands. For example:

import git_rag

print(git_rag.__version__)

For development from a clone, continue with the uv-based setup below.

cp .env.example .env      # then set VAULT_PATH
uv sync
docker compose up -d      # or: docker-compose up -d
uv run git-rag init
uv run git-rag index    # first run downloads the embedding model (~90 MB)

The vault must be a git repository with a main branch:

cd "/path/to/vault" && git init -b main && git add -A && git commit -m "initial vault"

Then install the triggers:

uv run git-rag hooks install

This installs four hooks into the vault's .git/hooks:

Indexing runs detached and behind a lock, so it never adds latency to a git command and concurrent hooks cannot race. Each run appends to .git-rag/index.log. Hooks no-op unless HEAD is the branch named in VAULT_BRANCH, so scratch-branch commits are never indexed.

Querying

uv run git-rag query "why did we pick Postgres" -k 8   # ranked hits with scores
uv run git-rag context "why did we pick Postgres"      # token-budgeted agent context
uv run git-rag note "Meetings/2026-02-11"              # one note in full
uv run git-rag related "Projects/Alpha.md"             # nearest notes
uv run git-rag tags                                    # topic overview
uv run git-rag status                                  # is the index current?

Useful flags: --mode vector|text|hybrid, --path "Meetings/", --tag project-x, --exclude "Archive/", --expand (include adjacent chunks), --json.

Agent access (MCP)

uv run git-rag mcp
# or: uv run git-rag-mcp

See examples/mcp.cursor.json and examples/mcp.claude-desktop.json.

Tools: vault_search, vault_context, vault_note, vault_related, vault_tags, vault_status.

.claude/skills/vault-search/SKILL.md teaches Claude Code when to reach for the vault and how to cite results. It prefers MCP tools when available, requires citations on every claim, and forbids answering from general knowledge when retrieval comes back empty.

GitHub Actions

Index updates on vault push via the reusable workflow (needs a reachable Postgres+pgvector DATABASE_URL secret — not your laptop Docker):

# in the vault repo — see examples/vault-repo-index.yml
jobs:
  index:
    uses: dogfoodlab-io/git-rag/.github/workflows/reusable-index.yml@main
    with:
      git_rag_repository: dogfoodlab-io/git-rag
    secrets:
      DATABASE_URL: ${{ secrets.DATABASE_URL }}

Copy examples/vault-repo-index.yml into the vault repo, or use the composite action under .github/actions/index. Prefer EMBED_PROVIDER=openai, openrouter, or voyage in CI for faster cold starts; cache ~/.cache if using local.

Configuration

OpenRouter

# .env
EMBED_PROVIDER=openrouter
EMBED_MODEL=openai/text-embedding-3-small
EMBED_DIM=1536
OPENROUTER_API_KEY=sk-or-...
# optional: EMBED_BASE_URL=https://openrouter.ai/api/v1

uv run git-rag init --reset && uv run git-rag index

Model IDs are OpenRouter slugs (e.g. openai/text-embedding-3-small). Match EMBED_DIM to the model (1536 for text-embedding-3-small unless you request fewer dimensions).

local is the default because personal notes should not need to leave the machine, and on-device embedding makes reindexing free. Switching provider or model changes vector geometry, so init refuses to mix and you must rebuild:

uv run git-rag init --reset && uv run git-rag index

Corporate TLS interception

If the model download fails with CERTIFICATE_VERIFY_FAILED, a proxy (Netskope, Zscaler) is re-signing TLS. Export the macOS trust store and point the config at it:

mkdir -p .git-rag
{ security find-certificate -a -p /System/Library/Keychains/SystemRootCertificates.keychain
  security find-certificate -a -p /Library/Keychains/System.keychain; } > .git-rag/ca-bundle.pem
echo "GIT_RAG_CA_BUNDLE=$PWD/.git-rag/ca-bundle.pem" >> .env

Tests

./tests/integration.sh

Drives a real git repo through assertions on its own throwaway database: add, edit, rename, cross-folder move, rename-plus-edit in one commit, delete, off-branch commits, rewritten history, ignore globs, embedding reuse, retrieval, and an installed git hook indexing a commit end-to-end. Set KEEP=1 to keep the fixture for inspection.

Schema

Indexes: HNSW (cosine) on chunks.embedding, GIN on chunks.tsv, GIN on documents.tags, trigram on documents.path. Chunks cascade-delete with their document.

Operational notes

• git-rag index --full rebuilds every note, ignoring content hashes. Use it after changing chunking or embedding logic; a plain index would consider everything unchanged.
• A full rebuild also prunes notes that no longer exist in the tree.
• git-rag status compares the indexed commit to the vault's HEAD, which is the fastest way to spot a hook that silently stopped firing.
• One malformed note is logged and skipped rather than aborting the run.