rubberduck-index 0.1.5

Local project indexer for RubberDuck Semantic Intelligence MCP

rubberduck-index

Local project indexer for RubberDuck Semantic Intelligence. Syncs your Python source code to the RubberDuck MCP server where it's analyzed as Code Property Graphs (CPGs) — enabling LLMs to query definitions, data flow, call chains, and more.

How It Works

Your Machine                          RubberDuck Server
┌──────────────┐    SHA-256 hashes    ┌──────────────────────┐
│  rubberduck  │───────────────────►  │  /index/manifest     │
│  -index      │◄──── need_upload ──  │  (diff check)        │
│              │                      │                      │
│  scanner.py  │──── changed files ─► │  /index/upload       │
│  syncer.py   │    (gzip tar/JSON)   │  (store + build CPG) │
│  watcher.py  │                      │                      │
└──────────────┘                      │  ProjectStore        │
                                      │  LocalProjects/      │
                                      │    u{user_id}/       │
                                      │      {project}/      │
                                      └──────────────────────┘

1. Scan — Finds Python files, computes SHA-256 hashes, respects .gitignore
2. Diff — Sends hashes to server; server replies with which files need uploading
3. Upload — Sends only changed files (JSON for small batches, gzip tar for large)
4. Build — Server stores files in user-scoped directories and builds CPG graphs
5. Query — LLMs use MCP tools (analyze_code, trace_variable, call_chain, etc.)

Install

pip install rubberduck-index

# With file watcher support (auto-sync on save):
pip install "rubberduck-index[watch]"

Requirements

• Python 3.9+
• requests (HTTP client) — installed automatically
• pathspec (.gitignore-compatible pattern matching) — installed automatically
• watchdog (optional — for watch command)

Quick Start

# 1. Initialize a project (first-time setup)
cd ~/my-python-project
rubberduck-index init
# Prompts for your auth token (one-time setup)
# Auto-detects project name from directory

# 2. Sync changes (incremental — only uploads what changed)
rubberduck-index sync

# 3. Watch for changes (auto-sync on file save)
rubberduck-index watch -d    # daemon mode (background)
rubberduck-index stop        # stop the daemon

# 4. Check status
rubberduck-index status      # local + server status
rubberduck-index list        # all your projects on server

Commands

init

Initialize a project directory for indexing. Creates .rubberduck/config.json, prompts for your token, scans files, and performs the first sync.

cd ~/my-project
rubberduck-index init [OPTIONS]

Flag	Description
--project	Project name on the server (default: current directory name)
--token	Bearer token (skips interactive prompt)
--server	Override server URL (default: https://semantic.rubberduck.com)
--directory	Project directory (default: current directory)
--include	File patterns to include (default: **/*.py)
--watch, -w	Start background watcher after init

Typical usage — no flags needed:

cd ~/my-project
rubberduck-index init
# Enter your token: ****
# Scanning project... 42 file(s)
# Uploading... Synced 42 file(s), 42 CPG graph(s) built

sync

Sync changed files to the server. Compares local hashes with server manifest and uploads only what's different.

rubberduck-index sync [--force]

Flag	Description
--force	Force full re-upload (ignore hash comparison)

watch

Watch for file changes and auto-sync. Uses OS-native file system events (FSEvents on macOS, inotify on Linux).

rubberduck-index watch [-d]

Flag	Description
--daemon, -d	Run in background. Stop with rubberduck-index stop.

Changes are debounced (default 500ms) and batched before uploading.

stop

Stop the background watcher daemon.

rubberduck-index stop

status

Show index status for the current project — local file count vs. server state.

rubberduck-index status

list

List all your indexed projects on the server.

rubberduck-index list

remove

Remove a project from the server (deletes synced files and CPG graphs).

rubberduck-index remove [--project NAME]

Authentication

The server requires a Bearer token. Get your token from your RubberDuck admin.

Three ways to provide your token (in priority order):

1. Interactive prompt (default on init):

   rubberduck-index init
   # Enter your token: ****
   # Token is saved to .rubberduck/config.json — you won't be asked again
   

2. Environment variable (any command):

   export RUBBERDUCK_TOKEN=your-token
   rubberduck-index init
   

3. --token flag (init only, for scripting):

   rubberduck-index init --token your-token
   

The token is saved in .rubberduck/config.json after init. All subsequent commands (sync, watch, status, etc.) read it from there automatically.

The .rubberduck/ directory is automatically added to .gitignore to prevent accidental token commits.

Configuration

All config lives in .rubberduck/config.json (created by init):

{
  "server": "https://semantic.rubberduck.com",
  "project": "my-app",
  "token": "your-bearer-token",
  "include": ["**/*.py"],
  "exclude_defaults": true,
  "max_file_size": 50000000,
  "watch_debounce_ms": 500
}

Field	Default	Description
server	https://semantic.rubberduck.com	MCP server URL
project	directory name	Project name on the server
token	—	Bearer token for auth
include	["**/*.py"]	Glob patterns for files to index
exclude_defaults	true	Use built-in exclude list (see below)
max_file_size	50000000 (50MB)	Skip files larger than this
watch_debounce_ms	500	Debounce interval for file watcher

Custom ignore patterns

Create .rubberduck/ignore with .gitignore-style patterns:

# Extra excludes
tests/fixtures/**
docs/**
*.generated.py

Built-in excludes

Always excluded regardless of config: __pycache__, .git, .venv, venv, node_modules, .tox, .mypy_cache, .pytest_cache, .eggs, *.egg-info, dist, build, .DS_Store, .hg, .svn, .env.

User Isolation

Each user's projects are stored in separate directories on the server. Two users can have projects with the same name without conflicts:

LocalProjects/
  u1/my-app/       ← User 1's "my-app"
  u2/my-app/       ← User 2's "my-app" (completely isolated)

The user_id is derived from your Bearer token — the CLI never needs to know it.

Server Limits

Limit	Default	Description
Max project size	2 GB	Total size of all files in a project
Max file size	50 MB	Individual file size limit
Compressed upload	2 GB	Max gzip-compressed body size

Examples

Index a Django project

cd ~/django-project
rubberduck-index init --watch

Index only specific directories

rubberduck-index init --include "src/**/*.py" "lib/**/*.py"

Use with Cursor / Claude Code

After indexing, tell the LLM in Cursor or Claude Code:

Load the project "my-app" and trace the data flow of the `request` variable.

The LLM will use MCP tools:

1. load_repo(repo="local/my-app") — loads CPG graphs
2. analyze_code(statement="trace data flow of request", graph_id="...") — queries the graph
3. Returns facts about definitions, assignments, and flow paths

Troubleshooting

"No .rubberduck/config.json found" Run rubberduck-index init first, or cd into the project directory.

"Token is required" Get your token from your RubberDuck admin, then run rubberduck-index init again.

"Request body too large" Your project exceeds 2GB. Use --include patterns to reduce file count.

"No matching files found" Check your include patterns in .rubberduck/config.json. Default is **/*.py.

Watcher not detecting changes Ensure watchdog is installed: pip install "rubberduck-index[watch]". On Linux, check inotify limits: sysctl fs.inotify.max_user_watches.