CodeIndex Client

Python client for monitoring and indexing code repositories with semantic search.

Features

• File Monitoring: Watches code files for changes (create, modify, delete)
• Smart Filtering: Only indexes recognized source code files
• Automatic Retry: Retries failed operations with exponential backoff
• In-Memory Queue: Maintains pending operations during network issues
• Multi-Language Support: Supports 40+ programming languages

Installation

Install from the repository:

pip install -e .

Or with development dependencies:

pip install -e ".[dev]"

Usage

Start the Indexing Agent

codeindex agent \
    --project-id my-project \
    --path /path/to/your/repo \
    --server https://indexer.example.com \
    --token YOUR_AUTH_TOKEN

The agent will:

1. Index all existing code files
2. Monitor for file changes
3. Send updates to the Indexer API
4. Retry failed operations automatically

Check Server Health

codeindex health \
    --server https://indexer.example.com \
    --token YOUR_AUTH_TOKEN

Configuration

Environment Variables

• CODEINDEX_TOKEN: Authentication token (alternative to --token flag)
• CODEINDEX_DEBOUNCE_SECONDS: Debounce time for file changes (default: 0.5)
• CODEINDEX_MAX_RETRIES: Maximum retry attempts (default: 10)
• CODEINDEX_INITIAL_RETRY_DELAY: Initial retry delay in seconds (default: 1.0)
• CODEINDEX_MAX_RETRY_DELAY: Maximum retry delay in seconds (default: 60.0)
• CODEINDEX_RETRY_BACKOFF_FACTOR: Backoff multiplier (default: 2.0)

Supported File Extensions

The client automatically detects and indexes these file types:

• Python: .py, .pyw
• JavaScript/TypeScript: .js, .jsx, .ts, .tsx, .mjs, .cjs
• Go: .go
• Rust: .rs
• Java/Kotlin: .java, .kt, .kts
• C/C++: .c, .h, .cpp, .hpp, .cc, .hh, .cxx, .hxx
• C#: .cs
• PHP: .php
• Ruby: .rb
• Swift/Objective-C: .swift, .m, .mm
• Scala: .scala
• Shell: .sh, .bash, .zsh
• PowerShell: .ps1
• Perl: .pl, .pm
• Elixir: .ex, .exs
• Erlang: .erl
• Haskell: .hs
• Clojure: .clj, .cljs, .cljc
• Lua: .lua
• R: .R, .r
• Dart: .dart
• HTML/CSS: .html, .htm, .css, .scss, .sass, .less
• SQL: .sql
• Config: .yaml, .yml, .toml, .json
• Markdown: .md, .markdown

Ignored Directories

The following directories are automatically excluded from monitoring:

.git, .svn, .hg, node_modules, __pycache__, .pytest_cache, .mypy_cache, .tox, venv, .venv, env, .env, dist, build, target, bin, obj, .idea, .vscode, .DS_Store, coverage, .coverage, htmlcov

Examples

Index a Python project

codeindex agent \
    --project-id my-python-app \
    --path ~/projects/my-python-app \
    --server http://localhost:8080 \
    --token dev_token_123 \
    --verbose

Index with environment variable

export CODEINDEX_TOKEN=dev_token_123

codeindex agent \
    --project-id my-project \
    --path . \
    --server http://localhost:8080

Development

Run tests:

pytest

Run with coverage:

pytest --cov=codeindex --cov-report=html

How It Works

1. Initial Scan: When started, the agent scans all code files in the directory
2. Queuing: Files are added to an in-memory queue for processing
3. Processing: The queue processor sends files to the Indexer API
4. Monitoring: The file watcher detects changes and queues new operations
5. Retry Logic: Failed operations are retried with exponential backoff
6. Graceful Shutdown: Pressing Ctrl+C stops the agent cleanly

Troubleshooting

Server not responding

If the Indexer API is down, the agent will:

• Continue monitoring file changes
• Queue operations in memory
• Retry with exponential backoff
• Log warnings but not crash

High memory usage

If you have many pending operations:

• Check if the Indexer API is healthy
• Reduce the repository size or use .gitignore patterns
• Restart the agent to clear the queue

Files not being indexed

Check:

• File extension is in the supported list
• File is not in an ignored directory
• Run with --verbose flag to see detailed logs