Skip to main content

wells-index

Fast structural repository indexer for Wells. Uses Tree-sitter for language parsing, SQLite for storage, and BLAKE3 for incremental hashing.

Install

Prebuilt wheels (Linux / macOS / Windows, Python 3.12 + 3.13) are on PyPI:

pip install wells-index

Note: the 0.1.0 wheels had a defect — files were indexed but zero symbols were extracted. Use 0.1.1 or later. Wells detects the stale core at runtime (/doctor) and self-repairs from the repo-bundled binaries.

Features

  • Multi-language symbol extraction — Python, JavaScript, TypeScript, Go, Rust, Java, C, C++
  • Incremental indexing — Only re-parses changed files (via BLAKE3 hashing)
  • Compressed storage — SQLite database with LZ4 compression
  • Fast queries — O(1) symbol lookups, reference finding, call site discovery
  • 98% token reduction — Compared to grep-based code retrieval
  • PyO3 bindings — Native Python extension for seamless integration

Building

Prerequisites

  • Rust 1.70+
  • Python 3.12+
  • maturin >= 1.7

Setup

  1. Clone the repository and navigate to the wells-index directory:

    cd Wells/wells-index
    
  2. Vendor tree-sitter grammar sources (one-time setup):

    The build system expects tree-sitter grammar C sources in grammars/<language>/src/:

    # Create grammar directories
    mkdir -p grammars/{python,javascript,typescript/typescript,go,rust,java,c,cpp}/src
    
    # Copy parser.c and scanner.c from official tree-sitter repos
    # Example for Python:
    curl https://raw.githubusercontent.com/tree-sitter/tree-sitter-python/master/src/parser.c \
        -o grammars/python/src/parser.c
    

    Or use git submodules:

    git submodule add https://github.com/tree-sitter/tree-sitter-python.git grammars/python
    git submodule add https://github.com/tree-sitter/tree-sitter-javascript.git grammars/javascript
    # ... etc for other languages
    
  3. Build the extension:

    maturin develop
    
  4. Verify installation:

    python -c "from wells_index import IndexEngine; print(IndexEngine.__doc__)"
    

Usage

Command Line

# Index the current directory
python -c "from wells_index import IndexEngine; e = IndexEngine('.'); e.index(); print(e.stats())"

Python API

from wells_index import IndexEngine

# Create indexer for workspace
engine = IndexEngine("/path/to/repo")

# Build/update index
stats = engine.index()
print(f"Indexed {stats['files_indexed']} files")

# Query the index
symbols = engine.find_symbol("MyClass")
for sym in symbols:
    print(f"{sym['file_path']}:{sym['start_line']} - {sym['name']} ({sym['kind']})")

# Find all references to a symbol
refs = engine.find_references("authenticate")
for ref in refs:
    print(f"{ref['file_path']}:{ref['start_line']}")

# Find all callers of a function
callers = engine.find_callers("process_request")
for caller in callers:
    print(f"{caller['file_path']}:{caller['start_line']} calls process_request")

# Prefix/substring search
results = engine.search_symbols("MyClass", limit=20)
for r in results:
    print(r)

# List all symbols in a file
symbols = engine.list_in_file("src/main.py")
for sym in symbols:
    print(f"  {sym['name']} ({sym['kind']})")

# Get repository stats
stats = engine.stats()
print(f"Total files: {stats['total_files']}")
print(f"Total symbols: {stats['total_symbols']}")
print(f"Total edges: {stats['total_edges']}")

# Clear index
engine.clear()

Index Format

The index is stored in .wells_index/index.db (relative to the workspace root):

  • Compressed: LZ4 compression applied on flush
  • Incremental: BLAKE3 file hashes skip unchanged files
  • Portable: SQLite database readable by standard tools

Grammar Support

Language Status Extension Tree-sitter Repo
Python ✓ .py tree-sitter-python
JavaScript ✓ .js, .mjs, .cjs tree-sitter-javascript
TypeScript ✓ .ts, .tsx tree-sitter-typescript
Go ✓ .go tree-sitter-go
Rust ✓ .rs tree-sitter-rust
Java ✓ .java tree-sitter-java
C ✓ .c, .h tree-sitter-c
C++ ✓ .cpp, .cc, .cxx, .hpp, .hh tree-sitter-cpp

Symbol Kinds

  • class — Class/struct/interface definition
  • function — Top-level function definition
  • method — Method within a class
  • variable — Variable/field definition
  • module — Module/file-level scope

Edge Kinds

  • calls — Function/method call
  • references — Symbol reference or usage
  • inherits — Class inheritance or trait impl
  • imports — Module import

Performance

On a typical large repository (10k+ files, 1M+ symbols):

  • Initial indexing: 5-10 seconds on modern hardware
  • Incremental update: <100ms for changed files
  • Query latency: <1ms for symbol lookups
  • Storage: ~10-20% of source code size

Architecture

  • Language detection: File extension-based with shebang fallback
  • Parsing: Tree-sitter C library (via Rust bindings)
  • Parallelism: rayon for multi-core file scanning and parsing
  • Storage: rusqlite with integer-mapped symbol names
  • Hashing: BLAKE3 for incremental change detection
  • Compression: LZ4 on database at rest

Known Limitations

  • No semantic analysis (only syntax-based extraction)
  • Symbol deduplication not yet implemented
  • Cross-file call resolution is name-based (not type-aware)

Development

Run tests:

maturin develop
pytest tests/ -v

Build in release mode:

maturin build --release

Releasing

Wheels are built and published by the repo's release-index.yml workflow (maturin on Linux/macOS/Windows × Python 3.12/3.13, PyPI trusted publishing):

# bump version in Cargo.toml + pyproject.toml + python/wells_index/__init__.py
git tag index-v0.1.2 && git push origin index-v0.1.2

The prebuilt .pyd files committed under python/wells_index/ are the no-toolchain fallback used by wells.bat and Wells' stale-core self-repair; refresh them from the built wheels when the Rust source changes.

License

MIT OR Apache-2.0

Metadata

Release files for wells-index 0.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distributions (wheels)

Table of built distributions (wheels) for wells-index 0.2.0
File Actions
wells_index-0.2.0-cp312-abi3-win_amd64.whl CPython 3.12 abi3 Windows x86-64 Details for wells_index-0.2.0-cp312-abi3-win_amd64.whl
wells_index-0.2.0-cp312-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl CPython 3.12 abi3 Linux glibc 2.17+ x86-64 Details for wells_index-0.2.0-cp312-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
wells_index-0.2.0-cp312-abi3-macosx_11_0_arm64.whl CPython 3.12 abi3 macOS 11.0+ ARM64 Details for wells_index-0.2.0-cp312-abi3-macosx_11_0_arm64.whl
wells_index-0.2.0-cp312-abi3-macosx_10_12_x86_64.whl CPython 3.12 abi3 macOS 10.12+ x86-64 Details for wells_index-0.2.0-cp312-abi3-macosx_10_12_x86_64.whl

Total release size: 7.1 MB

Release files / wells_index-0.2.0-cp312-abi3-win_amd64.whl

Download URL wells_index-0.2.0-cp312-abi3-win_amd64.whl
Size 1.7 MB
Tags CPython 3.12 Windows x86-64 abi3
SHA-256 checksum
How to use checksums
831633828204cbb7d589b146f3918f9a3dca56110e10987c78b979061f8d9668
BLAKE2b-256 checksum
How to use checksums
b29f1becc51bbfd18690358d941d2dc2dc42048cb5935754bb06df56a83c2375
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 20, 2026.

Transparency log

Release files / wells_index-0.2.0-cp312-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl

Download URL wells_index-0.2.0-cp312-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Size 1.9 MB
Tags CPython 3.12 Linux glibc 2.17+ x86-64 abi3
SHA-256 checksum
How to use checksums
f3c0217cdf3305ff7551c82429123c6250274df347bd5bd0c089d784028c9fc3
BLAKE2b-256 checksum
How to use checksums
d2370fe028f36636fdb33d2315dd86156435276669a075da1e62615ddadf49d8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 20, 2026.

Transparency log

Release files / wells_index-0.2.0-cp312-abi3-macosx_11_0_arm64.whl

Download URL wells_index-0.2.0-cp312-abi3-macosx_11_0_arm64.whl
Size 1.7 MB
Tags CPython 3.12 abi3 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
dce3bf1cb939bbd9a6b9c4c16e7a2003930a8a575bd6888cbe8ae51c3dcaff29
BLAKE2b-256 checksum
How to use checksums
69595ba0eba52b0f78ac460ab7bff6ac6bf7d121fdee5134c572173dcd193ece
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 20, 2026.

Transparency log

Release files / wells_index-0.2.0-cp312-abi3-macosx_10_12_x86_64.whl

Download URL wells_index-0.2.0-cp312-abi3-macosx_10_12_x86_64.whl
Size 1.8 MB
Tags CPython 3.12 abi3 macOS 10.12+ x86-64
SHA-256 checksum
How to use checksums
f9fc297fd0de3d5f6d1fd573bf3c20c6830b90d1dc202d494010ca75f01fe4a6
BLAKE2b-256 checksum
How to use checksums
362f315963c3e175f2490513a570a234c04450787fa212cf3be6d384f3a543e4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 20, 2026.

Transparency log

Security for wells-index 0.2.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

All 4 files in this release have been uploaded with provenance metadata.

Select from 4 files

Attestation for wells_index-0.2.0-cp312-abi3-win_amd64.whl

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 20, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page