cpnlookup

██████╗██████╗ ███╗   ██╗██╗      ██████╗  ██████╗ ██╗  ██╗██╗   ██╗██████╗
██╔════╝██╔══██╗████╗  ██║██║     ██╔═══██╗██╔═══██╗██║ ██╔╝██║   ██║██╔══██╗
██║     ██████╔╝██╔██╗ ██║██║     ██║   ██║██║   ██║█████╔╝ ██║   ██║██████╔╝
██║     ██╔═══╝ ██║╚██╗██║██║     ██║   ██║██║   ██║██╔═██╗ ██║   ██║██╔═══╝
╚██████╗██║     ██║ ╚████║███████╗╚██████╔╝╚██████╔╝██║  ██╗╚██████╔╝██║
 ╚═════╝╚═╝     ╚═╝  ╚═══╝╚══════╝ ╚═════╝  ╚═════╝ ╚═╝  ╚═╝ ╚═════╝ ╚═╝

Forged for faster indexing and smarter retrieval.

cpnlookup is a local-first CLI tool for indexing and querying GitHub repositories through natural language — without ever running git clone. It combines FAISS vector similarity search with a SQLite-backed static call graph to deliver a Hybrid RAG pipeline that understands both the semantic meaning and structural relationships of a codebase, entirely on your own machine.

This project is under active development. See Known Limitations and Roadmap for current status.

Features

Remote indexing — Fetches repository file trees and raw content via the GitHub API. Supports Python source files and Markdown documentation.

Logic-aware chunking — Uses Python's ast module to parse source files into meaningful units (functions and classes) rather than splitting on arbitrary character counts or line boundaries.

Static call graph construction — Performs static analysis to identify function invocations and stores a directed call graph in SQLite. Used at query time to expand retrieval context with structurally adjacent code.

Hybrid RAG retrieval — Combines FAISS vector similarity search with bidirectional call graph traversal. Top-k vector matches are retrieved first, then both their callees and callers are included to give the LLM full architectural context.

Incremental indexing — File content is hashed on index. Re-indexing only fetches, chunks, and embeds files that have actually changed. Unchanged files reuse their stored embedding vectors.

Write consistency — An index_status flag is written to SQLite at the start of every indexing run and cleared only on clean completion. Interrupted runs are detected and reported on next use.

Local inference — All inference is handled by a locally running Ollama instance (default: Mistral). No source code, queries, or embeddings leave your machine.

Global registry — A registry of all locally indexed repositories is maintained, allowing management of multiple indexes from a single interface.

Architecture

GitHub API  (file tree + SHAs — one call)
    |
    v
Change detection  (compare SHAs against stored file_hash in SQLite)
    |
    |-- Unchanged files  -->  reuse stored embedding BLOBs
    |-- New / changed    -->  fetch content  -->  chunk  -->  embed
    v
AST / Regex Chunker
    |-- Python   -->  ast module       -->  function / class chunks
    |-- Markdown -->  regex headers    -->  section chunks
    v
Embedding Model  (sentence-transformers/all-MiniLM-L6-v2)
    |
    +---> Embedding BLOBs stored in SQLite  (chunks.embedding)
    +---> FAISS index rebuilt from all BLOBs  (faiss.index)
    +---> Static call graph  (SQLite: graph_nodes, graph_edges)
    |
    v
Query pipeline
    |-- Vector search    (top-k FAISS matches via faiss_id)
    |-- Graph traversal  (callers + callees, bidirectional)
    v
Local LLM via Ollama  -->  Response

Each indexed repository produces a .cpnlookup/ directory containing index.db and faiss.index. A global registry at ~/.cpnlookup/registry.json tracks all indexes on the local system.

Installation

Prerequisites: Python 3.9+, Ollama running locally, and a GitHub Personal Access Token (Classic) with repo scope.

ollama pull mistral
pip install cpnlookup
lookup auth <your_github_token>

Usage

lookup                             # Welcome screen and quick start
lookup desc                        # Project info, version history, acknowledgements
lookup profile <username>          # Browse a user's repositories
lookup init <username>/<repo>      # Index a repository
lookup ask "How does X work?"      # Query the codebase in natural language
lookup functions                   # List all indexed functions and classes
lookup indexed                     # List all locally indexed repositories
lookup config <key> <value>        # Configure model or search depth
lookup clone <username>/<repo>     # Git clone a repository
lookup drop                        # Remove the local index for the current repository

Known Limitations

The following limitations are understood, documented for transparency, and tracked in the roadmap.

Call graph accuracy. The call graph is built via name-only static analysis. When a call to process() is detected, an edge is recorded to any indexed function named process regardless of module or class. Codebases with common function names will produce false edges. Dynamic dispatch — getattr, decorator-wrapped functions, factory patterns — is invisible to this analysis. The graph is a useful structural approximation, not a semantically precise call graph.

Language support. Only Python source files and Markdown documentation are supported. All other file types are skipped during indexing. Multi-language support via Tree-sitter is planned for V3.

Cold-start latency. Commands that invoke the embedding model still carry a load cost on first use per session. Lazy imports (shipped in v1.2.2) eliminated startup lag for non-model commands; a persistent model daemon to eliminate the remaining load cost is planned for a future v2.x release.

FAISS scalability. IndexFlatL2 performs exhaustive linear search. For repositories producing more than roughly 10,000 indexed chunks, query latency will degrade. Migration to IndexIVFFlat is planned for V3.

Roadmap

Version 2.0 — Forge ✦ current

Version 3.0 — Intelligence and Scalability

Version History

Privacy

No source code, query text, or generated embeddings are transmitted to any external service. Embedding generation runs locally via sentence-transformers. Inference is handled by your local Ollama instance. The GitHub API is used only to fetch the content of repositories you explicitly choose to index.

Acknowledgements

Developed by @projcjdevs. Special thanks to @2nieGarcia, @renzv-compsci, and @NIghtIngale340 for feedback and testing.