CodeCortex

Adaptive semantic intelligence engine for large-scale code understanding.

Run codecortex --version for the installed version — see CHANGELOG.md for release history.

CodeCortex transforms source code into a living, queryable semantic graph — combining Code Property Graphs, real embedding models, approximate nearest-neighbor retrieval, intelligent graph traversal, and an interactive human-facing visualization layer into a unified platform purpose-built for production-scale repositories.

What CodeCortex Is

CodeCortex is not a linter, not a static analyzer, and not a dependency graph visualizer. It is a semantic intelligence infrastructure layer that answers questions like:

• Which components are structurally central to this codebase?
• What is the full blast radius if UserService.authenticate changes?
• Which functions are semantically similar to "authentication middleware"?
• What does the dataflow look like through this API endpoint?
• How are these modules architecturally coupled?

It ingests multi-language source code and produces a compressed, semantically-enriched graph that supports sub-20 ms retrieval across repositories of 100k+ nodes — and renders as a fully interactive visual graph for human exploration.

Core Architecture

Source Files (16 languages — see Supported Languages)
        │
        ▼
┌───────────────────────────────────┐
│   Tier 1 — Semantic Index         │  LSP · SCIP · Jedi · Tree-sitter
│   Symbols · APIs · Imports        │  ~O(files) build time
└──────────────────┬────────────────┘
                   │
                   ▼
┌───────────────────────────────────┐
│   Tier 2 — Hybrid Code Property   │  AST + CFG + DFG + Endpoints
│   Graph  (pruned, bounded)        │  60–80% node reduction
└──────────────────┬────────────────┘
                   │
                   ▼
┌───────────────────────────────────┐
│   Tier 3 — Embedding Layer        │  CodeBERT · local models · OpenAI
│   FAISS IVF · HNSW · cosine ANN  │  Batched · incremental · cached
└──────────────────┬────────────────┘
                   │
                   ▼
┌───────────────────────────────────┐
│   Clustering & Community          │  Louvain · HDBSCAN · adaptive
│   Semantic group labeling         │  Cluster confidence scoring
└──────────────────┬────────────────┘
                   │
                   ▼
┌───────────────────────────────────┐
│   Traversal & Ranking             │  Beam BFS · Personalized PageRank
│   Semantic-aware path scoring     │  Impact radius · Taint flow
└──────────────────┬────────────────┘
                   │
             ┌─────┴──────┐
             ▼            ▼
┌────────────────┐  ┌──────────────────────┐
│   Retrieval    │  │  Graph Visualization  │
│   Layer        │  │  Interactive Explorer │
│   AI-ready     │  │  Human-centric UI     │
│   context      │  │  Zero pipeline impact │
└────────────────┘  └──────────────────────┘

Supported Languages

Every grammar ships as a wheel, so parsing works offline. --languages accepts a language name, alias or extension (py, rust, cs, tsx).

CFG/DFG passes are tuned for Python, JavaScript/TypeScript and PHP; other languages get the structural graph (definitions, calls, imports, inheritance).

Key Design Decisions

Major Capabilities

Semantic Retrieval

Natural language queries return ranked, structurally-aware results. Results are fused from embedding similarity, graph centrality, and cluster membership.

from codecortex import CodeCortexPipeline

pipeline = CodeCortexPipeline.from_directory("/path/to/repo")
context = pipeline.query("authentication middleware", max_tokens=3000)
print(context)  # AI-ready context string

Impact Analysis

Trace the full blast radius of any change — direct callers, transitive dependents, affected tests, and cross-file propagation.

impact = pipeline.impact("UserService.authenticate", max_depth=4)
print(impact.to_dict())
# {root, direct_callers, affected_files, affected_tests, transitive_affected}

Hotspot Detection

Identify structurally critical nodes by composite centrality: PageRank, betweenness, fan-in/fan-out, eigenvector.

hotspots = pipeline.top_nodes(n=20)

Interactive Graph Visualization

Launch a full-featured browser-based graph explorer. Navigate the codebase visually, inspect semantic clusters, trace impact paths, and identify hotspots — all without modifying a line of pipeline code.

codecortex visualize /path/to/repo
# Opens http://localhost:7979 in the browser

Incremental Indexing

codecortex query and explain load the saved index from .codecortex/ and re-parse only files that changed since it was written. From Python, reindex a single changed file without rebuilding the full graph:

pipeline.reindex_file("/path/to/repo/auth/service.py")

Interactive Graph Explorer

The visualization layer is a purpose-built human interface on top of the semantic graph. It runs as a local web server and opens automatically in the browser.

Features

Launch the Explorer

# Basic launch
codecortex visualize .

# Point at any repository
codecortex visualize /path/to/your/repo

# Custom port, no auto-open
codecortex visualize . --port 8080 --no-open

# Limit to specific languages
codecortex visualize . --languages py ts

What You Can Do in the Explorer

Architecture discovery — use the Hierarchy layout to see module structure from top to bottom.

Hotspot review — click "Hotspots" in the header to highlight the top 10% most-connected nodes. These are your highest-risk change points.

Cluster exploration — click a semantic cluster chip to isolate one community and understand what it does.

Impact tracing — click any node, inspect its connections in the detail panel, and jump through the call chain.

Dependency audit — filter to IMPORTS_FROM edges only to map inter-module dependencies cleanly.

Onboarding — point a new team member at the graph to let them explore the codebase structure visually before reading a single line of code.

Repository Structure

codecortex/                 Main package + unified CLI
  __init__.py               Public API surface
  cli.py                    Unified CLI (index / query / explain / install / visualize)
  config.py                 YAML config loader
  index.py                  Index subcommand
  query.py                  Query subcommand
  explain.py                Explain subcommand
  integrate.py              install / uninstall for AI assistants
  hook.py                   Claude Code PreToolUse hook (hook-guard)

  core/                     Shared types and parser framework
    types.py                NodeKind, EdgeKind, NodeInfo, EdgeInfo
    parser_framework.py     LanguageParser ABC + ParserRegistry
    parsers/                Language parsers (16 languages)
      registry.py           The one list of parsers
      treesitter_parser.py  Table-driven tree-sitter parser
      languages.py          Go, Rust, Java, C#, C, C++, Ruby, Kotlin, …
      python_parser.py      javascript_parser.py  typescript_parser.py  php_parser.py

  graph/                    Code Property Graph
    schema.py               CPGNode, CPGEdge
    graph_store.py          SQLite-backed persistent store
    cpg_builder.py          AST → CPG ingestion
    cfg_builder.py          Control Flow Graph
    dfg_builder.py          Data Flow Graph
    endpoint_detector.py    HTTP API / route detection
    graph_pruner.py         Semantic node filtering (60–80% reduction)

  indexing/                 Semantic symbol indexing
    semantic_index.py       In-memory symbol lookup
    symbol_resolver.py      Cross-file name resolution
    symbol_cache.py         SQLite persistent symbol table
    reference_graph.py      Cross-file occurrence store
    jedi_provider.py        Python semantic analysis (Jedi)
    lsp_provider.py         JSON-RPC LSP client
    scip_provider.py        SCIP index reader
    graph_enricher.py       Upgrades unresolved CALLS edges

  embeddings/               Vector embedding layer
    code_embedder.py        Code → text normalization + rich context
    embedding_pipeline.py   CPG → FAISS index (batch, incremental)
    embedding_store.py      FAISS wrapper with cosine similarity
    provider_factory.py     lexical | local | openai | stub providers

  clustering/               Semantic community detection
    semantic_clusters.py    HDBSCAN + KMeans fallback
    louvain_clusterer.py    Graph-based Louvain community detection
    cluster_labeler.py      Token frequency + module dominance labels
    cluster_quality.py      Silhouette, cohesion, separation metrics
    cluster_store.py        SQLite cluster persistence

  traversal/                Graph traversal engine
    traversal_engine.py     AdjacencyCache, BFS, DFS, TaintFlow
    beam_traversal.py       Beam search with semantic scoring
    query_optimizer.py      Query plan optimization

  ranking/                  Centrality and ranking
    centrality_engine.py    PageRank, betweenness, closeness, eigenvector
    ppr_engine.py           Personalized PageRank (query-relative ranking)

  retrieval/                Retrieval orchestration
    retrieval_layer.py      Hybrid semantic + structural retrieval

  pipeline/                 End-to-end orchestration
    context_builder.py      CodeCortexPipeline (build → query → impact)
    state.py                Saved per-project index (.codecortex/)

  performance/              Incremental performance optimizations
    graph_snapshot.py       JSON graph persistence + mtime delta
    parallel_indexer.py     ThreadPoolExecutor multi-file indexing
    delta_embedder.py       Mtime-driven incremental re-embedding

  visualization/            Interactive graph explorer (human-facing)
    __init__.py
    graph_exporter.py       Pipeline state → visualization JSON
    server.py               Local HTTP server + browser launcher
    static/
      index.html            Full interactive graph UI (Cytoscape.js)

tests/                      Test suite
  test_phase1_core.py       … through test_phase10_performance.py
  test_languages.py         Per-language extraction
  test_saved_index.py       Saved index, install, hook
  benchmarks/               Performance benchmarks
    benchmark_runner.py     Composite score benchmark
    bench_v01.py            Stage-wise evaluation
    bench_v02.py            Per-phase architectural evaluation

codecortex.yaml             Default configuration

Use with Your AI Assistant

CodeCortex is meant to be asked, not loaded: build the graph once, and the assistant queries it when a question needs it instead of grepping and reading files.

pipx install codecortex-engine

cd your-project
codecortex install          # writes the assistant instructions (Claude Code by default)
codecortex index .          # builds the graph once, saved to .codecortex/

codecortex install adds a short, marked ## codecortex block to CLAUDE.md, a codecortex skill under .claude/skills/, and a PreToolUse hook in .claude/settings.json. When the project has a saved index and the assistant is about to grep or read a source file, the hook adds a one-line note that codecortex query / explain can answer more cheaply. It never blocks a tool, speaks at most once per session for searches and once for reads, and takes ~0.1 s. Skip it with --no-hooks. Use --platform agents (AGENTS.md: Codex, OpenCode, Amp, Copilot), --platform cursor, --platform gemini, or --platform all. The block tells the assistant to:

Each call reuses .codecortex/ (graph, ranking, clusters and search vectors) and re-parses and re-embeds only files whose size or mtime changed, so there is nothing to keep running and nothing to remember to refresh. On a 636-file repository a query or explain takes about a second. codecortex index . rebuilds everything (centrality and clusters are reused across incremental refreshes and recomputed by a full index). .codecortex/ ignores itself, so it never appears in git status.

codecortex uninstall removes exactly the block and files it added (--purge also deletes .codecortex/).

The default lexical embedding backend matches query words against symbol names, kinds and modules with no model download; set embedding_backend: local in codecortex.yaml (with the embeddings extra) for model-based semantic search.

Installation

Prerequisites

• Python 3.10+
• pip / pip3

macOS note: Homebrew Python enforces PEP 668 and blocks global pip install. Always install into a virtual environment (step 2 below). If pip is not found, use pip3 or python3 -m pip instead.

Local Development Setup

# 1. Clone the repository
git clone https://github.com/NafeesMansoor/codecortex.git
cd codecortex

# 2. Create a virtual environment (required on macOS Homebrew Python)
python3 -m venv .venv
source .venv/bin/activate         # Windows: .venv\Scripts\activate

# 3. Install core dependencies
pip install -r requirements.txt

# 4. Install in editable mode
pip install -e .

# 5. Optional: model embeddings + full semantic stack
pip install -e ".[all]"

# 6. Verify
pytest tests/ --ignore=tests/benchmarks
codecortex --help

Production Setup

CodeCortex is distributed through GitHub Releases; install a specific, immutable version rather than a branch.

For the CLI, install it as a tool. pipx (or uv tool) gives CodeCortex its own virtualenv and puts only the codecortex command on your PATH, so its dependencies never mix with a project's. To use the Python API, pip install it into your application's environment instead; everything lives under the single codecortex package, so it adds no other top-level names.

# Recommended: isolated tool install (~100 MB, all 16 grammars included)
pipx install codecortex-engine
# or: uv tool install codecortex-engine
# or a tagged release from GitHub:
#     pipx install "codecortex-engine @ git+https://github.com/NafeesMansoor/CodeCortex.git@v2.0.1"

# Optional model embeddings (sentence-transformers, torch, faiss) — large
pipx install "codecortex-engine[embeddings]"

# Or from a release artifact
pipx install ./codecortex_engine-2.0.1-py3-none-any.whl

# Verify
codecortex health

To upgrade a tool install, run pipx upgrade codecortex-engine (or uv tool upgrade codecortex-engine). A tool installed before 2.0.1 was named codecortex; replace it with pipx uninstall codecortex && pipx install codecortex-engine.

Environment Variables

Versioning & Updates

CodeCortex follows Semantic Versioning, with the version defined once in codecortex/version.py and read from there by the packaging metadata.

codecortex --version          # CodeCortex 2.0.1
codecortex version --check    # is a newer release available?
codecortex update             # back up, install, migrate, verify, roll back on failure
codecortex update --rollback  # restore the previous version
codecortex health             # verify this installation

Updates detect how CodeCortex was installed (git checkout or pip package), move to an immutable release tag, preserve configuration, rebuild stale index caches, and verify the result in a fresh interpreter before committing to it. An update check never blocks startup and never sends information about your installation.

Upgrading from 1.1.0 or earlier

codecortex update ships with 1.2.0, so an installation older than that has no such command yet — those versions predate the updater entirely. Upgrade once by hand; every release after this one is reachable with codecortex update.

Check what you have, then pick the line matching how it was installed:

codecortex --version    # or: pip show codecortex  (codecortex-engine from 2.0.1)

Installed with pip:

# Installed 2.0.0 or earlier with pip? The package was renamed in 2.0.1;
# remove the old name first so both don't claim the same files.
pip uninstall -y codecortex
pip install codecortex-engine

Installed as a git checkout (git clone + pip install -e .):

cd /path/to/CodeCortex
git fetch --tags origin
git checkout v2.0.1        # commit or stash local changes first
pip install -e .

Then confirm, and re-index once:

codecortex --version       # CodeCortex 2.0.1
codecortex health          # expect Status: HEALTHY
codecortex index .         # index caches from older versions are rebuilt

Your codecortex.yaml and any .env are left untouched by both routes. Existing index caches carry no schema stamp, so the first run rebuilds them — nothing is lost, because every cache is derived from your source tree.

Anything older than 1.2.0 upgrades directly; there are no staged upgrades.

Applications embedding CodeCortex should declare the range they support:

from codecortex import require_version

require_version(minimum="1.1.0", below="2.0.0", consumer="my-service")

Full detail — channels, migrations, rollback, offline installs, and the release process — is in UPDATING.md. Release history is in CHANGELOG.md.

Quick Start

Index a Repository

# Fast structural index
codecortex index .

# Full CPG with embeddings and clustering
codecortex index . --mode cpg --enable-embeddings --enable-clustering

# Hybrid: Tier-1 at build time, CFG/DFG on demand
codecortex index . --on-demand-cfg --on-demand-dfg

# The index is saved to .codecortex/ for later queries; skip that with
codecortex index . --no-save

# Also export a JSON snapshot
codecortex index . --save-snapshot snapshots/graph.json

# Limit languages
codecortex index . --languages py ts

# Exclude extra project paths, on top of the built-in defaults
codecortex index . --languages php --exclude storage/ bootstrap/cache/

Dependency directories are skipped automatically. Virtualenvs (venv/, .venv/), node_modules/, vendor/, site-packages/, __pycache__/, build output and VCS metadata are pruned before the walker descends into them, and the repository .gitignore is honoured. Pass --no-default-excludes / --no-gitignore to opt out, or set exclude_dirs in codecortex.yaml to replace the built-in list.

Query

# Natural language query
codecortex query --query "authentication middleware" --target .

# Impact analysis
codecortex query --impact UserService.authenticate --target .

# One symbol: definition, callers, callees, base types
codecortex explain UserService.authenticate --target .

# Ignore the saved index and rebuild it
codecortex query --query "authentication middleware" --rebuild

# JSON output
codecortex query --query "HTTP routing" --json --target .

# With centrality ranking and larger context budget
codecortex query \
  --query "database connection pooling" \
  --enable-centrality-ranking \
  --max-tokens 5000 \
  --target .

Visualize

# Launch graph explorer (opens browser automatically)
codecortex visualize .

# Custom port
codecortex visualize /path/to/repo --port 8080

# Headless (server only, no browser open)
codecortex visualize . --no-open

# Limit to Python and TypeScript
codecortex visualize . --languages py ts

Python API

from codecortex import CodeCortexPipeline, PipelineConfig

config = PipelineConfig(
    embedding_backend="local",
    enable_clustering=True,
    use_louvain=True,
    use_beam_traversal=True,
    use_ppr=True,
    use_graph_pruning=True,
)

pipeline = CodeCortexPipeline.from_directory("/path/to/repo", config=config)

# Semantic query
context = pipeline.query("authentication flow", max_tokens=3000)

# Ranked results
results = pipeline.retrieve("database models")
for r in results:
    print(r.qualified_name, r.score)

# Impact analysis
impact = pipeline.impact("UserService.authenticate", max_depth=5)

# Hotspot detection
top = pipeline.top_nodes(n=20)

# Incremental update
pipeline.reindex_file("/path/to/repo/auth/service.py")

# Export for visualization
from codecortex.visualization import export_pipeline

graph_json = export_pipeline(pipeline, project_name="my-repo")

Configuration

codecortex.yaml in your project root:

codecortex:
  # Graph construction
  enable_cpg: true
  on_demand_cfg: false
  on_demand_dfg: false

  # Embedding provider: stub | local | openai
  embedding_backend: stub
  embedding_dimensions: 384

  # Clustering
  enable_clustering: true
  min_cluster_size: 5

  # Retrieval
  retrieval_top_k: 20
  token_budget: 4000
  min_semantic_score: 0.0

  # Traversal
  bfs_depth: 3
  max_nodes_per_query: 500

  # Centrality
  centrality_ranking: true

  # Paths excluded from indexing (relative to the indexed root)
  # Useful for Laravel/Node projects with large vendor/node_modules trees
  exclude_paths:
    - vendor/
    - node_modules/
    - storage/
    - bootstrap/cache/

Performance Modes

Lightweight Mode

Fast index, structural retrieval only. Builds in <1s for 100 files.

codecortex:
  enable_cpg: false
  enable_embeddings: false
  enable_clustering: false
  centrality_ranking: false

Balanced Mode

Default. Full CPG with stub embeddings. Builds in ~8s for 60 files.

codecortex:
  enable_cpg: true
  embedding_backend: stub
  enable_clustering: true

Deep Semantic Analysis Mode

Real code-aware embeddings, Louvain clustering, beam traversal, personalized PageRank. Best retrieval quality.

codecortex:
  enable_cpg: true
  embedding_backend: local
  enable_clustering: true
  use_louvain: true
  use_beam_traversal: true
  use_ppr: true
  use_graph_pruning: true

Requires: pip install "codecortex-engine[all]"

Enterprise Scale Mode

On-demand CPG expansion + graph pruning + parallel indexing + incremental updates. Designed for monorepos with 500k+ LOC.

codecortex:
  on_demand_cfg: true
  on_demand_dfg: true
  use_graph_pruning: true
  embedding_backend: openai
  embedding_dimensions: 3072

Benchmarking

# Run against any codebase
python tests/benchmarks/benchmark_runner.py --target /path/to/repo --phase "v1.0"

# Stage-wise evaluation
python tests/benchmarks/bench_v01.py --target /path/to/repo

# Per-feature architectural evaluation
python tests/benchmarks/bench_v02.py --target /path/to/repo

# Quiet mode (results appended to docs/results.md)
python tests/benchmarks/benchmark_runner.py --target . --quiet

Composite Score Formula

CodeCortex Score = 0.25×retrieval + 0.25×graph + 0.20×semantic + 0.15×efficiency + 0.15×ranking

Testing

# All 223 unit tests
pytest tests/ --ignore=tests/benchmarks

# With coverage
pytest tests/ --ignore=tests/benchmarks --cov=. --cov-report=term-missing

# Specific phase
pytest tests/test_phase3_cpg.py -v

Developer Experience

Debugging

CODECORTEX_LOG_LEVEL=DEBUG codecortex index .

Profiling

python -m cProfile -o profile.out tests/benchmarks/benchmark_runner.py --target . --quiet
python -c "import pstats; p=pstats.Stats('profile.out'); p.sort_stats('cumulative'); p.print_stats(20)"

Troubleshooting

No nodes in visualization / empty graph Ensure the target directory contains supported source files (.py, .js, .ts, .php). Check CODECORTEX_LOG_LEVEL=DEBUG codecortex visualize . for parser errors.

8000+ files indexed / graph polluted with framework internals (Laravel, Node) You are likely indexing vendor/ or node_modules/. Target the application directory directly or use --exclude:

codecortex index /path/to/laravel --languages php --exclude vendor/ node_modules/

Or add exclude_paths to codecortex.yaml (see Configuration above).

Clustering skipped — scikit-learn / hdbscan not installed Install the optional clustering extras:

pip install scikit-learn hdbscan umap-learn

HDBSCAN also requires real (non-stub) embeddings. Set embedding_backend: local.

Clustering produces 0 clusters HDBSCAN requires real (non-stub) embeddings. Set embedding_backend: local and install sentence-transformers.

Slow CPG build (>10s for 60 files) Enable graph pruning and on-demand CFG/DFG:

use_graph_pruning: true
on_demand_cfg: true
on_demand_dfg: true

Browser does not open Use --no-open and navigate manually to http://localhost:7979.

pip: command not found on macOS Use pip3 or python3 -m pip. macOS Homebrew Python does not symlink pip.

error: externally-managed-environment (PEP 668) Create a virtualenv first: python3 -m venv .venv && source .venv/bin/activate, then run pip install.

Extensibility

Add a Language Parser

Most languages need no parsing code: subclass TreeSitterParser, name the grammar node types, and add the class to PARSER_CLASSES in core/parsers/registry.py. The pipeline, the parallel indexer and the --languages filter all resolve parsers there.

from codecortex.core.parsers.treesitter_parser import TreeSitterParser
from codecortex.core.types import EdgeKind, NodeKind


class ZigParser(TreeSitterParser):
    language = "zig"
    extensions = [".zig"]
    grammar = "zig"  # add the grammar wheel to core/parsers/grammars.py
    function_types = frozenset({"function_declaration"})
    type_kinds = {"struct_declaration": NodeKind.STRUCT}
    call_types = frozenset({"call_expression"})
    import_types = frozenset({"builtin_function"})
    base_clauses: dict[str, EdgeKind] = {}

Override a hook (container, name_of, owner_of, callee_of, imports_of, bases_of, is_test) only where the grammar does not fit the tables, and add a case to tests/test_languages.py.

Add an Embedding Provider

from codecortex.embeddings.provider_factory import EmbeddingProvider
import numpy as np


class MyProvider(EmbeddingProvider):
    def embed(self, texts: list[str]) -> np.ndarray: ...

Export Graph Programmatically

from codecortex.visualization import export_pipeline
import json

pipeline = CodeCortexPipeline.from_directory(".")
data = export_pipeline(pipeline, project_name="my-project")
json.dump(data, open("graph.json", "w"), indent=2)

License

MIT

Author

Nafees Mansoor