ContextForge

Build bounded, reviewable repository context for coding agents.

ContextForge scans local repositories, builds deterministic structural maps, supports bounded task-aware discovery, and produces portable context packages and handoffs. It helps you decide what an external coding agent should see without giving ContextForge permission to edit source code or run arbitrary commands.

Quick start · CLI · Configuration · Wiki · Discussions · Contribution policy

Why ContextForge

• Deterministic repository inventory. Scan files with stable ordering, portable paths, hashes, language classification, ignore rules, and bounded reads.
• Reviewable selection. Choose exact files, directories, GitWildMatch patterns, or line ranges—or ask a configured model for a bounded suggestion.
• Local repository intelligence. Publish a usable structural Index v3.1 generation first, then optional grounded Semantic Cards and deterministic repository maps, all in immutable generations under .contextforge/index.
• Evidence-first retrieval. Rank exact paths and symbols before persisted BM25, then add bounded graph, centrality, diff, and Working Set signals. Centrality can refine relevant evidence but cannot admit a file by itself.
• Coverage-aware compilation. Track complementary task roles from candidate selection through materialization. A capsule reports effective sufficiency only after required evidence is actually present.
• Portable artifacts. Export legacy ContextPackage v1/TaskHandoff artifacts or a token-budgeted Context Capsule v2 and stable XML prompt.
• Explicit trust boundaries. ContextForge does not edit repository source, execute repository code, expose shell tools, or mutate Git state.
• Automation-friendly output. Structured results stay on stdout while normal progress and diagnostics stay on stderr; index jobs can opt into a pure schema-3 JSONL progress stream on stdout.

Representative workflow

flowchart LR
    R["Source identity"] --> C["CodeMaps"]
    C --> G["relationship graph"]
    G --> I["BM25 + graph retrieval"]
    I --> E["bounded evidence plan"]
    E --> P["MAP / SUMMARY / SLICE / FULL"]
    P --> X["Context Capsule v2"]

A model is optional for scanning, trees, manual context packages, and structural-only indexing:

contextforge scan .
contextforge index build . --provider none
contextforge context create . \
  --include pyproject.toml \
  --directory src/contextforge/context \
  --exclude "**/__init__.py" \
  --format markdown \
  --output context.md

Installation

ContextForge requires Python 3.12 or newer. Install the published distribution:

python -m pip install contextforge-repo

For an isolated command-line installation, use either tool manager:

pipx install contextforge-repo
# or
uv tool install contextforge-repo

The PyPI distribution is named contextforge-repo; the import package remains contextforge, and the installed commands remain contextforge and ctxf. The similarly named context-forge-cli distribution is a different, unaffiliated project.

To install a checked-out source tree instead:

git clone https://github.com/waterflane/ContextForge.git
cd ContextForge
python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install .

Windows PowerShell:

git clone https://github.com/waterflane/ContextForge.git
Set-Location ContextForge
python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install .

The installation provides equivalent contextforge and ctxf console commands. python -m contextforge is also supported.

Quick start

Inspect a repository without writing ContextForge state:

contextforge scan .
contextforge tree . --depth 2
contextforge context create . `
  --include 'pyproject.toml' `
  --directory 'src/contextforge/context' `
  --exclude '**/__init__.py' `
  --format json `
  --output 'context.json'
contextforge context inspect 'context.json'

Build a structural-only local index and inspect its status:

contextforge index build . --provider none
contextforge index status .
contextforge map .
contextforge map . --kind all --format json
contextforge context suggest . --task "Trace configuration precedence"
contextforge context create . --task "Trace configuration precedence" `
  --context-tokens 32768 --response-tokens 4096 `
  --format json --output capsule.json --prompt-output prompt.xml

Common use cases

For a compact model input, export numbered materialized source while keeping the full capsule for verification:

contextforge context create . --task "Explain the context budget calculation" \
  --planning off --compact-prompt --format json \
  --output capsule.json --prompt-output source.xml

The compact prompt contains paths, source ranges and code once, while the capsule retains hashes, evidence identities and sufficiency diagnostics. The same projection is available as render_capsule_source_context from contextforge.context. FunctionCatalogue from contextforge.intelligence offers source-bound function descriptions and deterministic function-level search over saved CodeMaps. These helpers do not replace verified graph dependencies, grounded semantic enrichment or compiler requirements. See the function lookup guide for runnable exact and descriptive queries and how to check their materialized source.

• create a compact review packet for an external coding agent;
• map a repository without sending source to a model;
• inspect stale, missing, or failed index records;
• discover likely entry points, tests, configuration, and dependencies for a task;
• preserve a validated handoff that can be reviewed without the original checkout;
• benchmark discovery quality and repeatability against versioned manifests.

Evidence quality and benchmark results

Candidate recall measures whether required files appear in retrieval; materialized recall measures whether those files actually reach the Capsule a downstream model receives. A candidate later dropped for budget or representation reasons is not materialized evidence.

Citation containment checks only that a citation is within supplied material. Semantic grounding is stricter: a blinded majority judge decides whether the material supports the assertion. Benchmark reports also record assertion-ID recall and lexical/identifier support; neither replaces grounding.

Token savings are headline results only when the quality gate passes: required file recall is at least 0.90, required range recall at least 0.85, citation validity is 1.0, and answer quality is not below the oracle. A failed gate is reported as quality_gate_failed; its apparent savings are excluded from the headline average. The 30% automatic allocation is a soft compilation ceiling, not a promised saving.

For one short exact file, the compiler can select a compact profile with a minimal snapshot envelope and verified usage information. It is used only when cheaper than the normal capsule while preserving verification rules. See the Index v3.1 architecture and migration guide for the full contracts.

The pinned real-repository gate uses a reviewed manifest with source ranges and evidence IDs. Run contextforge benchmark real-repositories --manifest benchmarks/real-repository-v31.json --source contextforge=PATH --source dsh-contextforge=PATH --source planup=PATH --source syncplayer=PATH with a local OpenAI-compatible model. It creates temporary clones, builds each index, and compares ordinary, oracle, and Capsule answers. The JSON report is saved under .contextforge/benchmark-results/; incomplete runs and failed quality gates remain red rather than contributing to headline savings.

CLI overview

Every command supports --help; run group help before using advanced or mutating operations.

Global diagnostic options are --log-level, --log-format, --log-file, repeatable --log-component, --no-log-file, --no-color, and -v/-vv. Detailed syntax, defaults, streams, side effects, mistakes, and examples are in the Wiki CLI reference. The local integration contract is documented in the bridge integration guide, with a runnable generic client.

Long model-backed index jobs can stop issuing new work with --fail-fast or --max-failures N. With --fail-on-error, ContextForge finishes the workload, publishes partial semantic results, then exits with an error listing failed files. Hosts that launch the CLI can consume full ProgressEvent schema 3 objects with --progress jsonl:

contextforge index update . --provider openai-compatible \
  --model exact/model-id --progress jsonl --max-failures 3 \
  --semantic-scope all --semantic-max-requests 100000 \
  --semantic-max-input-tokens 100000000

Configuration

Project configuration is closed, versioned TOML. Resolution order is:

1. command-line option;
2. supported CONTEXTFORGE_* environment variable;
3. .contextforge/config.local.toml;
4. .contextforge/config.toml, or an explicit --config PATH;
5. built-in default.

The primary supported environment variables are:

• CONTEXTFORGE_MODEL_CONTEXT_WINDOW;
• CONTEXTFORGE_MODEL_CONNECT_TIMEOUT;
• CONTEXTFORGE_MODEL_READ_TIMEOUT;
• CONTEXTFORGE_MODEL_OPERATION_TIMEOUT;
• CONTEXTFORGE_JSON_REPAIR_ATTEMPTS;
• CONTEXTFORGE_LOG_LEVEL, CONTEXTFORGE_LOG_FORMAT, CONTEXTFORGE_LOG_FILE, and CONTEXTFORGE_LOG_COMPONENTS.

The default provider is local Ollama at http://127.0.0.1:11434/api/chat using model qwen2.5-coder:7b. Use --provider none for structural-only indexing. The openai-compatible provider and its lmstudio CLI alias require an exact model ID and a suitable base_url. To use the signed-in Codex CLI for model-backed indexing and planning, select it explicitly in .contextforge/config.toml:

[models]
provider = "codex"
model = "gpt-5.6-sol"
local_only = false
external_data_policy = "allow_repository"

Run codex login with ChatGPT first. ContextForge sends source through codex exec in an isolated read-only workspace; it does not read login tokens or fall back to a different provider. Code/test files retain structural search tags if model analysis is incomplete, and index update resumes missing function descriptions.

Model-backed discovery requires the configured provider to be running with the named model available. The requested context_window defaults to 16,384 tokens. If a server reports a smaller limit, callable analysis retries without external callee code before declaring a full-file overflow; diagnostics show the server-reported limit. Inspect the resolved policy before a long run with contextforge diagnostics provider PATH.

Credential configuration stores only the name of an environment variable in credential_env; the credential value is resolved at request time. See the configuration guide and Wiki configuration reference.

Retrieval and legacy discovery

With an active Index v3.1 generation, context suggest uses deterministic CandidateCard retrieval by default. Exact path, qualified-symbol, symbol, and source-identifier groups precede approximate scores. Approximate ranking uses persisted BM25 plus bounded graph proximity, centrality, current-diff, and Working Set signals. --planning auto|off|required controls bounded model-assisted evidence planning; auto safely falls back to deterministic selection and required fails when no validated plan is available. The legacy --rerank/--no-rerank flags remain compatibility aliases. Working Set and current-diff membership are retained in each CandidateCard's provenance. Automatic compilation requires an exact, lexical/grounded, graph, diff, or Working Set signal; centrality alone is not sufficient.

In model-assisted mode the provider may use at most three bounded rounds of search, symbol, graph, and map actions before finalize. Actions only expand verified ContextForge candidates and evidence; they cannot invent paths, symbols, ranges, or source facts. Use --planning-rounds N to lower the project ceiling for one command. Invalid, stale, incomplete, or over-budget plans are discarded as a whole in auto mode and replaced by deterministic complementary selection.

--legacy-discovery retains the former model-assisted discovery flow during deprecation. Its modes remain:

• Fresh builds current structural evidence in memory and does not load persisted semantic records or repository maps.
• Indexed requires a readable active index and uses current indexed structure, semantics, and maps.
• Hybrid is the default. It starts with current index evidence, fills structural gaps from the live snapshot, and explicitly falls back to fresh structure when no valid index exists.

All successful selections are verified against current source identities. Model-backed runs can produce different valid selections; ContextForge claims deterministic rendering for the same validated result, not deterministic model behavior. See Discovery output and benchmarks.

Context package generation

Manual packages use explicit selectors. With no include selector, all selectable snapshot files are included up to the configured limits:

contextforge context create . --include README.md --format markdown
contextforge context create . --directory src --exclude "**/__init__.py"
contextforge context create . --glob "tests/test_*.py" --no-include-tree
contextforge context create . \
  --include pyproject.toml \
  --include-lines pyproject.toml:1-24 \
  --format json

Task-based creation uses Context Capsule v2 by default when Index v3.1 is active:

contextforge context suggest . \
  --task "Trace configuration precedence" \
  --working-file src/contextforge/project_config.py \
  --planning auto \
  --planning-rounds 3 \
  --format markdown

contextforge context create . \
  --task "Trace configuration precedence" \
  --working-file src/contextforge/project_config.py \
  --context-tokens 32768 \
  --history-tokens 4000 \
  --response-tokens 4096 \
  --git-diff working \
  --format json \
  --output capsule.json \
  --prompt-output prompt.xml

Use --legacy-discovery for suggestion and --legacy-handoff for automatic creation to retain the deprecated flow. Manual context create without a task is unchanged.

Fully automatic task material uses 30% of the available context budget as a ceiling, not a target to fill: compilation stops when the evidence plan is covered and no candidate adds useful coverage. Explicit Working Set files/ranges, pinned FULL files, and required Git material may exceed that soft ceiling, but never the hard budget. A planner may give its suggested representation a bounded 10% utility bonus; freshness, range, FULL, and budget checks remain authoritative.

context suggest does not write source or index state, but current diagnostics policy may write a safe summary under .contextforge/runs. Output artifacts are written atomically; existing destinations require --force where that option is available. In-repository package, capsule, and prompt outputs are registered by path and SHA-256 in .contextforge/generated-artifacts.json; the scanner skips them only while that digest still matches. Editing an output makes it a normal source file again. Outputs outside the repository are not registered, and no filename or content heuristic is used.

Benchmark workflow

contextforge benchmark discovery 'C:\Repositories' `
  --tasks '.\benchmarks\discovery.json' `
  --modes 'fresh,indexed,hybrid' `
  --repeat 3 `
  --format json `
  --output '.\benchmark-report.json'

The runner is repository/index read-only, disables configured file logging, and records complete, failed, and cancelled runs in the result. Exit code 3 means the command produced a complete benchmark report containing at least one task, expectation, or budget failure. Do not discard stdout or the requested output file when handling that code. Every run remains bounded by manifest limits, provider retry limits, operation timeouts, and the configured context window. For Index v3.1 answer regressions, the ordinary baseline is the complete set of required/working files a client would otherwise send, while the manual range oracle is used only as a quality reference. The ContextForge and oracle answers use identical model settings and citation schemas. Three blinded groundedness votes check the ContextForge answer against only its materialized ranges. Offline indexing, planner, final-answer, judge, estimated-input, provider-input, HTTP-call, and latency costs are reported separately. The pinned real-repository benchmark accepts --provider codex --model-id ID --allow-repository-code for an explicitly authorized Codex run. Its build report separates tagged files, model-tagged files, described functions, and file_only coverage. Savings exclude incomplete paired answers.

Output formats and streams

• scans: table, json;
• trees: text, markdown, json;
• suggestions: text, compatibility alias table, markdown, json;
• context packages: markdown, json;
• index status and diagnostics: table, json;
• discovery benchmarks: text, markdown, json.

When no output path is supplied, the selected result is written to stdout. Progress, logs, and errors use stderr, preserving parseable JSON stdout. Some commands print a confirmation to stdout after writing a file; benchmark output files are the exception and leave stdout empty.

Common process exit codes are 0 for success, 1 for operational failure, 2 for invalid usage or configuration, and 130 for cancellation. Exit code 3 has command-specific meaning: unreadable entries with scan --fail-on-error, or a completed discovery benchmark with regression failures.

Architecture

ContextForge is a typed Python modular monolith. Core application and domain logic remain independent from Typer, FastAPI, model-provider implementations, storage adapters, and editor integrations. The scanner creates a verified snapshot; intelligence publishes CodeMaps, graph, grounded cards, maps, and retrieval postings; the context compiler materializes a hard-budgeted Capsule v2; CLI, Bridge, HTTP, and MCP are thin interfaces.

Read the architecture overview for dependency boundaries and the security policy for trust and path-safety details.

Documentation

• CLI logging and diagnostics
• Configuration
• Discovery and benchmarking
• Index v3.1 migration
• Development
• Troubleshooting
• Architecture notes
• Index v3.1 retrieval and Context Capsule compiler
• Complete GitHub Wiki

The Wiki is maintained in its separate GitHub Wiki repository. A prepared local wiki/ workspace is intentionally ignored by the main repository.

Development setup

python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -e ".[dev]"
python -m ruff check .
python -m mypy src
python -m pytest

Build validation and owner-only publication steps are in the release checklist. Release publication is an owner-triggered workflow protected by GitHub environments and PyPI OIDC.

Project status

ContextForge is pre-alpha and under active solo-maintainer development. Manual scanning, trees, context packages, local indexing, diagnostics, and read-only MCP are implemented. Semantic enrichment and optional evidence planning depend on the configured provider and its structured-output behavior. Remote MCP transport, autonomous source edits, shell/process tools, embeddings, IDE extensions, and coding-agent orchestration are not implemented.

Support and contributions

Use the Q&A category in GitHub Discussions for usage and support questions. Use GitHub Issues for reproducible bugs and focused feature suggestions. Do not post secrets, repository source, full prompts, or private logs in either place.

External contributions are welcome through a fork and pull request into dev. Discuss large changes in an Issue before implementation. Merges require owner approval and passing CI; contributors do not need direct repository write access. See CONTRIBUTING.md.

License

ContextForge 0.4.2 and later are licensed under the Apache License 2.0. Earlier tagged releases remain available under the license included in those release snapshots. See NOTICE for the project attribution notice.