codedoc-ai

codedoc-ai is an incremental documentation engine that treats documentation as reusable state instead of regenerating it on every run. It reuses compatible completed records and recovery checkpoints, then sends only the remaining provider-bound work to the configured LLM.

codedoc-ai generates structured, incrementally reusable documentation for source repositories. It scans source locally, builds a deterministic dependency graph, sends only files that need analysis to a configured LLM, and writes JSON, Markdown, or both.

Contents

• Why codedoc-ai
• Architecture at a glance
• Core design principles
• Highlights
• Installation
• Quick start
• Using the output with AI assistants
• File contract
• Configuration
  • Configuration reference
  • Large files
  • Response correction
• Command-line options
• Providers and environment variables
• Inline instructions
• Output, incremental reuse, and ownership
• Crash recovery
• Planning and diagnostics
• Python API
  • Offline format conversion
  • Exported Python surface
• Troubleshooting

Why codedoc-ai

Most documentation generators run once and regenerate everything from scratch. codedoc-ai is built as a documentation memory layer for AI-assisted development: it treats documentation as durable, reusable state rather than throwaway output.

• It documents incrementally — compatible records are reused; new, forced, changed, and cache-incompatible records go through provider-free planning before any remaining work reaches the LLM.
• Its local assembly is deterministic — given the same completed records, graph, and run statistics, serialization produces the same bytes and adds no run-varying timestamp. Provider responses themselves are not assumed to be deterministic.
• It is crash-safe — a compatible interrupted ordinary run can reuse completed records, reducing repeated provider work.
• It validates final-output ownership before replacement, so a foreign final target is refused.
• It emits structured JSON and Markdown meant to be re-read by both humans and tools.

The result is documentation that can be refreshed without paying again for records that still satisfy the current reuse contract.

Architecture at a glance

flowchart TD
    A["Repository"] --> B["Scan - local, deterministic"]
    B --> C["Dependency graph, entry-based selection"]
    C --> D["Plan"]
    D -->|"unchanged (content hash + analysis identity)"| E["Reuse"]
    D -->|"compatible crash-recovery work"| F["Resume"]
    D -->|"remaining provider-bound work"| G["LLM"]
    E --> H["Write JSON / Markdown - per-artifact atomic, ownership-guarded"]
    F --> H
    G --> H

        repository
            │
            ▼
        scan (local, deterministic)
            │
            ▼
        dependency graph  →  entry-based selection
            │
            ▼
        plan  ─── reuse compatible records (content hash + analysis identity)
            │  ── resume compatible crash-recovery work
            │  ── send remaining provider-bound work to the LLM
            ▼
        write JSON / Markdown  (per-artifact atomic, ownership-guarded)

For the full phase-by-phase run lifecycle — including cache and recovery identity and the failure invariants — see RUN_FLOW.md.

Core design principles

These principles are enforced by the code, not aspirational:

• Deterministic local output assembly — identical completed records, graph, and run statistics serialize byte-identically; completed output carries no run-varying timestamp.
• Incremental by default — records whose content hash and complete cache identity remain reusable do not require another provider call.
• Fail-closed validation — unknown configuration, malformed instruction profiles, and foreign output files stop the run rather than being silently ignored.
• Centralized cache identity — every registered cache-identity key is checked by one reuse predicate; a mismatched record is not silently reused.
• Explicit ownership — final and recovery artifacts are replaced only after CodeDoc recognizes their ownership metadata.
• Readable output contracts — completed JSON and Markdown are structured for humans, scripts, and AI assistants.
• Compatibility by validation — CodeDoc reads recognized CodeDoc documents and recovery files, and refuses foreign or malformed artifacts.
• Config over CLI sprawl — deep customization lives in codedoc.config.json (codedoc --init-config); the command-line surface stays small.

Highlights

• Explicit or auto-detected entry files, with entry or all documentation scope.
• One initial combined call for an ordinary file in the default single mode; optional triple-agent analysis, retries, correction, and split planning have their own explicit accounting.
• Incremental reuse across JSON and Markdown based on source hashes and analysis identity.
• One fixed crash-recovery file that preserves completed work after interruption.
• Opt-in, provider-free complete-source division planning for oversized files.
• Config-only, validated instruction customization; non-default blocks that will reach provider work undergo the mandatory semantic review.
• OpenAI, Anthropic, Gemini, and OpenAI-compatible endpoint support.
• Read-only dry runs, paid-file caps, deterministic ownership guards, and stable CI-oriented exit codes.

Installation

pip install codedoc-ai

Quick start

Set a provider credential in the process environment, then run CodeDoc:

export OPENAI_API_KEY="your-key"
codedoc --entry src/main.py

PowerShell:

$env:OPENAI_API_KEY="your-key"
codedoc --entry src/main.py

The default output is codedoc/codedoc.json. Common alternatives:

codedoc --format md
codedoc --format both
codedoc --output docs/report.json
codedoc --documentation-scope all
codedoc --dry-run --max-files 25

An entry is optional: CodeDoc can recover it from the selected output, auto-detect a configured candidate, or document all scanned files when no candidate exists.

On later runs, CodeDoc reuses compatible owned records and replans new, changed, forced, or cache-incompatible files. If you switch between JSON and Markdown and the requested target does not yet exist, the exact opposite-format sibling is validated and used as the conversion source; reusable files require no provider call.

Empty and whitespace-only source files are identified during provider-free planning and skipped before their per-file documentation calls. They are not failures and incur no per-file provider charge. The run reports them through files_skipped_insufficient_source; if a skipped path had documentation in an older output, that stale record is omitted from the new completed output.

Using the output with AI assistants

CodeDoc output is designed to be pasted into, indexed by, or attached to AI coding assistants. For the strongest results:

• Use --format both when humans and tools will read the same run: Markdown is easier to skim, while JSON is easier for agents and scripts to query.
• Use --entry plus the default documentation_scope: entry for application flows, CLIs, services, and libraries with a clear starting point.
• Use --documentation-scope all for package indexes, SDK-style references, or repositories where there is no meaningful entry file.
• Run codedoc --dry-run --max-files N before a large or first-time run to see how many files would reach the provider.
• Keep codedoc/codedoc.json or codedoc/codedoc.md in a stable location so future runs and AI assistants can compare against the same documentation memory.
• Prefer analysis_mode: single for fast, economical documentation. Use analysis_mode: triple when dependency reasoning and role separation matter more than provider-call count.

File contract

CodeDoc uses a deliberately small set of persistent files:

The only persistent runtime configuration file is the exact project-root codedoc.config.json; defaults, environment values, CLI arguments, and in-memory overrides participate in normal resolution without adding another config file. Stable reuse reads only the selected final target, or its exact same-stem opposite-format sibling when the selected target is absent. Recovery uses only the fixed path above. CodeDoc does not search the output directory for alternate filenames or recovery candidates.

Temporary atomic-write siblings and writability probes are short-lived implementation details. They use unique names in the target directory, are cleaned up best-effort, and provide per-artifact atomic replacement rather than a cross-file transaction.

Output formats

Supplying --output docs/report.json or --output docs/report.md selects that exact file and infers the format from its extension. --format both writes two files and therefore requires an output directory.

Run metadata

Completed JSON documents contain one canonical run-metadata block: last_run. It includes the entry file, entry source, documentation scope, analysis mode, scanned/selected counts, and the exact partition of what happened in the most recent run.

If you are reading a CodeDoc document programmatically, use last_run for run metadata and files[] for per-file documentation.

The truthful last_run partition is:

files_selected == files_reused_unchanged
                + files_reused_identical_content
                + files_documented_by_llm
                + files_failed
                + files_unattempted
                + files_skipped_insufficient_source

The number of file records in the document may be less than last_run.files_selected when a first-run file failed, was locally skipped, or was unattempted before any prior record existed. files_resumed_from_recovery is an overlapping provenance count, not a separate partition category: a restored completed record can be classified as reused.

Ordinary identical-content reuse (files_reused_identical_content) is same-path only: a record documents exactly its own path, and CodeDoc never copies documentation from one path to a different path even when their content is byte-identical. An upgrade that changes the ordinary record identity regenerates every ordinary and truncate-strategy record once, which raises that run's total_calls_planned. max_planned_calls is evaluated against the complete selected run before usage accounting, provider creation, or any confirmation callback, so an exceeded cap blocks the entire run rather than throttling it — this one-time regeneration cannot be spread across runs under one unchanged cap. To span it across runs instead, select a smaller scope (a narrower --entry, additional --ignore paths, or a reduced --documentation-scope), or let individual file failures leave the remainder for a later run.

Every key beginning with _ inside a files[] record is internal to CodeDoc. External consumers should ignore those keys; they are persisted for cache, resume, and dependency reuse and are not a stable public contract.

Ownership markers

Completed JSON output is recognized only after strict CodeDoc document validation. Current versionless output requires a canonical last_run object containing entry_file; files, when present, must be a list of structured records. Supported legacy CodeDoc shapes remain readable through their own validation path. Foreign JSON is refused before overwrite.

Other CodeDoc-managed artifacts still need internal ownership metadata:

Completed JSON is the public machine-readable contract. Recovery JSON and Markdown carry their own internal ownership markers because they serve different runtime roles. JSON-to-Markdown, Markdown-to-JSON, embedded-view reads, and direct public-view rendering all apply the same recursive public projection before deriving visible or lightweight metadata, so private split provenance cannot influence or escape through a format conversion.

Configuration

Generate a complete, valid, editable configuration from the canonical defaults:

codedoc --init-config

This writes codedoc.config.json in the current directory. It includes every generated setting, api_key: null, and editable versionless single/triple instruction defaults (requested_shape syntax). Initialization does not copy a credential from the environment into the file.

Existing targets are refused unless --force is supplied. Forced regeneration validates the existing file and atomically replaces only prompt_profiles; every other top-level setting and value is preserved, and no backup is created. CodeDoc reads subsequent edits from this exact active file.

Configuration reference

Every key emitted by codedoc --init-config is listed below. Run codedoc --init-config for the exact defaults, value types, and editable instruction schema instead of copying a partial configuration.

supported_extensions is not emitted by --init-config. The loader accepts an explicit non-default value only as a backward-compatibility filter over the resolved extension_language_map, then derives supported_extensions from that map. New configurations should use extension_language_map. For list and map families, the base key replaces the default while matching _add and _remove keys adjust the resolved value.

Custom endpoints and endpoint-trust approval

Setting api_base_url (in codedoc.config.json, API_BASE_URL, or an in-memory config_overrides) routes every provider call for that run to that endpoint instead of the default provider endpoint — which means your API key, your project's source, and every prompt built from it are sent there once the run is authorized. Because a project-controlled config file could otherwise redirect this traffic with no runtime decision by the user, a non-empty api_base_url additionally requires runtime endpoint-trust approval from exactly two sources, evaluated before any credential is read:

• the --trust-api-base-url URL CLI option, or
• the CODEDOC_TRUST_API_BASE_URL environment variable.

--trust-api-base-url wins when both are set. Approval is compared to the configured api_base_url as a canonical identity — scheme, lowercased host, port (defaulted per scheme when omitted), and path with trailing slashes stripped — so the approval URL does not need to be byte-identical, only equivalent, and it must not carry a username, password, query string, or fragment (neither may api_base_url itself); any of those is rejected. Approving an endpoint while api_base_url is unset is also rejected, so a stale approval can never sit unnoticed in the environment.

codedoc.config.json and config_overrides can never satisfy this gate — setting a trust_api_base_url-shaped key there is rejected outright — so approval is always a deliberate, runtime, per-invocation decision, and it applies identically to --dry-run. A refused run creates no provider, sends no request, and never prints the raw endpoint URL, the approval URL, or any credential; it identifies the endpoint only by its canonical digest.

codedoc --entry src/main.py --trust-api-base-url http://localhost:11434/v1

Large files

Reuse and recovery boundary

The default large_file_strategy: truncate keeps the established head-plus-tail behavior. With analysis_mode: single, large_file_strategy: split supports both provider-free dry-run planning and real execution over the complete source. Any triple + split request fails configuration validation before scanning, recovery inspection, output-directory creation, prompt review, or provider construction; it never silently falls back to truncation.

single + split execution, completed-record reuse, and node-level recovery are fully supported; triple + split remains unavailable. An exactly compatible same-path completed split record is reused without provider construction, review calls, documentation calls, partial writes, or paid-cap usage. Cross-path identical-content split reuse remains unavailable because the split plan and completed identity are path-bound. An explicit force bypasses reuse and recovery for that path while preserving prior stable output and recovery until replacement succeeds.

Split leaf signatures are internal split matching metadata only — never part of any public schema, and never present in codedoc.json or the Markdown. They are bounded to a 2,000-character ceiling; a model-returned signature over that bound fails through the normal correction/failure contract and is never silently truncated into a shortened accepted value. When a fully visible declaration is longer than the ceiling, the model is asked to report only its leading source-backed portion — roughly 600 to 1,000 characters, and never above the ceiling — so the declaration is recorded shortened rather than dropped. A signature shortened this way is a truthful, expected answer, never an omission and never a system-applied cut; a fully visible declaration that already fits the ceiling is always recorded in full.

Each accepted leaf, reduction, and final-synthesis result is checkpointed only after it has been cleaned and validated. A compatible interrupted run resumes only unpaid nodes in dependency order.

Recovery written by a different CodeDoc version. A crash_recovery.json this version cannot resume is recognized and preserved exactly as found. It is never resumed, rewritten, or silently discarded, and the check happens before any node is read and before planning, output writing, or provider construction, so the run pays for nothing. You have two supported options: finish the run with the CodeDoc version that wrote the file, or move crash_recovery.json aside — deleting it is an explicit discard of that state — and start fresh with the current version. The same preserve-first rule covers an unknown, foreign, aliased, or duplicated recovery container.

When an upgrade invalidates earlier split work. CodeDoc versions the internal contract each fragment is documented under. When an upgrade changes that contract, earlier work goes stale. That is not the same as an unreadable recovery container, which is refused outright: stale work is still yours and is simply redone. In an unfinished file the affected fragments are set aside and re-executed, along with every reduction and synthesis step that depended on them, and a file already completed is reprocessed in full. Everything still compatible is reused. Even an upgrade that invalidates every fragment of the largest file CodeDoc will split recovers this way instead of aborting the run. Rolling back to an older version has the same effect in reverse. In every case you rerun the same command — expect one extra pass over your large files, and use --dry-run first to see the exact call count.

Every other recovery rejection stays fail-closed: a malformed container, a foreign owner, an unsupported container version, an unplanned or duplicate node ID, and a set-aside map that exceeds its bound all raise and stop the run.

Imports-only changes preserve compatible leaves and reducers but invalidate final synthesis. Provider, model, or effective-endpoint changes invalidate partial nodes, while completed cache reuse remains provider-agnostic. Files at or below max_content_chars continue through the ordinary whole-file path.

A split dry-run scans the canonical source snapshot, builds the same deterministic semantic or lexical chunks, verifies complete coverage, and constructs the bounded reduction topology. For a resolved-valid single + split route it is a read-only preview of the same payable work a real run would do at the same repository state: it loads completed records and crash_recovery.json, classifies them through the same reuse and node-recovery rules, and reports the remaining call categories and capacity reasons over the unpaid nodes only. It makes no provider call, and it never creates, rewrites, quarantines, replaces, or removes recovery or any other file.

The optional structure installation extra provides syntax-aware planning boundaries for supported languages:

pip install "codedoc-ai[structure]"

This extra is optional. Without the optional package, a matching grammar, or a usable parse, planning falls back to deterministic lexical atoms. CodeDoc does not download grammars or create a grammar cache at runtime. A file may contain at most 4,096 planned lexical atoms; exceeding that limit reports atom-cap before any provider call. The optional package can reduce atom count only when it supplies a usable parser for the language. It cannot repair malformed or error-dominated source, and raising max_content_chars cannot clear the line-counted atom cap.

Planning and execution use the exact canonical JSON representation for each bounded manifest, including quotes, backslashes, controls, newlines, and Unicode. It reserves the complete 3,000-character canonical ledger-synopsis allowance when estimating the final input, including valid ledgers whose whole-item trimming fills the allowance more densely than maximum-width escaped items. It reports the first applicable local capacity reason: atom-cap, symbol-cap, unit-cap, chunk-cap, reduction-envelope-cap, reduction-fan-in-cap, reduction-depth-cap, or final-synthesis-envelope-cap.

Module exports in a split file

What CodeDoc reports. Each fragment of a split file is asked only for the module exports its own visible source declares:

• a name the module or package exposes through a declaration or re-export that is visible in that fragment is reported as an export;
• data carried inside an exported value — array elements, object properties, keys, values, IDs, labels, and nested members — is not an export merely because the value containing it is exported;
• where a language declares its exports as a list or an object — an exported-names manifest, a brace-enclosed export list, or an assignment to the module's export table — those entries are the exported names and are reported as such;
• a fragment showing only the interior of a large exported value reports no exports at all: declaration visibility decides this, never the fragment's position in the file; and
• the same definition is sent with the optional single repair call, so a retry cannot reinterpret it.

A fragment may return at most 32 export names, each at most 256 characters. A response above either limit is rejected and reported, never silently shortened.

How you use it. There is nothing to configure. This applies to every file large enough to take the split path:

• run CodeDoc once; a large file is divided, documented, and reassembled automatically;
• if a run is interrupted, rate-limited, or fails on one file, rerun the identical command — completed files and per-fragment checkpoints resume automatically, and only unpaid work is repeated;
• when an upgrade changes the split-leaf contract, the first run afterwards redoes each large file's split work once, whether that file was finished or still in progress, because fragments produced under the previous contract are not reused. Run with --dry-run first to see the exact call count; later runs reuse normally again; and
• one repair call per rejected response is made automatically; set "response_correction_enabled": false to turn it off.

Routing overview

large_file_strategy controls what happens after a readable source file exceeds max_content_chars. The default, truncate, keeps the existing head-plus-tail prompt. split is opt-in through config, CODEDOC_LARGE_FILE_STRATEGY=split, or --large-file-strategy split, and currently requires analysis_mode: single — triple plus split fails with "currently unavailable" guidance before scanning or any other side effect. Resolution follows the normal precedence: defaults < codedoc.config.json < environment < explicit programmatic/CLI override. Values must be exactly lowercase truncate or split.

flowchart TD
    A["large_file_strategy + analysis_mode"] --> B{"split with triple mode?"}
    B -->|"Yes"| C["Reject before scanning or other side effects"]
    B -->|"No"| D{"Decoded source length exceeds max_content_chars?"}
    D -->|"No"| E["Ordinary whole-file analysis"]
    D -->|"Yes"| F{"large_file_strategy?"}
    F -->|"truncate"| G["Head-and-tail truncation"]
    F -->|"split"| H["Extract deterministic semantic boundaries"]
    H --> I["Pack fitting units; create continuation chunks for oversized units"]
    I --> J{"Complete plan fits all capacity bounds?"}
    J -->|"No"| K["Named provider-free capacity block"]
    J -->|"Yes"| L["Document every leaf chunk"]
    L --> M["Same-unit consolidation when needed"]
    M --> N["General reduction only until final manifest fits"]
    L --> O["Merge structured facts into a local lossless ledger"]
    N --> P["One final synthesis per divided file"]
    O --> P
    Q["Actual path, language, and imports"] --> P
    P --> R["Publish final file documentation only"]

large_file_strategy + analysis_mode
├─ split + triple
│  └─ reject before scanning or other side effects
└─ valid combination
   ├─ source length <= max_content_chars
   │  └─ ordinary whole-file analysis
   └─ source length > max_content_chars
      ├─ truncate
      │  └─ head-and-tail truncation
      └─ split
         └─ semantic boundaries
            └─ packed fitting units + continuation chunks for oversized units
               ├─ capacity exceeded
               │  └─ named provider-free block
               └─ complete plan
                  └─ document every leaf
                     ├─ narratives
                     │  └─ same-unit consolidation when needed
                     │     └─ general reduction only until final manifest fits
                     └─ structured facts
                        └─ local lossless ledger

general roots + fact ledger + actual path/language/imports
└─ one final synthesis per divided file
   └─ publish final file documentation only

The diagram describes both provider-free planning and active split execution.

Semantic division and synthesis

Split planning and reconstruction are defined against the canonical decoded snapshot of the file, not its raw bytes. That snapshot is produced once per run: a leading byte-order mark is stripped, undecodable bytes become the Unicode replacement character, and \r\n and lone \r are normalized to \n. Every leaf, coverage check, and boundary offset is measured in Unicode code points of that snapshot, and the pieces of a divided file reconstruct it exactly. CodeDoc also records a separate content hash over the file's original bytes for change detection; it does not promise to preserve or round-trip the raw file itself.

A non-empty provider-bound file at or below max_content_chars uses the ordinary whole-file single-mode request, whether split is selected or not. With split, an oversized file is instead divided at semantic boundaries first (functions, classes, top-level declarations), derived locally and deterministically from the canonical decoded snapshot. A semantic unit that fits max_content_chars stays whole in one chunk and keeps its exact canonical bytes, source range, and identity. Only a semantic unit whose own source exceeds max_content_chars is subdivided, and it is subdivided toward a balanced piece length rather than filled greedily to the ceiling: each cut is placed at a nested syntax boundary or a physical-line boundary within about ten percent of the balanced target, and at the nearest safe character boundary otherwise. Every piece stays at or below max_content_chars. So one indivisible 2,010-character unit at a 1,000-character ceiling becomes three pieces of about 670 characters each, not 1,000 + 1,000 + 10; an 8,292-character span at a 2,000-character ceiling becomes five pieces, never a power-of-two halving into eight. Given natural units of 1,243, 482, and 285 characters, only the 1,243-character unit is subdivided (about 622 + 621); the 482 and 285 units keep their exact bytes, ranges, and identities and may still share one 767-character leaf call. Several adjacent fitting semantic units may share one packed leaf call, but they retain their own identities and remain separate in split_units; a packed call group is not a replacement semantic unit, and CodeDoc does not promise that co-packed units are byte-identical to a global concatenation. For cut placement a \r\n pair is treated as one indivisible unit, so a defensive subdivision may occasionally use one extra piece — and one extra call — rather than split the pair; the normal filesystem pipeline normalizes \r\n and lone \r to \n before planning, so that case cannot arise there and its reported count is always zero. Packing also applies a fixed ceiling to the exact ordered unit/range metadata rendered in the leaf prompt. A chunk closes before that metadata would exceed its ceiling, so every unit and range remains explicit without allowing short lexical atoms to create an unbounded prompt. The optional structure package described above supplies syntax-aware boundaries; the same complete lexical fallback and runtime-offline guarantees apply to execution as well as dry-run.

A very large file can require several paid hierarchical reduction calls above its leaf chunks before final synthesis: chunks belonging to the same oversized semantic unit are consolidated first, in source order; then general reduction levels combine sibling narratives only until the complete final manifest fits the configured ceiling. Final synthesis may therefore receive several ordered roots; planning does not add unnecessary reducers merely to force one root. Continuation chunks of the same qualified unit are always consolidated before their narrative mixes with any other unit's, and equal short names in different scopes are never merged. This internal chunk and reduction work is fixed and not user-configurable — leaf descriptions are required, optional fact lists have explicit prompt-visible bounds (a leaf accepts up to 32 functions and up to 32 classes, matching the same count of known-symbol names the leaf prompt may list), and no separately parsed whole-file imports reach a leaf or reducer request. A combined reduction narrative is capped at 300 characters, and the reducer prompt states that bound explicitly — together with a recommended 260-character target to write toward — so a truthful longer narrative is never rejected without the model having been told the limit. A response that exceeds a fixed leaf or reduction fact bound is rejected through the normal correction/failure contract instead of being silently truncated or published with facts removed. Every distinct accepted structured fact (function, class, export) is retained losslessly in a local, deterministic fact ledger, independent of narrative reduction. Parser symbol IDs and ranges distinguish same-named declarations when available; ambiguous packed facts receive distinct deterministic occurrence scopes, while repeated reports from continuation chunks still consolidate under their one source unit. Only the final, synthesized file-level documentation is ever published — no chunk, unit, or reduction content, and no division or documentation_units object, appears in output. Customizing the final documentation shape never changes the fixed internal chunk/reduction contracts. Before publication or format conversion, functions, classes, and exports are projected through the ordinary file-level schema and limits; internal signatures, source provenance, IDs, and ranges are never public.

max_content_chars is the source ceiling for every ordinary whole-file request and every split leaf input. Reduction manifests and the complete final-synthesis manifest use a separate automatic synthesis ceiling — the larger of max_content_chars and a fixed 12,000-character floor — so lowering the source ceiling below 12,000 never shrinks internal synthesis below its safe default size, and raising it above 12,000 raises both. Both are content ceilings; the complete provider prompt is always larger than either, because it also carries system instructions, shape rules, metadata, and framing, and neither ceiling is a provider context-window guarantee. Final capacity is planned from the actual path, language, and parser-derived imports plus distinct maximum-size root narratives and the largest bounded fact ledger reachable from the planned leaves. If needed, only the already-lossy ledger synopsis is trimmed deterministically; path, language, imports, narratives, and coverage are never dropped. If those authoritative fields cannot fit, planning reports final-synthesis-envelope-cap before provider creation.

Capacity and failure behavior

The max_file_size_kb scanner limit is applied first, in bytes, during scanning. A file rejected there is a scan skip — it never reaches character-based split planning and never produces a capacity reason below.

Past that gate, an extremely large file can instead block provider-free, before any call, with one of eight named capacity reasons reported in a fixed evaluation order (atom-cap, symbol-cap, unit-cap, chunk-cap, reduction-envelope-cap, reduction-fan-in-cap, reduction-depth-cap, final-synthesis-envelope-cap) — split never silently falls back to truncation. A blocked file makes no provider call and is excluded from max_files; dry-run reports every blocked path and reason with exit 0, while a real run stops before writing or contacting a provider. Inspect the reported reason before choosing a remedy, because they do not share one:

• atom-cap, symbol-cap, and unit-cap are structural counts — simplify or exclude the file; a larger ceiling does not change them.
• chunk-cap is the only reason a larger max_content_chars (when the provider supports the larger input) or splitting the source differently can clear.
• reduction-envelope-cap, reduction-fan-in-cap, and reduction-depth-cap cannot be reached by a legal source ceiling alone under supported settings; if one occurs, report it as an internal planning-capacity defect rather than raising the source ceiling.
• final-synthesis-envelope-cap means the authoritative path, language, or import metadata will not fit — inspect and shorten those inputs or exclude the file; report it if they are already ordinary.

Choosing truncate is appropriate only when incomplete-source analysis is acceptable. Lowering max_content_chars can create more leaves, more reduction levels, and more paid calls even though each call is smaller. A genuine internal division-plan defect is a different, rarer case — a programming-invariant failure, not a capacity outcome. It propagates uncaught and aborts the whole run (dry or real) before any provider or writer side effect, leaving prior stable output completely untouched; it is never a per-file failure statistic.

Split accounting, identity, and provider checks

Split dry-runs and real runs report ordinary, leaf, unit-consolidation, general-reduction, and final-synthesis calls as separate categories. The synthesis input-token estimate is a deterministic worst-case envelope, not a tokenizer-exact count. An oversized requested-split record carries a private _large_file_identity cache key bound to its exact division plan and reduction tree; ordinary records retain ordinary identity behavior. This private key is persisted in machine-readable JSON and the embedded Markdown view for safe round trips, but never appears in visible documentation prose.

The exact initial-call plan is P = R + O + (C - Hc) + (U - Hu) + (G - Hg) + (F - Hf): R is prompt-customization review calls; O is ordinary provider-bound whole-file calls remaining after reuse; C/Hc are planned/restored leaf calls; U/Hu are planned/restored unit-consolidation calls; G/Hg are planned/restored general reductions; and F/Hf are planned/restored final syntheses. Retries and corrections are additional attempts attached to an existing logical call. max_files and max_planned_calls count exact unpaid work; a fully restored or completed-reused split file contributes neither a paid candidate nor a review.

Provider construction must attest to the provider, model, and effective endpoint used by the plan. Missing attestation fails closed. Implicit HTTP/HTTPS default ports normalize identically to explicit :80/:443 endpoints, and trailing slashes do not create a different effective OpenAI-compatible endpoint. A malformed HTTP(S) URL, host, or port is rejected as configuration before provider creation using a value-free diagnostic that does not echo credentials or URL details.

Completed split reuse and node recovery

CodeDoc checkpoints every leaf, unit-consolidation, general reduction, and final-synthesis node independently and by its own node ID, each carrying a provider/model/effective-endpoint execution identity; resuming revalidates every checkpoint against its exact planned node type, ordered children, ordered coverage, stage-local input digest, node-specific identity, and the same live cleaner/required-field schema. Recovery is dependency-closed: an invalid or missing descendant prunes every affected reducer/final ancestor, while unrelated valid leaves remain reusable. Equal-length imports-only changes preserve compatible leaves and reducers but rerun final synthesis. Provider, model, or effective-endpoint changes invalidate partial nodes; completed cache reuse remains provider-agnostic.

Response correction

Provider responses must satisfy a deterministic JSON contract: the requested keys, the requested types, a non-empty value for every required field, and at least one usable requested field. A response that fails the contract is rejected with a bounded, structured diagnostic. A final (non-retryable) response-contract failure names its closed reason code in the visible failure message, so the user-facing error states which contract failed, not merely that one did; it still includes no source text, prompt text, raw or truncated provider response, credential, endpoint, or per-field removal detail.

Response correction is enabled by default. It makes at most one targeted correction call per rejected response — a single extra paid provider call that asks the model to repair the response to the exact schema, preserving valid facts and inventing nothing. Set "response_correction_enabled": false to turn correction off, so a rejected response fails the file with no correction call.

• With correction on (the default), one repair call is made per rejected response; if the repair also fails the contract, the file fails without a further retry. Worst case this adds one call for every rejected documentation response — up to a 100% increase over the initially planned documentation calls — counted per rejected agent, leaf, reducer, or final response, not per source file, so one split file can make more than one correction call.
• With correction off, a rejected response receives no correction call; the file fails without a correction-triggered retry.
• The default applies only when the key is absent. An existing project configuration that sets response_correction_enabled to false — including one generated before the default changed — stays off and is never rewritten by an ordinary run; only a freshly generated configuration template carries the new default.
• max_planned_calls authorizes only the initially planned calls; corrections and file_retry_attempts transport retries are both outside it, so it is not a hard final-billing ceiling.
• Correction is not a factuality bypass. Malformed output or a missing or empty required field can still fail a file whether correction is on or off.
• file_retry_attempts remains the policy for transport, rate-limit, and other recoverable failures; it is unaffected by response correction.
• At --verbose (log_level: DEBUG), diagnostics add only bounded structural metadata (removed field paths and reason codes, returned value types, a parse position, and a response character count). CodeDoc's rejection diagnostic records do not include raw provider-response text, source, prompts, or credentials.
• CodeDoc verbosity raises only the codedoc logger namespace. It never raises the root logger or lowers the reviewed OpenAI, Anthropic, Gemini, authentication, HTTP-client, or transport logger floors. In an embedding application, unrelated logging explicitly enabled by the host remains the host's responsibility.

Command-line options

Ignore rules are resolved from --ignore/ignore_paths and the --skip-dirs/--add-skip-dir/--remove-skip-dir family. Paths are project-relative; skip-directory values are directory names.

For example, documenting CodeDoc's own codedoc/ source (whose name matches a default skip-dir) while writing output elsewhere:

codedoc . --remove-skip-dir codedoc --output docs_output

The exact generated JSON, Markdown, and recovery target files are excluded from scanning by resolved-path equality; the output directory itself is not automatically excluded. Other supported source files co-located there remain scannable when ordinary skip rules allow them, while generated targets are never re-scanned as source.

Providers and environment variables

The credentials and overrides listed below may come from operating-system environment variables. CodeDoc does not read .env files.

Provider defaults are OpenAI gpt-4o-mini, Anthropic claude-haiku-4-5-20251001, and Gemini gemini-2.5-flash. Select explicitly with --provider and --model. With auto, configured Anthropic prefixes are checked first, then Gemini prefixes; every other model uses the OpenAI adapter. The default prefixes are editable through the provider_prefixes settings.

Inline instructions

The only runtime instruction source is prompt_profiles inside the exact codedoc.config.json (or an in-memory Python override). Generated profiles are versionless and use requested_shape.

Every present mode uses a required common envelope and an optional per_extension complete replacement, keyed by file extension:

{
  "prompt_profiles": {
    "single": {
      "common": {
        "requested_shape": {
          "description": "Explain what this file does.",
          "role_in_system": "Explain its architectural role."
        }
      },
      "per_extension": {
        ".js": {
          "requested_shape": {
            "description": "Explain this JavaScript module for a reviewer."
          }
        }
      }
    }
  }
}

Triple mode carries all three agent keys inside each per_extension override (complete replacement), and a non-empty triple.per_extension requires triple.common.documentation:

{
  "prompt_profiles": {
    "triple": {
      "common": {
        "structure": { "requested_shape": { } },
        "dependency": { "requested_shape": { } },
        "documentation": {
          "requested_shape": {
            "description": "Explain this file for a maintainer."
          }
        }
      },
      "per_extension": {
        ".cs": {
          "structure": { "requested_shape": { } },
          "dependency": { "requested_shape": { } },
          "documentation": {
            "requested_shape": {
              "description": "Explain this C# file for a maintainer."
            }
          }
        }
      }
    }
  }
}

Extension resolution. For each file the effective block is chosen by longest matching per_extension > common > built-in default. Matching is on the file's lowercased basename, so multi-part suffixes work and the longest match wins: .d.ts beats .ts for types.d.ts, and matching is case-insensitive (Types.D.TS selects .d.ts). A file whose entire name is .ts is not treated as a .ts-suffixed file. An override is a complete replacement of the block, never a field-by-field merge. Each per_extension key must be a lowercase dotted suffix whose final segment is one of the project's configured extensions (from extension_language_map); .pyy is rejected when only .py is configured, which prevents silently dead overrides. An entry that matches no scanned file is validated but costs nothing — it renders no prompt, makes no review call, and invalidates no cache. Editing a used override changes cache identity only for files whose basename resolves to it; files that fall back to an unchanged common retain reuse eligibility.

Unsupported profile layouts are rejected with targeted guidance.

analysis_mode: single exposes one combined editable instruction JSON at single.common. Triple mode exposes three independently editable instruction JSON blocks at triple.common.structure, .dependency, and .documentation. Supported field order, optional fields, and bounded instruction text are editable; fixed system, safety, factuality, scanning, retry, cache, and serialization rules are not. per_extension remains a complete-block replacement.

An effective non-default instruction is reviewed only when it will reach a planned LLM documentation call. SAFE continues, RISKY requires explicit per-run confirmation, and TOO_RISKY always stops. Initialization, unedited defaults, dry runs, cache-only work, and deterministic JSON↔Markdown conversion make no security-review call. There is no stored bypass. CodeDoc also computes deterministic, non-blocking feasibility advisories when a custom field appears to require cross-file context that a per-file pass cannot see. These advisories are provider-free, appear in dry-run and real-run summaries, never block a run, and never change the standards/safety review verdict. Fixed system roles, factuality/safety rules, parser facts, cleaners, provider selection, scanning, retry, cache, ownership, and artifact serialization are not customizable.

Use codedoc --init-config as the registry-backed reference for exact fields, types, and complete defaults.

Output, incremental reuse, and ownership

In a single-format run, an existing requested target is authoritative. If it is missing, CodeDoc may strictly validate and reuse only its exact opposite-format sibling: codedoc.json pairs with codedoc.md, and a named docs/report.json pairs only with docs/report.md. Unchanged compatible records are converted without provider calls; changed, forced, missing, or cache-incompatible files continue through normal planning. The sibling is read-only and only the requested format is written.

A present fallback that is foreign or malformed blocks before provider contact instead of silently starting a paid fresh run. No directory walk, modification- time choice, unrelated default filename, or extra candidate is used. Entry recovery remains tied to the selected output; when it is absent, ordinary source entry auto-detection runs.

Both mode reads its exact two targets and blocks before provider contact if entry, path set, hashes, or cache identity disagree. When only one valid target exists, it supplies the records used to create both outputs.

CodeDoc refuses to overwrite foreign, empty, or malformed final targets. Custom output names remain supported when supplied explicitly:

codedoc --output docs/report.json
codedoc --output docs/report.md

--format both requires a directory because it writes two files.

Crash recovery

Every real run that reaches the finalization pipeline selects exactly <resolved-output-directory>/crash_recovery.json; a no-supported-files run returns without creating it. The file is initialized only when provider work remains, after ownership/path checks, deterministic validation, read-only planning, paid caps, and any mandatory semantic review have succeeded. It is updated atomically after each completed ordinary file and after every returned, cleaned, live-schema-valid split node. A checkpoint is committed before its dependent is scheduled.

The recovery file includes a versioned identity covering project root, exact selected targets, entry, documentation scope, analysis mode/revision, and large-file strategy. It no longer binds a profile-wide digest: each recovered completed record is instead re-validated individually against the current per-file _prompt_profile_digest, so an unrelated profile edit or a newly added file no longer discards a resumable run. Compatible completed ordinary and split records may be reused. A compatible split container is validated in plan order; valid siblings remain reusable, rejected nodes are retained in a bounded, non-executable set-aside map, and affected ancestors rerun. A container written by a CodeDoc version whose recovery format is unsupported by this build is rejected on that format alone, before any node is read. Provider changes invalidate partial nodes but not a compatible completed record. Imports-only changes retain compatible leaves and reducers while rerunning final synthesis. A foreign, completed, unsupported, or identity-mismatched recovery file blocks without mutation. Restore the prior configuration or matching CodeDoc version to resume it, or move crash_recovery.json aside before starting fresh. Recovery-file deletion is an explicit choice to discard that recovery state.

During provider work, prior stable output is untouched. On successful finalization, each selected artifact is atomically replaced and recovery is removed only after all selected writes succeed. both mode is not a cross-file transaction: if its second write fails, the first artifact may already have been replaced. Any existing or initialized recovery file remains; an all-reused run does not create one solely for finalization. Dry-run may inspect the exact recovery path but never creates, changes, or deletes it.

On Ctrl-C, parallel execution sets one shared cancellation signal, cancels queued files, and allows only already-running provider calls to return. No later initial, retry, or response-correction call begins. Recovery remains available and the CLI exits with code 130.

Planning and diagnostics

--dry-run performs scanning and planning without persistent mutation, provider creation, or API calls. It reports empty/whitespace-only files separately and excludes their expected documentation calls and prompt tokens. --max-files N caps files with at least one unpaid provider action. This is measured after completed-record reuse and dependency-closed recovery, so a completed-reused or fully restored split file is not a paid candidate. --force-files PATH bypasses reuse and recovery for the selected file while preserving prior state until replacement succeeds. In split mode the dry run reports exact ordinary-file, leaf, unit-consolidation, general-reduction, and final-synthesis call counts; structural routing and capacity-block reasons; and a deterministic worst-case final-synthesis input estimate rather than a tokenizer-exact prediction. For a resolved-valid single + split route the dry run reads completed records and crash_recovery.json and classifies them exactly as a real run does, so its reported payable work matches a real run at the same state and a fail-closed recovery container raises the same error in dry-run as in a real run; it still writes nothing and constructs no provider. A separately resolved ordinary/truncate run keeps its established dry-run behaviour and is not changed by this. A real run prints the same Planned provider work (before calls) summary from the same snapshot builder before any provider is constructed or any documentation call is made, so a run that is about to be capped or blocked is explained before it fails; --dry-run remains the review-and-exit path that never proceeds to calls.

Default output shows the resolved strategy, the source ceiling, split/truncate routing counts, the effective source and synthesis ceilings, the aggregate oversize size range and block reasons, retained and omitted truncation character totals, and a bounded per-file summary ordered most-work-first, with an explicit count of how many files were omitted from that summary. --verbose adds every retained file and its ordered piece transforms — including 2010 -> 670 + 670 + 670, metadata-driven chunk closures, any continuation-cut call delta, and truncation head/tail/omission counts — and states the omitted count and a full-stream digest whenever the retained list is capped.

Scanner diagnostics describe only the final authoritative scan generation for a run. When a detected concurrent source change triggers one complete rebuild, that rebuild's scan atomically replaces the earlier scan's totals and details rather than merging the two, so exact counts and the reported digest never double-count a file across generations. An explicit entry that exists but admits no source — an empty explicit directory, or explicit targets that are all size-skipped, unreadable, ignored, unsupported, or missing — still publishes that final generation's bounded scanner evidence and an empty payable-work report before the established "no files" error is raised, in both --dry-run and a real run, so a dead entry is explained rather than only rejected.

Issues are bounded in memory and reported through terminal/log output. Hard-error summaries are included in completed JSON or Markdown; warning-only issues are not. Recovery files do not contain an issue log, and CodeDoc does not write error.log.

Exit codes:

Python API

Use run_pipeline for the complete configuration, planning, provider, recovery, and output lifecycle:

from codedoc import run_pipeline

stats = run_pipeline({
    "entry_file": "src/main.py",
    "output_format": "json",
    "max_parallel_files": 3,
})

stats = run_pipeline("/path/to/project", {"output_format": "both"})

In-memory overrides are supported but do not create another persistent config source. Unsupported settings such as external prompt paths, risky-review bypass, safe mode, and managed output ignore files raise targeted configuration errors.

Offline format conversion

CodeDoc-format JSON and Markdown data can be converted locally without constructing a provider or making an LLM call:

from pathlib import Path

from codedoc.core import json_from_markdown, markdown_from_json

markdown_text = Path("codedoc/codedoc.md").read_text(encoding="utf-8")
json_text = json_from_markdown(markdown_text)
Path("codedoc/converted.json").write_text(json_text, encoding="utf-8")

json_text = Path("codedoc/codedoc.json").read_text(encoding="utf-8")
markdown_text = markdown_from_json(json_text)
Path("codedoc/converted.md").write_text(markdown_text, encoding="utf-8")

The conversion helpers operate on text or parsed data; file reading and writing remain under the caller's control.

Exported Python surface

The following names are intentionally exported. run_pipeline is the recommended end-to-end entry point; codedoc.core exposes conversion helpers and lower-level components for integrations that manage more of the lifecycle themselves.

Troubleshooting

• Missing credential: set the matching provider environment variable.
• Unexpected paid work: run the identical command with --dry-run and inspect the exact output selection and analysis mode.
• Recovery conflict: follow the error's expected/found field, then restore the prior configuration or move the exact crash_recovery.json aside. Delete it only when intentionally discarding its checkpoint history.
• Missing files: check entry selection, documentation_scope, skip_dirs, ignore_paths, extension_language_map, and max_file_size_kb.
• Rate limits: lower max_parallel_files; adaptive stepping is enabled by default.
• Interrupted, rate-limited, or partly failed run: rerun the identical command. Compatible completed files and split node checkpoints resume automatically; only unpaid work is re-executed. Never hand-edit crash_recovery.json.
• Large files cost one extra pass after an upgrade that changes the split-leaf contract: each is paid for again, once, whether it was finished or still in progress. See Module exports in a split file. Budget for that pass, use --dry-run first to see the exact call count, and expect normal reuse from the next run on.
• Repeated response-contract rejection on one file: response_correction_enabled is an optional one-time project setting that spends at most one extra provider call per rejected response. If the same closed reason repeats, try a more capable model, or report the bounded reason code and file type — never source, prompts, credentials, or the recovery file.

License

See LICENSE.