ConfigReach

Your tests have 94% code coverage. But only 31% configuration coverage. ConfigReach tells you the difference.

ConfigReach is a deterministic, CPU-only, offline configuration coverage analyzer that shows which runtime configuration inputs, values, branches and important combinations your tests actually exercise. Think Codecov for configuration space.

ConfigReach needs no GPU, no LLM, no API key, no hosted service, no telemetry and no paid dependency. Static analysis does not execute the target repository. Machine-readable results are deterministic for the same repository state and configuration.

Why configuration coverage?

Code coverage can tell you that a line executed. It cannot tell you whether the configuration states that change that line's behavior were exercised.

mode = os.getenv("PAYMENT_MODE", "sandbox")
if mode == "live":
    charge_real_card()
else:
    simulate_charge()

A suite can execute that block every time and still never test PAYMENT_MODE=live. ConfigReach inventories configuration reads and declarations, maps them to test evidence, tracks known values and branches, measures configuration combinations, and reports the gaps.

Quick start

git clone https://github.com/sauravsingla/ConfigReach.git
cd ConfigReach
python -m pip install -e .
configreach scan .

Useful examples:

configreach scan examples/polyglot
configreach explain PAYMENT_MODE examples/polyglot
configreach matrix examples/polyglot
configreach scan examples/polyglot --format html --output configreach.html
configreach plan examples/combinations --format markdown
configreach plan examples/combinations --fixture pytest --output configreach_cases.py
configreach workspace . --format json --output configreach-workspaces.json
configreach adapters --format json
configreach reproduce examples/combinations --runs 3
configreach schema --format json

Observable metrics — no opaque AI score

ConfigReach reports evidence-based metrics independently:

• Key coverage — discovered configuration inputs with detected test/runtime evidence.
• Value coverage — explicitly tested values / explicitly known values.
• Boolean coverage — tested true/false states where a boolean domain is known.
• Enum coverage — exercised known discrete values for non-boolean finite domains.
• Branch-state coverage — configuration-dependent branch states with explicit test-value evidence.
• Pairwise key coverage — interacting keys receiving joint test evidence.
• Pairwise value-state coverage — known value combinations observed together in the same detected test scenario.
• Blast radius — files and top-level modules reading a configuration key.
• Workspace coverage — weighted configuration coverage across independently cached monorepo workspaces.

Unknown domains remain unknown. ConfigReach never asks a model whether something is “probably covered.” See docs/metrics.md.

Discovery coverage

Runtime reads

Ecosystem	Examples	Current analysis
Python	os.getenv, os.environ[...], os.environ.get, setdefault	AST-backed
Pydantic Settings	BaseSettings, aliases, Literal, Enum, bool domains, Field validators	AST-backed
Python CLI	argparse, common Click/Typer option forms	AST-backed/conservative
Feature flags	is_enabled, feature_enabled, LaunchDarkly-style variation calls	AST + deterministic patterns
JavaScript / TypeScript	process.env, Deno/Bun environment access, function-scoped comparisons	deterministic semantic adapter
Zod	z.enum, z.boolean, z.literal, min/max/regex/url/email/nonempty	deterministic validator adapter
Go	os.Getenv, os.LookupEnv, t.Setenv, function-scoped comparisons	deterministic semantic adapter
Java / Spring	System.getenv, System.getProperty, @Value, Environment.getProperty, @ConfigurationProperties	deterministic semantic adapter
Java Bean Validation	@Min, @Max, @Size, @Pattern, null/blank/sign constraints on @Value fields	deterministic validator adapter
.NET / C#	environment variables, IConfiguration, GetValue, feature flags	deterministic semantic adapter
Rust	env::var, env::var_os	deterministic pattern adapter
Ruby	ENV[...], ENV.fetch(...)	deterministic pattern adapter
PHP	getenv(...), common env(...)	deterministic pattern adapter
Shell	$VAR, ${VAR}	deterministic pattern adapter

Declarations, schemas and deployment sources

ConfigReach recognizes .env.example, .env.template, .env* templates, JSON, TOML, INI/CFG, Java properties, YAML environment declarations, Dockerfiles/Containerfiles, Docker Compose, Kubernetes-style environment declarations, Helm values.yaml, GitHub Actions ${{ vars.* }} and ${{ secrets.* }}, Terraform variables, Makefile variables, Pydantic settings, JSON Schema finite domains/validators, Zod schemas and CLI options.

Commands

configreach scan [PATH]
configreach coverage [PATH]
configreach explain KEY [PATH]
configreach matrix [PATH]
configreach plan [PATH]
configreach plan [PATH] --strength 3 --max-cases 40
configreach plan [PATH] --fixture pytest|jest|go|shell|junit|xunit
configreach workspace [PATH]
configreach adapters
configreach reproduce [PATH] --runs 3
configreach schema
configreach schema --kind report --check configreach.json
configreach diff origin/main...HEAD [PATH]
configreach pr-comment origin/main...HEAD [PATH]
configreach doctor [PATH]
configreach export [PATH] --format json
configreach export [PATH] --format sarif
configreach export [PATH] --format html
configreach baseline create [PATH]
configreach cache clear [PATH]
configreach init [PATH]
configreach trace --path . -- pytest -q

Deterministic test planning

configreach plan converts already-known finite configuration domains into bounded 1-wise, 2-wise or 3-wise suggestions. Existing test scenarios are subtracted first, sensitive-looking keys are excluded, and CPU-safety limits prevent Cartesian-product explosions.

configreach plan . --strength 2
configreach plan . --format json --output configreach-plan.json

The planner does not invent values, execute the application, synthesize assertions or call a model. See docs/planning.md.

Fixture exporters

Turn the deterministic plan into lightweight scaffolding for your own tests:

configreach plan . --fixture pytest --output test_configreach_cases.py
configreach plan . --fixture jest --output configreach.cases.ts
configreach plan . --fixture go --output configreach_cases_test.go
configreach plan . --fixture shell --output configreach_cases.sh
configreach plan . --fixture junit --output ConfigReachCases.java
configreach plan . --fixture xunit --output ConfigReachCases.cs

Exporters provide configuration cases only; they deliberately do not invent expected business outcomes. See docs/fixtures.md.

Monorepos and workspace-local incremental caching

configreach workspace detects Python, Node, Go, Rust, Maven/Gradle and .csproj workspace roots. Each workspace receives an independent .configreach/cache/ boundary, so changing one package does not invalidate unrelated warmed workspace caches. Parent workspaces ignore nested workspace directories to avoid double counting.

configreach workspace .
configreach workspace . --format markdown
configreach workspace . --format json --output workspaces.json

The normal configreach scan . remains the combined repository view. See docs/workspaces.md.

Adapter API and optional parser-backed plugins

The base package remains dependency-free, but external deterministic adapters can register through the configreach.adapters entry-point group. Adapter API v1 includes compatibility version, parser identity, determinism declaration and capability metadata.

configreach adapters
configreach adapters --format json

ConfigReach rejects incompatible or explicitly non-deterministic plugins without crashing the core scanner. A real optional tree-sitter JavaScript adapter example lives under examples/plugins/tree_sitter_js; installing it is separate from installing ConfigReach. See docs/plugin-sdk.md and docs/adapter-capabilities.md.

Reproducibility verification

configreach reproduce .
configreach reproduce . --runs 5 --format json --output repro.json

Repeated scans are uncached and converted to canonical JSON before SHA-256 hashing. Absolute root, timing and cache metadata are excluded because they are execution-environment metadata, not analysis semantics. The repository's reproducibility workflow compares canonical digests produced on Ubuntu, macOS and Windows and fails if they differ. See docs/reproducibility.md.

Schema compatibility

ConfigReach publishes explicit versions for scan reports, workspace reports, deterministic plans, reproducibility results and adapter capability inventories.

configreach schema
configreach schema --format json
configreach schema --kind report --check configreach.json

The validator rejects unsupported future schemas instead of guessing their meaning. Report schemas 3-5 are accepted for structural compatibility checks, while new scan output remains report schema v5. Golden compatibility fixtures live in the test suite. See docs/schema-compatibility.md.

CI gating

configreach scan . --fail-under 70
configreach scan . --fail-on error
configreach scan . --fail-on uncovered
configreach scan . --fail-on untested-values
configreach scan . --fail-on default-only
configreach scan . --fail-on global-env-overwrite

--fail-under gates key coverage. --fail-on can gate finding aliases, severities or exact CRxxx rule IDs.

Pull-request configuration diff

configreach diff origin/main...HEAD
configreach pr-comment origin/main...HEAD --output /tmp/configreach-comment.md

ConfigReach resolves the local Git merge base, scans the base snapshot and current tree, and reports new/removed keys, changed domains/defaults, newly introduced untested configuration, new values without test evidence, changed-line configuration impact and blast radius. No external service is required.

Searchable static HTML report

configreach scan . --format html --output configreach.html

The report is a single self-contained HTML file with no CDN/network dependency and includes searchable key/value/branch/combination evidence plus source links.

Baselines and cache

configreach baseline create .
configreach scan . --no-cache
configreach cache clear .

Baseline keys remain visible but are excluded from CI key-coverage gating. Cache/timing state is excluded from JSON/SARIF result semantics. See docs/baselines.md.

Optional lightweight runtime tracing

configreach trace -- pytest -q
configreach scan .

Tracing is explicit opt-in. The Python tracer records key names plus short SHA-256-derived value fingerprints; it does not persist raw runtime values.

Deterministic findings

Rule	Meaning	Default severity
CR001	configuration has no detected test/runtime evidence	warning
CR002	application read without a recognized declaration	warning
CR003	declaration without a recognized application read	note
CR004	known values are not all exercised	warning
CR005	sensitive-looking configuration has a non-empty static default	error
CR006	likely inconsistent names normalize to the same identifier	warning
CR007	production-like known value is not exercised	warning
CR008	explicit test values only exercise defaults	warning
CR009	a test mutates the global environment in a potentially leaky way	warning

Every finding retains source provenance.

GitHub Actions

name: Configuration coverage
on: [pull_request]

jobs:
  configreach:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - uses: sauravsingla/ConfigReach@main
        with:
          path: .
          format: markdown
          fail-under: "60"
          fail-on: error

Markdown output can be appended to the job summary, SARIF can be uploaded to Code Scanning, and the repository includes an optional PR-comment workflow.

Benchmark, performance budget and testing

python benchmarks/bench_scan.py 1000
python benchmarks/perf_budget.py --files 800 --min-files-per-second 150
python -m pip install -e ".[dev]"
pytest
python -m compileall -q src tests
configreach reproduce examples/combinations --runs 3

The dedicated performance workflow runs the full semantic engine over a synthetic Python/TypeScript/Go/Java/.NET repository and uses a deliberately conservative throughput floor to catch order-of-magnitude regressions without turning runner noise into flaky CI.

The suite covers language/config discovery, deployment sources, validators, Pydantic/feature flags, branch provenance, combination metrics, real Git PR comparison, baselines, cache behavior, workspace-local invalidation, planners, fixture exporters, plugin compatibility, schema compatibility, reproducibility, performance gating, HTML/SARIF/JSON/Markdown reporters and CLI policies.

Design principles

• CPU-only and zero runtime dependencies.
• No network, telemetry, model inference or paid API in the core.
• Static scanning never executes target application code.
• Runtime tracing is explicit opt-in.
• Unknown semantics stay unknown rather than being guessed.
• Machine output is designed for deterministic CI use.

See docs/architecture.md, docs/threat-model.md, docs/schema-compatibility.md, docs/roadmap.md and CONTRIBUTING.md.