ShockHarness

A strict, deterministic static-analysis quality gate for Python, JavaScript and TypeScript. Its one idea: unexpected failures, missing information, and unexplained decisions must not silently become apparently valid program behavior.

Three default-on hard policies:

1. Exception transparency. No catch except at narrow, explicitly approved external boundaries. No catch-and-continue, no success-shaped fallbacks.
2. No implicit defaults. No fallback values for missing required data, no function-parameter defaults in strict mode.
3. No magic numbers. Every numeric literal in executable code must be exactly 0 or 1. range(2) is a violation. The tool invents no exemptions.

Plus a separate, advisory complexity family (CPLX001-CPLX006: cyclomatic, cognitive, nesting, NPath, a confidence-classified call graph with cycles and fan-out, git churn and hotspots). Hard violations are never averaged into a score; complexity never fails a plain check.

ShockHarness complements Ruff and ESLint; it is not a formatter or a general linter. It runs offline, needs no LLM, and never uploads source.

What ShockHarness insists on

Each rule family enforces one of these. docs/rules.md has the rules, and the write-compliant-code skill teaches the same list to coding agents.

• Failures are visible. A failure is raised and reaches someone. Code catches only at a declared boundary, where the program meets the outside world, and there the failure becomes a typed error, never a success-shaped value. A finally does not swallow it, a batch loop does not log it away, a retry gives up. (EXC001-EXC005, DEF005, BATCH001, RETRY001, ASYNC001-ASYNC003)
• Absence is a decision, not a default. Missing data fails loudly, once, where it enters. No parameter defaults, .get(k, v), x or v, x ?? v or environment fallbacks. A value that is really optional says so in its type, and the code branches on it by name. (DEF001-DEF006, INPUT001, INPUT002)
• Numbers have reasons. Every number other than 0 and 1 is a configuration parameter or a named constant that says what it is for, and a constant is never named after its value. (MAGIC001, MAGIC003, MAGIC004)
• The verifier is not negotiable. A change does not get through by skipping a test, weakening an assertion, raising a threshold, widening a suppression or loosening the policy file. Each of these is reported at review. (VERIFY001-VERIFY004, ROLE001, SUPP001, SUPP002)
• Unfinished work says so. A stub, a mock in production code or a check that always passes is reported, never mistaken for an implementation. (STUB001-STUB003)
• What a value must carry is written down. Facts the value cannot show, such as normalized, redacted or sanitized, are declared where they are established and checked on every path to where they are needed. (PROP001-PROP008, opt-in)
• Hard rules are not averaged. A violation is never offset by a good score. Complexity is advisory and never fails a plain check.

Status

Pre-release. Milestones A (skeleton and contracts), B (parsers and MAGIC001), C (exception rules EXC001-EXC003), D (default and fallback rules DEF001-DEF006), F (baselines, revision diff and gate, SARIF output, git churn and co-change metrics, built-in profiles, the check command, tag-triggered PyPI publishing, a reusable GitHub Actions workflow and two consumer examples) and G (bundled agent skills, skills install / skills update with a manifest, and a skill evaluation corpus) and E (per-unit complexity metrics with versioned conventions, a call graph whose edges are resolved, probable or unresolved, module cycles, fan-in/out, shared-mutable-state notes, hotspots from churn and cognitive complexity, and per-unit regressions in revision diffs) are implemented, as are the first five milestones of the advanced guardrails: H1 (an explicit unknown finding status and file roles), H2 (verification integrity: in a revision diff, report test skips and focus markers, blanket suppressions, neutralized assertions and looser policy, baselines, workflows or tool settings that a change adds; an organization --policy file the repository cannot loosen; and a fifth agent skill, verification-integrity), H3 (syntax-level guardrails: return in finally, stubs and production imports of test doubles, removable asserts, positional Boolean flags, module state written from functions, side effects at import time; advisory until their precision is measured) and H4 (async lifecycle: coroutines and Promises created and dropped, tasks started without an owner, settled batches whose failures nobody checks; a sixth skill, async-lifecycle) and H5 (API-catalog rules: shell commands from data and ignored exit statuses, must-use results, type escape hatches, fixed sleeps, unbounded retries, batches that drop failures, checks that always say yes). Every registered rule is implemented; a report lists any rule registered but not built under rules_not_implemented, so a clean run never implies coverage the tool does not have. See STATUS.md for exact evidence.

Implemented rules:

Install

Requires Python 3.11+. Runtime dependencies are tree-sitter and tree-sitter-javascript only. Pin an exact version; releases are tagged and published to PyPI from v* tags (see docs/releasing.md).

pip install "shockharness==0.10.0"          # Python project: put the pin in pyproject.toml
uv tool install "shockharness==0.10.0"      # any project (JavaScript included): isolated tool env
pipx install "shockharness==0.10.0"

From a checkout:

uv pip install -e ".[dev]"

Use

shockharness version                    # same as --version
shockharness init                       # writes .shockharness.toml (strict profile)
shockharness init --profile recommended # or minimal: a short file selecting the profile
shockharness config validate            # exit 0 if valid; prints the profile and every deviation
shockharness config effective           # merged config with per-key origins
shockharness check src/                 # human-readable report, exit = gate status
shockharness check src/ --summary       # first contact: findings by rule x directory, worst files, complexity counts; same gate line
shockharness check src/ --format json --output report.json
shockharness check src/ --format sarif --output report.sarif  # GitHub code scanning
shockharness check src/ --profile minimal                      # try a profile without editing config
shockharness check src/ --parse-errors warn                    # the only other CLI override
shockharness check src/ --exit-zero                            # advisory: full report, exit 0 on violations
shockharness check src/ --no-cache                             # bypass the content-hash cache (.shockharness-cache/) for one run
shockharness baseline create src/ --output .shockharness-baseline.json
shockharness check src/ --baseline .shockharness-baseline.json # legacy findings do not gate
shockharness diff --base main --head HEAD                      # new / fixed / unchanged, exit 0
shockharness gate --base main --head HEAD                      # exit 1 only on new hard violations
shockharness exceptions src/                                   # every suppression, boundary, override, exclusion and baseline entry, with its source (exit 0)
shockharness rules list
shockharness rules explain MAGIC001
shockharness skills list                                       # the six bundled agent skills
shockharness skills install --agent claude                     # .claude/skills/<name>/SKILL.md + manifest
shockharness skills update --agent claude --check              # exit 1 when a skill is stale or edited

check is the canonical analysis command; scan, the name used before Milestone F, still works as a hidden alias with identical arguments.

Baselines are reviewed files of stable finding fingerprints (rule, path, scope, whitespace-free evidence, occurrence index) that survive line moves; nothing ever updates one automatically. diff and gate compare the full trees of two committed revisions through local git only (uncommitted changes are not part of HEAD). Every CLI report also carries metrics.history: per-file churn and co-change pairs from local git log, at file granularity, never gating, and explicitly unavailable outside a repository.

Exit codes: 0 gates pass, 1 policy violations, 2 invalid configuration or invocation, 3 internal error or (with parse_errors = "fail") a file that could not be parsed. A report is still produced when the gate fails.

Example:

src/client.py:12:11: MAGIC001 error: Numeric literal 30 has no permitted justification in executable code  [timeout = 30]
src/client.py:15:24: DEF001 error: parameter 'port' of connect() has a default value: 5432  [def connect(host, port=5432):]
src/client.py:18:15: DEF006 error: environment variable "RETRIES" falls back to "3"; os.getenv(...) substitutes a fallback when environment variable "RETRIES" is unset (getenv)  [retries = os.getenv("RETRIES", "3")]
src/client.py:24:5: DEF005 error: except handler converts the exception into a fallback value: return None (EXC002 classification return_fallback; the handler is reported once, here)  [except OSError:]
src/client.py:24:5: EXC001 error: except handler of 'OSError' is forbidden: catch of 'OSError' is not approved by any boundary for this file  [except OSError:]

shockharness 0.10.0: 1 file(s), 1 analyzed, 0 failed; 5 finding(s), 0 suppressed
  by severity: error=5
  rules applied: MAGIC001, EXC001, EXC002, EXC003, DEF001, DEF002, DEF003, DEF004, DEF005, DEF006, CPLX001, CPLX002, CPLX003, CPLX004, CPLX005, CPLX006, SUPP001, SUPP002
  complexity: 3 unit(s) in 1 file(s); conventions cyclomatic/1, cognitive/1, nesting/1, npath/1; 0 NPath saturated; advisory (CPLX001-CPLX004 and CPLX006 are warnings)
  graph: call edges 2 resolved, 1 probable, 1 unresolved; module edges 0 resolved, 2 external, 0 unresolved; 0 cycle(s); 0 shared mutable binding(s)
gate: FAIL (exit 1)
  5 hard violation(s) and gates.new_hard_violations = "fail"

Profiles

Three built-in profiles ship with the package and are selected with profile = "..." (or --profile): strict (default: every rule at its documented severity, unparseable files fail), recommended (strict, but DEF003/DEF004 fallbacks are review, module-level named constant definitions such as MAX_PAGES = 8 pass MAGIC001, fan-out is review above 30, and DEF* findings the analyzer could not establish (unknown) are reported without gating; included files no adapter parses are warned about, as in strict), minimal (MAGIC001, EXC* and DEF* all capped at review, syntax failures warn, complexity off: everything is reported, nothing but malformed suppressions fails). Precedence is bundled defaults < profile < repository configuration < --profile / --parse-errors. Any profile below strict is printed as a policy deviation in every report, and the report's effective_profile names it.

Adopting ShockHarness

This section is written for a coding agent (or a person) that was told to set up ShockHarness in a repository and is reading it over HTTP. It is complete on its own: nothing here depends on another page, and the detailed documents are linked with absolute URLs at the end. Run every command from the repository root.

Step 1: install a pinned version

The version is always an exact pin; nothing updates to latest. Pick one:

uv tool install 'shockharness==0.10.0'                    # isolated tool environment; any repository, JavaScript included
uvx --from 'shockharness==0.10.0' shockharness version     # one-off run, nothing installed
pip install 'shockharness==0.10.0'                        # Python project: keep the pin in pyproject.toml (a dev extra) or requirements-dev.txt

The analyzer needs Python 3.11 or newer to run; the analyzed code may be Python, JavaScript or TypeScript. Its only runtime dependencies are tree-sitter, tree-sitter-javascript and tree-sitter-typescript. The pin that CI installs must live where an updater can read it (pyproject.toml, a requirements file, or the workflow input in step 4), never only inside a shell command.

Step 2: write the policy file

shockharness init                 # writes .shockharness.toml: strict profile, every default spelled out, tests exempt
shockharness config validate      # exit 0; prints the profile and every policy deviation

Commit .shockharness.toml and adjust include/exclude so only source is analyzed; init already exempts tests from MAGIC001 and DEF001. Never switch a rule off to get past a finding (an override, a disabled rule, a lower severity, a wider exclude): that is the user's decision, and check and the hook report it as VERIFY003 until a person commits it. A catch boundary naming one file is the remedy EXC001 asks for and needs no sign-off.

Step 3: first run, and the four onboarding paths for an existing codebase

shockharness check .              # exit 0 pass, 1 violations, 2 invalid configuration, 3 tool failure or unparseable file

A codebase that predates the policy will have findings. Choose one of four paths. All four are visible in every report, so none of them hides anything.

On pull requests prefer shockharness gate --base origin/main --head HEAD (use the repository's default branch; both revisions must be committed): it exits 1 only on hard violations that are new relative to the base.

Step 4: CI

Direct invocation works on any CI system. Install the pin from the manifest, then run the analyzer; a report is written even when the gate fails:

- run: pip install -e ".[dev]"      # the pin lives in pyproject.toml; a JavaScript repository uses the reusable workflow below instead
- run: shockharness check . --format sarif --output shockharness.sarif

On GitHub, call the reusable workflow. Pin its ref to a release tag or a full commit SHA. The analyzer version is the shockharness-version input, not the ref; the two are separate pins:

name: ShockHarness
on:
  pull_request:
  push:
    branches: [main]
permissions:
  contents: read
jobs:
  shockharness:
    uses: evolvingstuff/ShockHarness/.github/workflows/check.yml@v0.10.0
    permissions:
      contents: read
      security-events: write
    with:
      shockharness-version: "0.10.0"
      base: ${{ github.event_name == 'pull_request' && github.event.pull_request.base.sha || '' }}

Add .github/dependabot.yml so both pins get upgrade pull requests. Dependabot bumps the uses: ref and any pyproject.toml or requirements pin; it does not rewrite the shockharness-version input, so bump that by hand in the same pull request:

version: 2
updates:
  - package-ecosystem: pip
    directory: /
    schedule:
      interval: weekly
  - package-ecosystem: github-actions
    directory: /
    schedule:
      interval: weekly

Step 5: install the agent skills

Six skills ship inside the package (shockharness skills list names them).

shockharness skills install --agent claude                              # .claude/skills/<name>/SKILL.md in this repository
shockharness skills install --agent claude --hook                       # also: findings after every edit (.claude/settings.json)
shockharness skills install --agent claude --hook --lock                # also: a lock file the project's configuration cannot loosen
shockharness skills install --agent codex                               # .agents/skills/<name>/SKILL.md in this repository
shockharness skills install --agent generic --project docs/agent-skills # any agent; prints the line to add to AGENTS.md
shockharness skills update --agent claude --check                       # exit 1 when an installed skill is stale or edited

Add --user to install into ~/.claude/skills or ~/.agents/skills instead. install writes .shockharness.toml (as init does) when the project has none, and never overwrites a file without --force. A manifest, .shockharness-skills.json, records a SHA-256 per file: skills update rewrites unedited files and reports edited ones as conflicts. Put skills update --check in CI to detect drift after an upgrade.

Step 6: verify

Each command's output confirms one step.

Detailed documents: configuration https://github.com/evolvingstuff/ShockHarness/blob/main/docs/configuration.md, rules https://github.com/evolvingstuff/ShockHarness/blob/main/docs/rules.md, CI and Dependabot https://github.com/evolvingstuff/ShockHarness/blob/main/docs/integration.md, skills https://github.com/evolvingstuff/ShockHarness/blob/main/docs/skills.md, limitations https://github.com/evolvingstuff/ShockHarness/blob/main/docs/limitations.md.

Configuration

.shockharness.toml or shockharness.toml (one of them; both is an error) or a [tool.shockharness] table in pyproject.toml (the dedicated file wins and the shadowed table is reported). Unknown keys, bad types, unknown rule IDs, and invalid globs are errors, never ignored. Every rule can be disabled; every disable is listed in the report. Per-file results are cached under .shockharness-cache/ (add it to .gitignore; cache.enabled = false or --no-cache turns it off; a warm run only redoes the cross-file work).

[tool.shockharness]
profile = "strict"
include = ["src/**/*.py", "src/**/*.js"]
exclude = ["**/node_modules/**", "**/.venv/**"]
parse_errors = "fail"

[tool.shockharness.rules.magic_numbers]
enabled = true
allowed_values = [0, 1]
allow_unary_negative_one = true     # -1 for items[-1] and indexOf() === -1 (the default)

[[tool.shockharness.overrides]]
roles = ["test"]          # or paths = ["tests/**"]; exactly one of the two
disable = ["MAGIC001"]
reason = "Fixtures encode literal expected outputs"

Every file gets one role (generated, vendor, config, test, docs, else production) from the [tool.shockharness.file_roles] globs, first match in that order; the defaults cover tests/**, **/test_*.py, **/*.test.js, docs/**, **/vendor/** and the like. Roles label files in the report and scope overrides and the MAGIC001 configuration-file exemption; they never change what is analyzed. Every finding has a status: violation, or unknown when the rule could not establish the property (an exception type that does not resolve, a .get(k, v) on a receiver not proven a mapping). fail_on_unknown per rule family decides whether unknown findings gate; the exceptions and defaults families keep failing closed by default. See docs/configuration.md.

Inline suppressions need an exact rule ID and a reason, on the same line:

status = 404  # shockharness: ignore[MAGIC001] reason="RFC 9110 section 15.5.5"

A malformed suppression is a SUPP001 error; one that silences nothing is a SUPP002 warning. --no-inline-suppressions disables them for centrally governed policies.

CI

Direct invocation works on any CI system: install the pin, run shockharness check . (or gate --base <base> --head HEAD on pull requests). GitHub users can call the reusable workflow, pinned to a tag or a full commit SHA; the analyzer version is the shockharness-version input, not the ref:

jobs:
  shockharness:
    uses: evolvingstuff/ShockHarness/.github/workflows/check.yml@v0.10.0
    permissions:
      contents: read
      security-events: write
    with:
      shockharness-version: "0.10.0"

It uploads SARIF to code scanning. Keep the version pin in pyproject.toml, a requirements file or that workflow input, never in a shell string, so Dependabot can open upgrade PRs; docs/integration.md has the full recipe and a Dependabot snippet, and examples/python and examples/js are complete consumers exercised by CI. Restoring .shockharness-cache/ between jobs is optional and safe: an entry from another tool version, configuration or interpreter is invalidated, never used.

A project tested on Python 3.10 runs the gate as its own job on 3.11+ (integration.md, section 4); an empty tree passes with 0 file(s) (integration.md, section 3); and limitations.md lists the checks a layered checker should keep until the planned milestones land. shockharness exceptions is the reviewer's view of every exception the code and configuration make (configuration.md).

See docs/ for configuration, rules, complexity conventions, architecture, integration, skills, releasing, and limitations; evals/ holds the skill evaluation corpus. Every JSON report carries metrics.complexity (per-unit numbers, conventions, thresholds, per-file totals), metrics.graph (edges with confidence, module edges, cycles, fan-in/out, shared mutable state) and metrics.history (churn, co-change, hotspots); none of them gates.

Library

from pathlib import Path
from shockharness import analyze_paths
from shockharness.analysis import CACHE_CONFIGURED, AnalysisOptions
from shockharness.config import load_config

config = load_config(Path("."), None)
options = AnalysisOptions(inline_suppressions=True, cache=CACHE_CONFIGURED)  # or CACHE_BYPASSED
report = analyze_paths([Path("src")], Path("."), config, options)
print(report.gate.status, len(report.findings))

# Optional: apply a reviewed baseline and attach git history metrics.
from shockharness.api import apply_baseline, with_history
from shockharness.gating.baseline import load_baseline

report = apply_baseline(report, load_baseline(Path(".shockharness-baseline.json")), config)
report = with_history(report, Path("."), "HEAD", config)

Development

uv pip install -e ".[dev]"
pytest -q
ruff check src tests scripts evals && ruff format --check src tests scripts evals
shockharness check src tests scripts evals   # the tool gates its own source
python -m build && python scripts/consumer_upgrade_check.py --dist dist --examples examples
python evals/run_evals.py --reference   # the skill eval corpus: every reference solution scores clean

ShockHarness runs on itself as a hard CI gate. Its one MAGIC001 exemption is src/shockharness/constants.py; its declared exception boundaries are listed in .shockharness.toml with reasons and enforced by EXC001-EXC003: every try in the tool wraps exactly one external call and converts the failure into a typed diagnostic. DEF001-DEF006 hold too: the tool has no parameter defaults outside tests, no mapping fallbacks, and no or/is None defaults.

License

MIT. See LICENSE.