checkwash
Check whether a code change weakens your tests.
A test can pass after it stops checking something useful. checkwash reads your Git diff and flags known patterns such as deleted assertions, skipped tests, relaxed expectations, and CI checks that stop failing.
- assert total == 105.3
+ assert total > 0
The new assertion accepts many incorrect totals. checkwash can flag changes like this for review, including changes written by coding agents.
Runs locally. No LLM. No network during analysis. Never executes your code.
Try it
You need Python 3.11+ and Git. From a repository with at least two commits:
curl -LO https://github.com/taipei49314/checkwash/releases/download/v0.2.13/checkwash.pyz
python checkwash.pyz check HEAD~1..HEAD
This checks your last commit. Start with a change you already understand. You can also download the file in your browser.
| Result | What to do |
|---|---|
| Pass · exit 0 | No finding requires blocking under your configuration. Keep running your normal tests and review. |
| Block · exit 1 | Read the finding and the diff. It may be weakened verification or a false positive. |
| Error · exit 2 | Resolve the input or analysis error before relying on the result. |
For JSON/SARIF output and more examples, see the usage guide.
Know the limits
v0.2.13 is alpha. A pass does not prove that a change is correct or honest. Python is the main language supported; JS/TS support covers a limited set of test patterns. Known gaps remain.
Legitimate refactors can be flagged: the dedicated honest-refactor corpus records 24 blocks out of 60 (40%). Try it on your own changes before making it required. Coverage and limitations · Known gaps
Use it in CI
To stop a merge, make the checkwash status check required in your
repository's branch rules. Installing the tool or adding a workflow alone
does not enforce its verdict.
The recommended Action is pinned to v0.2.12; the CLI above is v0.2.13. Record which version you use. Full setup and exemptions
Copy the GitHub Actions workflow and require the check
Save this as .github/workflows/checkwash.yml:
# .github/workflows/checkwash.yml
on: [pull_request]
permissions:
contents: read
jobs:
checkwash:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
persist-credentials: false
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
- uses: taipei49314/checkwash/action@e05c37f0e1673cdf218ec62fcfb7c6712cce704b # v0.2.12
After the workflow runs, open Settings → Rules → Rulesets and require
the checkwash status context. Keep the job unconditional.
If you administer the repository and have this project's ruleset file, you can instead create the rule with:
gh api repos/OWNER/REPO/rulesets --method POST --input action/required-ruleset.json
This adds a ruleset; it does not replace existing ones. checkwash doctor
can inspect the local workflow, but cannot verify live branch protection.
Why the older Action pin? A release cannot embed its own commit SHA,
so the documented Action adopts a verified pin from the prior release.
This release updates documentation, package metadata and the trusted pin;
its detector logic is unchanged from v0.2.12. To verify another trusted
release, use git rev-parse 'vX.Y.Z^{commit}'.
Action reference
More options and evidence
Install as a CLI or try the included examples
If you already use pipx, install the fixed version:
pipx install checkwash==0.2.13
# or from the release tag:
pipx install git+https://github.com/taipei49314/checkwash@v0.2.13
checkwash check HEAD~1..HEAD
checkwash demo # 8 real tampering cases, blocked, offline
checkwash demo replays eight real tampering cases and one honest fix.
It illustrates known patterns; it is not a coverage guarantee.
You can run the same examples with python checkwash.pyz demo.
Read the measurements and their limits
The historical six-repo sweep recorded 42 / 1800 = 2.33% blocks: 27 false positives (1.50%), 15 legitimate policy blocks (0.83%). The tracked artifacts record engine v0.1.46 on a corpus used to tune the detectors, not a fresh v0.2.13 or held-out result.
1.33% of the corpus (24/1800) records opaque production changes. That flag does not establish that each verdict changed or each diff was unanalyzed.
Review methods vary: the three-rater agreement study covers an older 35-diff cohort, not all 42 blocks. The dedicated 24/60 refactor result above is a separate population; the general-commit rate does not predict it.
Measurements and source data · Generated results · Failure ledger
| Looking for… | Start here |
|---|---|
| Installation checks, versions and first use | v0.2.13 guide |
| JSON/SARIF contracts and upgrades | Stability |
| Required checks and reviewed exemptions | Enterprise setup |
| Contributing or reporting a problem | Contributing · Issues · Security reports |
| Readiness for 1.0 | Criteria — not met |
Related projects: checkwash-corpus and smallestlie hold evaluation work.
Alpha pre-release. 21 detectors, 578 tests in the current source tree. Zero runtime dependencies. Apache-2.0.
1 maintainer
Metadata
1 maintainer
Release files for checkwash 0.2.13
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| checkwash-0.2.13.tar.gz | 788.0 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| checkwash-0.2.13-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 976.5 kB
Release files / checkwash-0.2.13.tar.gz
| Download URL | checkwash-0.2.13.tar.gz |
|---|---|
| Size | 788.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
983375950c3c09db4f54b25591c69360c463fb7470a2c5003edf7ca0688fa672
|
|
BLAKE2b-256 checksum How to use checksums |
415fcb126b877ad95372ffc0c9da7235c358f0d357c3a1211fe64547eb3e5b53
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 5, 2026.
Transparency logRelease files / checkwash-0.2.13-py3-none-any.whl
| Download URL | checkwash-0.2.13-py3-none-any.whl |
|---|---|
| Size | 188.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
c1a127c824b0f1903951c5e0811bd8bf601612b4a5fbfa9216fe37b70e15c2b9
|
|
BLAKE2b-256 checksum How to use checksums |
882091465b967ed8e167370dd13971e9b08eb5bd1c7256758acf65fd5e66075e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 5, 2026.
Transparency log
