Skip to main content

hitch

Your code depends on things being true. This writes them down and tells you when they stop.


selftest recall deps python license


What goes wrong

Your agent uses a plain in-memory queue instead of pulling in Celery. Good call, as long as one thing writes to it.

Six months later someone adds a second worker in a different file. The queue starts losing jobs. No crash, no error log, no failing test. And nothing in the repo ever said "one writer only".

The code is fine. The thing it was counting on stopped being true, and nobody noticed.

What you do about it

Write the condition down, right next to the code:

# hitch[queue-single-writer]: exactly one writer touches this deque
#   | holds: !grep:^\s*def add_worker@backend/**/*.py; !dep:celery
#   | breaks: two writers means dropped jobs, silently
#   | then: add a lock in enqueue(), or move to a real queue

Then let CI watch it:

$ hitch check
SLIPPED  exactly one writer touches this deque  (backend/queue.py:12)
        slipped  !grep:^\s*def add_worker@backend/**/*.py
          backend/workers.py:14: def add_worker(n):
        breaks: two writers means dropped jobs, silently
        then:   add a lock in enqueue(), or move to a real queue

7 hitches  1 slipped  0 error  0 unchecked  6 hitched  (143 files, 13ms)
$ echo $?
1

Build goes red the day someone breaks a call you made months back.

Install

pip install -e .          # gives you the `hitch` command
hitch selftest            # check the checker before you trust it
hitch check .

No dependencies. Python 3.11+.

Ships as hitch-check on PyPI. There's an unrelated abandoned package called hitch from 2016 that also installs a hitch command. If you have that one, the commands will clash.

Skills go wherever your agent keeps them:

cp -r skills/hitch skills/hitch-triage ~/.claude/skills/

Pre-commit hook and GitHub Action are in integrations/.

Commands

hitch check [path]     check them all, exit 1 if one broke
hitch ls [path]        list them, flag the ones with no check
hitch lint [path]      grade the markers themselves
hitch stats [path]     coverage numbers
hitch selftest         prove the checker catches its own bad cases

Everything takes --json.

Checks you can write

All of these run in microseconds. No model involved. Put ! in front to flip it.

check passes when
grep:PATTERN@GLOB the pattern shows up in a matching file
only:PATTERN@GLOB it shows up in exactly one file
count:PATTERN@GLOB<N fewer than N hits
dep:NAME the package is in any manifest
loc:PATH<N size:PATH<N file is under N lines / bytes
files:GLOB<N fewer than N files match
exists:PATH the path is there

Manifests get found anywhere in the tree: npm, pip, poetry, uv, Pipenv, Go, Cargo, Bundler, Composer, Gradle, Maven, pub. Globs handle ** and {a,b}.

Two things worth getting right:

Anchor your patterns. !grep:add_worker also matches a commented-out line and your own design doc. !grep:^\s*def add_worker doesn't. Measured: that's worth 11 points of accuracy.

Only point globs at files you know exist. A check aimed at **/*.tf in a repo with no terraform never passes and never fails. It just errors forever. Use dep: or exists: if you aren't sure.

Why not just write an ADR

ADRs are fine. Problem is they're dead the moment you save them. Prose in a folder nobody opens, not linked to the code, and no way to tell when they've gone out of date.

ADR TODO fitness functions hitch
sits next to the code no yes no yes
a machine can read it no no yes yes
says why you picked it yes no no yes
gets re-checked no no yes yes
can be retired no no no yes

Fitness functions (ArchUnit, import-linter, dependency-cruiser) are the closest thing. Those say "this must always be true". A hitch says "this was true when I picked, here's what I turned down, here's what to do when it changes". Rules are permanent. Decisions run out.

The reason this works: the agent that just made the call already knows all of it. Writing it down costs one line. That's the whole trick, and it's why ADRs die as a human habit but don't have to here.

Numbers

Does the checker work

precision and recall

18 fake commits against a repo with 7 hitches in it. 8 of them should trip a check, 10 shouldn't. It caught all 8 and named the right one every time.

The 10 that shouldn't trip it include some nasty ones on purpose: the word sitting in a doc, a similar-looking function name, a commented-out definition. That last one is the single miss with lazy patterns, and it goes away if you anchor. Both numbers are published because the gap is the point.

Plus 65 unit cases, one good and one bad reference per check type. A checker that never fires passes all the good ones for free, so the bad ones are what actually matter.

Does the skill work

agent benchmark

36 agents, each in a fresh session with no access to the repo. Four setups:

  • no skill does nothing
  • one sentence ("leave a comment about assumptions") gets you a prose note 100% of the time, but nothing a machine can check
  • syntax only gets the format and the full list of checks in about 15 lines
  • hitch skill gets the real thing

syntax only is the one that mattered. If a skill can't beat its own cheat sheet, it isn't worth having. Here's why it does:

markers written checks per marker mostly used
syntax only 76 1.00 grep
hitch skill 14 1.50 dep

It writes fewer markers and better ones. It pushes agents off brittle text matching onto dependency checks, and gets them covering more than one way a thing can break. 76 markers vs 14 for the same work.

The triage skill didn't help

There's a second skill for deciding what to do when a check fails. Tested it on four scenarios against a bare "work it out and say why" prompt. Both got 4/4. The skill made the answers consistent and about 40% shorter, but it did not make the decisions better. Reported as the weak result it is.

Method, iteration log, limitations: docs/BENCHMARK.md.

What this doesn't claim

  • Small study. 18 fake commits, 9 tasks, two runs each, one model.
  • These checks are a floor, not proof. !dep:celery doesn't prove nobody hand-rolled a queue.
  • Nobody's actually been saved by one yet. All of this measures whether agents write good markers, not whether good markers pay off. That needs a real repo and a few months.
  • It will never print a "you saved N bugs" number. That number doesn't exist.

Docs

Credit

Benchmark approach taken from ponytail: good and bad reference pairs, prove the instrument before spending anything, keep a control arm, publish the result that makes you look worse. The marker-plus-harvester idea is its ponytail: comment, generalised.

Trap framing (name the thing that looks fine but isn't, and say why) from adhd.

MIT.

Metadata

Release files for hitch-check 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for hitch-check 0.1.0
File Size Uploaded
hitch_check-0.1.0.tar.gz 29.4 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for hitch-check 0.1.0
File Interpreter ABI Platform
hitch_check-0.1.0-py3-none-any.whl Python 3 none any Details

Total release size: 57.4 kB

Release files / hitch_check-0.1.0.tar.gz

Download URL hitch_check-0.1.0.tar.gz
Size 29.4 kB
Tags Source
SHA-256 checksum
How to use checksums
76e8706dfdfc160cdcc9c8d707e24c1fc0595c113d530ec3d87fd8d255b2d0e9
BLAKE2b-256 checksum
How to use checksums
beb9f5064904d5f71ab82bf74a649c6160dc8917ffaba7c2d152f68038118f3f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.14.0

Release files / hitch_check-0.1.0-py3-none-any.whl

Download URL hitch_check-0.1.0-py3-none-any.whl
Size 28.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
5a9e1b8a4586c13accb05f6f4a38b1c65d96c5c57f8021227e2a50a540f9ec8d
BLAKE2b-256 checksum
How to use checksums
bf2de1bf93536e30becda5811b92b5e41ee79fee086f235d70292aa11fa7ed48
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.14.0

Release history Release notifications | RSS feed

0.1.1

2 release files

This release

0.1.0 This release

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page