Hungry Crab

                              o    .
                           .   o   .   O
     /\                                                     /\
    /  \       _____________________________________       /  \
   |    |     /                                     \     |    |
   | /\ |     |     \\\\\\\             ///////     |     | /\ |
   | \/ |     |      (@@@)               (@@@)      |     | \/ |
    \  /      |                                     |      \  /
     \ \      |      /\/\/\/\/\/\/\/\/\/\/\/\/      |      / /
      \ \_____|                                     |_____/ /
       \______\_____________________________________/______/
                  /     /      |   |      \     \
                 /     /       |   |       \     \

Eat a foreign repository. Digest it. Serve what is worth keeping, legally.

Somewhere out there is a repository that fixed your flaky CI two years ago, wrote the AGENTS.md you keep meaning to write, and learned the hard way which file breaks every single time anyone touches it. Reading it properly costs you an afternoon. Reading fifty of them costs you a month.

The crab reads them for you. It drags the prey into a local cache, dissects it with fifteen deterministic miners, and boils a whole repository down to a digest small enough for an agent to actually read. Then it holds that digest against your repository and serves a ranked menu: what they have, what you lack, what it would cost you, and exactly what their license lets you take. Approve a few and they land as issues in your tracker, each with evidence links and a trace footer. Say no to one and the crab remembers, so it never offers it again.

The prey is never executed. Not one line of its text reaches your issues unless the license says it may.

Status: 0.6.2 is the latest release, a security release on 0.6.0 Everywhere, a limited stable one, and the first one built and published to PyPI from CI. Update to it: it fixes a published vulnerability in the prey guard and the regressions of 0.6.1 (see the changelog). The supported scope is deterministic, human-supervised analysis: the CLI, scoped local MCP, portable skills and offline HTML reports. PR publication, Scheduled Crab work autonomy and measured learning/usefulness remain experimental; their live acceptance criteria are deferred to post-release work, not declared complete. Start with the Everywhere guide and the release record.

The metaphor, in five words

Install

Prerequisites: Python 3.11+, git, and gh authenticated for the GitHub API.

The CLI, which every agent then calls, is the hungry-crab package on PyPI:

uv tool install hungry-crab

pipx install hungry-crab works as well. Either installs the latest release, and uv tool upgrade hungry-crab (or pipx upgrade hungry-crab) takes the next one.

The installation keeps PyYAML as its only runtime dependency. GitLab.com public reads do not require gh; private reads use GITLAB_TOKEN. GitHub Discussions and JUnit artifact reads need the corresponding authenticated API permissions.

To track master instead, which is always green and carries fixes before they are released, install from this repository:

uv tool install "hungry-crab @ git+https://github.com/drevendev/HungryCrab"

Claude Code, which adds the eat/license/serve/cleanroom/loop skills, sniff/menu commands, three subagents and two PreToolUse hooks:

claude plugin marketplace add drevendev/HungryCrab

claude plugin install crab@hungry-crab

Claude Code offers a plugin update when the plugin's version changes, which happens at each release rather than at every commit to master. It does not refresh third-party marketplaces on its own: turn that on once in /plugin → Marketplaces → hungry-crab → Enable auto-update, or refresh by hand with claude plugin marketplace update hungry-crab followed by claude plugin update crab@hungry-crab.

Codex, same plugin, same marketplace:

codex plugin marketplace add drevendev/HungryCrab

codex plugin add crab@hungry-crab

Restart the agent session afterwards so it picks the plugin up. Cursor and anything else that reads the open SKILL.md format can use the skills/ folder of this repository directly; the only hard requirement is crab on PATH. The two safety hooks need no install of their own: each runs hooks/guard.py from the plugin with the first Python 3.11+ the shell finds (python, python3, or one uv python find knows), and the launcher imports the guard from the plugin's own src/, so a plugin-only install is guarded too.

To see what is out of date across the CLI and every agent you have, ask the crab:

crab update

It reports the installed and available versions (the latest release on PyPI for a PyPI install, master for one from git), notices which agents are on this machine and whether the plugin is installed in each, and prints exactly what to run. crab update --run does the plugin work for you.

Verify the CLI installation

After installing the CLI, verify that the crab command is available without GitHub authentication or accessing a remote repository:

crab --version
crab --help

Both commands should exit with status 0. crab --version prints crab followed by the installed version, while crab --help displays usage information, available commands, and options.

These read-only smoke checks verify CLI availability and argument parsing only. They do not validate repository fetching, digest generation, comparisons, or GitHub issue/PR publishing.

For the complete workflow, see Feed the crab below. The first crab sniff example accesses GitHub and requires network connectivity. Authentication requirements depend on the operation; use only the permissions necessary for the intended workflow.

Feed the crab

For repeated improvement, Scheduled Crab persists one phase per scheduler wake-up: crab loop init, next, record, pause/resume and measured cost history. The /crab:loop skill supplies judgement; the CLI bounds publication and waits for human merges. Work needs explicit maw-owner consent. Live rollout evidence is tracked separately from tests.

With an agent, one line does the whole protocol: judge the menu, ask you, create the issues.

/crab:eat pallets/click

By hand, the same protocol is six commands:

crab sniff pallets/click --maw .                  # license, size, activity: is it worth eating?

crab init                                          # .crab.yml: hunger, serve policy, ledger mode

crab compare pallets/click --maw . --issues 300   # digest both sides, diff, score -> gap.md, menu.md

crab menu pallets/click --top 15 --category ci     # the ranked menu of nutrients

crab serve pallets/click --maw . --ids crab:ci:ci.cache --as dry-run   # preview the issue

crab ledger mark crab:tooling:tooling.renovate rejected --reason "dependabot is enough"

crab compare catches and digests the prey the first time; digests are addressed by commit SHA, so eating the same commit twice is free. Giants take --since 2y (history newer than two years) or --shallow (default branch, tree only).

Feeder: a menu in CI

Run crab eat pypa/pipx --deterministic --maw . to export a complete meal without a model, issue creation or ledger writes. The Feeder guide shows the composite action, reusable workflow, weekly schedule, artifact outputs and strict mode.

Local MCP and HTML reports

Install with the optional [mcp] extra and run crab mcp --root /absolute/workspace to expose crab_digest and crab_menu. Local roots are explicit, network reads are opt-in and neither tool publishes or writes the ledger. crab mcp --check checks the optional install and exits.

crab report /path/to/digest-or-meal --out report.html creates a self-contained browser report from existing artifacts, with no network, JavaScript or model calls. See the Everywhere guide for the development-candidate install command, Codex/Cursor/Claude configurations, portable skills and the clean-install verification.

Configuration

crab init writes .crab.yml in the maw. Every key is optional; omitted keys use the defaults below. Unknown top-level, hunger, budget, serve, and trust keys are usage errors (exit 2), with a suggested spelling where available; scoring retains its own flexible nested schema. serve.max_prs_per_run must be a non-negative integer. trust.same_owner and trust.bypass_license accept YAML booleans only: write false, not the string "false".

• license: SPDX id for the maw, or auto-detect from LICENSE when omitted.
• mode: normal or strict; strict downgrades COPY code to clean-room REIMPLEMENT while configurations and templates stay copyable.
• profile: reviewed hunger defaults for balanced, library, CLI, service or frontend projects.
• memory: bounded learning from confirmed owner decisions, with an explicit off switch.
• hunt: bounded GitHub discovery queries, size/stars/license filters and revisiting policy.
• hunger: enable, disable, or cap each nutrient category with issues-only / ideas-only.
• ignore: maw-side git-style paths excluded from its own digest, in crab compare and in crab digest <maw> --maw <maw>; a prey's own .crab.yml is data and is never read.
• budget: whole-digest Markdown budget policy: warn (default), enforce, or off.
• serve: issue/PR serving policy, labels, assignees and token source; serve.prs gates crab serve --as pr-branch, which publishes REIMPLEMENT nutrients from clean-room receipts and COPY nutrients from materialization receipts, with their attribution.
• trust: same-owner and explicit-owner relationships plus the visible license-bypass escape hatch.
• attribution_file: the notice file for copied sources, rendered by crab attribution from the receipts in .crab/attributions.json that crab serve --as pr-branch writes when it carries prey files in; crab attribution --check is the CI gate. The file is headings and paragraphs, plus the source's licence and NOTICE texts in fenced blocks, so that a Markdown formatter's defaults leave it alone; if yours rewrites it anyway, exclude the file from the formatter, not from the check.
• ledger: store meal history in the repository, cache, or nowhere.
• scoring: per-section overrides for data/scoring.yml; crab tune can suggest them.
• loop: scheduled phase policy, owner work consent, fixed prey, budgets and recovery limits; see Scheduled Crab.

This is the current commented template written by crab init:

# Hungry Crab maw configuration. Every key is optional; these are the defaults.
license: null              # SPDX id of this repository; detected from LICENSE when null
mode: normal               # normal | strict. Strict downgrades COPY code to clean-room
                           # REIMPLEMENT; configs and templates remain copyable.
profile: balanced          # balanced | library | cli | service | frontend; explicit hunger wins
memory:
  enabled: true            # confirmed decisions influence ranking; explicit weights still win
  min_decisions: 3
  strength: 0.3            # bounded multiplier; never changes license or hunger ceilings
hunt:
  queries: []              # derive up to four GitHub searches from enabled maw gaps
  exclude: []
  licenses: []
  min_stars: 20
  max_repo_kb: 307200
  max_candidates: 50
  limit: 10
  include_seen: false
  allow_unknown_size: false
hunger:                    # per nutrient category: true | false | issues-only | ideas-only
  security: true
  ci: true
  tests: true
  tooling: true
  ai-config: true
  hygiene: true
  docs: true
  deps: true
  history-lesson: true
  issue-lesson: true
  architecture: issues-only
  code: ideas-only         # produced by nothing until 0.4's postponed syntax index: inert
ignore: []                 # globs excluded from this repository's own digest, so that test
                           # fixtures and vendored trees are not mistaken for your code, e.g.
                           # [tests/fixtures/**, examples/**]. Patterns are case-sensitive on
                           # every platform, like the git paths they match.
budget:
  policy: warn             # warn | enforce | off for the whole digest Markdown budget
serve:
  issues: ask              # auto | ask | off
  prs: ask                 # auto | ask | off; gates --as pr-branch (REIMPLEMENT and COPY)
  max_prs_per_run: 3
  labels: [hungry-crab]
  assignees: []
  token_env: ""            # environment variable holding the token to file issues as, e.g.
                           # CRAB_BOT_TOKEN with a GitHub App installation token. Empty means
                           # gh's own login: your issues carry your name, not the crab's.
trust:                     # a license is a promise to strangers; these are not strangers
  same_owner: true         # prey owned by the account that owns this repository is your own code
  owners: []               # other accounts whose repositories count as your own, e.g. [acme-inc]
  bypass_license: false    # last resort: treat every prey as COPY. Every card says so, and the
                           # verdict is flagged for human review, because this is not a finding
                           # about the license but a decision to stop asking.
attribution_file: THIRD_PARTY_NOTICES.md
                           # where COPY records its sources: rendered by `crab attribution`
                           # from the receipts in .crab/attributions.json, which
                           # `crab serve --as pr-branch` writes when it carries prey files in.
ledger: repo               # repo (.crab/ledger.json, committed) | cache | none
scoring: {}                # overrides for data/scoring.yml sections; `crab tune` suggests them
loop:
  cadence: daily           # cadence belongs to your scheduler; the CLI never starts one
  autonomy: serve          # read | serve | work; merging always belongs to a human
  work_authorized: false   # the maw owner must explicitly authorize work, including tags
  discovery: false         # opt in to a lease-bound HUNT shortlist
  prey: []                 # fixed allowlist; empty requires discovery
  budget:
    phases_per_day: 4
    prey_per_round: 2
    open_issues_max: 10
    open_prs_max: 2

What the miners extract

crab digest owner/repo writes a token-budgeted digest/ folder that an agent reads progressively:

manifest.json is the entry point: every file with a token estimate, the miners that ran, a small summary and the suggested reading order. Markdown is paged at 3,500 tokens per page by default (history.md, history.2.md, and so on; nothing is dropped), and the whole digest has a policy for its 30,000-token reading budget — warn by default, enforce for a budgeted loop, off for a human, set under budget in .crab.yml; JSON keeps the full data for scripts.

For crab digest --out, use an empty directory or an existing digest directory. The crab replaces and removes only artifacts recorded in that directory's previous valid manifest. Other files are preserved; a conflicting caller-owned name is refused before publication, including case-insensitive collisions.

Guiding principle: scripts squeeze out everything that can be squeezed deterministically; the model is spent only on judgment.

From digest to issues

• compare turns the two digests into candidate nutrients: trait rules (ci, security, tooling, hygiene, docs, ai-config, tests), tool and dependency diffs within shared ecosystems, README sections, commit and changelog discipline, history and issue lessons, architecture raw material. Every candidate carries a stable id, evidence linked to the prey commit, effort, risk and a license mode.
• scoring is a formula you can read: category weight x value x uptake x license mode x effort, minus risk. Weights live in data/scoring.yml and are overridable per maw.
• serve creates issues with a hidden crab:<id> marker, a label and a trace footer, so a rerun creates no duplicates. The why and how come from the agent through a notes file.
• ledger (.crab/ledger.json by default) remembers every meal and decision; rejected and served nutrients vanish from later menus. crab tune reads it and tells you which weights to move, which categories to switch off, and which prey were a waste of time.

Licenses are decided by an engine, not an opinion

A deterministic maw license x prey license matrix gives every nutrient one mode:

The three cases where the engine will not help you look alike and are not:

Issue and discussion text is always IDEAS_ONLY, because the copyright belongs to the commenters. This is a compliance aid, not legal advice.

Safety

• Prey code is never executed. The miners read files and run read-only git plumbing. No npm install, pytest or make ever runs inside the cache.
• Prey content is untrusted data. Markdown summaries carry structure (headings, names, counts), never the body of README or agent-instruction files, and instruction-like fragments are flagged in the JSON.
• Least privilege. sniff and catch need read access to the GitHub API; serve uses your own authenticated gh and creates nothing until you ask it to.
• Two PreToolUse hooks make the first rule mechanical in Claude Code. crab-prey-guard refuses a Bash or PowerShell command that touches the prey cache unless it is one of a few audited read-only shapes (in PowerShell, a read cmdlet such as Get-Content or Get-ChildItem with literal arguments), and crab-cleanroom-guard keeps the clean-room implementer out of the cache altogether. They run from the plugin's own source tree through hooks/guard.py, with the first Python 3.11+ the shell finds, so nothing has to be installed or signed for them; when not even an interpreter is there, the hook says so and the call proceeds, because a guard that blocked every call over its own setup would be switched off rather than fixed. Codex does not run them at all: a plugin with a root plugin.json is loaded through its Agent Plugins loader, which has no hook support (openai/codex#39895). The prey guard has been observed refusing a command in a live session; the clean-room guard has not yet (#83, #141).

Benchmarks

uv run python benchmarks/run.py pallets/click colinhacks/zod

The milestone limits are 120 seconds and 30,000 Markdown tokens per digest. Results land in benchmarks/results/; see benchmarks/README.md.

Development

uv sync                      # environment with the dev group

uv run pytest                # tests build three synthetic git repositories as fixtures

uv run ruff check . && uv run ruff format --check . && uv run mypy

uv run crab digest . --maw . --out digest-out --maw-license MIT   # the crab eats itself

See CONTRIBUTING.md and AGENTS.md, the guide coding agents read.

Design

The design documents live in docs/design:

• Concept and skill form factor: the three layers, the digest format, the license engine, security, the decisions log.
• MVP: commands, miners, milestones 0.1 to 0.3, acceptance criteria.
• Roadmap: base crab, evolving crab, forks.
• Evolving Hungry Crab: the self-improving loop.

License

MIT. The tool helps with license compliance; it is not legal advice.