toolrack

toolrack is the dispatcher engine behind a user-owned scripts repository. It reads a registry of script paths, loads YAML sidecars beside those scripts, and builds a typed Click CLI at runtime.

If you want the starter repository that uses this engine, see toolrack-template.

Quickstart

git clone https://github.com/anfit/toolrack-template my-tools
cd my-tools
python setup_toolrack.py   # venv, installs toolrack-cli from PyPI, wrappers, PATH, completion

The engine is published on PyPI as toolrack-cli; the command it installs is toolrack. To work on the engine itself, clone this repo and pass --toolrack ../toolrack to the setup script for an editable install.

Open a new shell, then add a script and its sidecar:

# scripts/hello/greet.py
import sys
print(f"Hello, {sys.argv[1]}!")

# scripts/hello/greet.yml
description: Print a greeting.
args:
  - name: name
    positional: true
    required: true
    help: Who to greet.

my-tools core register scripts/hello/greet.py
my-tools hello greet Alice        # -> Hello, Alice!
my-tools hello <TAB>              # completes commands, options and choices

Why This Exists

AI has made it dramatically easier to write small, useful scripts. That is the good news. The follow-up news is that once you have twenty of them, "somewhere in a folder" stops being a system.

toolrack is for that moment.

The basic idea is simple:

• AI is very good at helping you produce small scripts quickly.
• A growing script library still needs structure, discoverability, and sane UX.
• AI also benefits from compartmentalization: small scripts with narrow scope are easier to write, review, test, and replace.
• AI work is non-deterministic by nature, so every stable, tested, deterministic thing you can hand off to automation is one less task balanced on your pet clanker's back.

This project is not trying to turn your script repo into a framework empire. It is trying to help your collection of tiny automations stay tidy, callable, and pleasant to use after the novelty phase.

Showcase

The intended flow looks like this:

1. Create a scripts repository with a clear layout. There is a starter repo for that: toolrack-template.
2. Add small scripts under folders that match their area of work: github/, jira/, db/, images/, deploy/, and so on.
3. Put a sidecar next to each script so the command has typed arguments, help, and completion.
4. Plug toolrack into that repo so the whole thing becomes one coherent CLI.

The result is one command on your PATH, available from PowerShell, Git Bash, or Cygwin, with shared help and completion instead of a growing folklore of "I think that one is in a repo called maybe-tools-old?"

Example:

my-tools github list-prs --state open --limit 20
my-tools jira show ISSUE-123
my-tools db query --env prod --file scripts/db/slow_report.sql

That is the main payoff: when you want to make a new script, it is not trapped in "that project you did two years ago". It lives next to two similar scripts in the folder dedicated to that area, where both humans and models can find the pattern immediately.

Which also makes the AI prompt much nicer:

Hey AI, make me one like this, but mauve.

Testing Your Tools

Installing toolrack-cli also gives your test suite a pytest plugin. It runs the real CLI in a subprocess against a repository, so tests cover what users run: sidecar parsing, argument mapping, exit codes and the script itself. Python, Bash and SQL scripts are all supported.

Check the whole repo in CI. core validate is read-only and exits non-zero on a missing script or sidecar, an invalid sidecar, a bad aliases.cfg, or two scripts that map to the same command. Unregistered sidecar pairs and a stale cache are warnings; --strict makes them failures.

my core validate --strict

Or from pytest, against your real repo (toolrack_project) or a throwaway one (toolrack_repo):

def test_repo_is_valid(toolrack_project):
    toolrack_project.validate(strict=True)   # AssertionError lists every problem

def test_python_tool(toolrack_repo):
    toolrack_repo.add_script(
        "greet/hello.py",
        "import sys\nprint('hi', sys.argv[2])\n",
        {"description": "Say hi.", "args": [{"name": "name", "required": True}]},
    )
    assert toolrack_repo.run("greet", "hello", "--name", "Ada").check().stdout == "hi Ada\n"

def test_bash_tool(toolrack_repo, toolrack_require_bash):      # skips without bash
    toolrack_repo.add_script("ops/fail.sh", "exit 3\n", {"description": "Fails."})
    assert toolrack_repo.run("ops", "fail").returncode == 3

def test_sql_tool(toolrack_repo, toolrack_require_posix):      # no database needed
    psql = toolrack_repo.stub_executable("psql", record_env=("PGDATABASE",))
    script = toolrack_repo.add_script("db/ping.sql", "SELECT 1;", {"description": "Ping."})
    toolrack_repo.run("db", "ping", env={"PGDATABASE": "dev"}).check()
    assert psql.calls()[0]["argv"] == ["-f", str(script)]

SQL scripts run as psql -f <file>, so stub_executable puts a recording fake psql first on PATH instead of needing a database. It works on POSIX hosts only. ToolrackRepo is in toolrack.testing if you prefer to use it without the fixtures.

Migrating From a Checked-Out toolrack

If you installed the engine editable from a clone (pip install -e path/to/toolrack, distribution name toolrack), switch your scripts repo to the PyPI release:

python3 ./sync_toolrack.py                              # refresh setup_toolrack.py
python3 ./setup_toolrack.py --yes --force --cli-name <your-command>

Setup removes the old editable install, installs toolrack-cli, rewrites the wrappers, and regenerates completion. Full steps, plus the manual route, are in the toolrack-template README.

Deployment Model

toolrack is meant to be installed into a scripts repository's virtualenv (pip install toolrack-cli) and launched through a repo-local wrapper in bin/. The PyPI distribution is toolrack-cli; the package and command are toolrack.

The wrapper is responsible for:

• choosing the visible command name
• setting TOOLRACK_REPO_ROOT
• setting TOOLRACK_SCRIPTS_ROOT
• setting TOOLRACK_REGISTRY_FILE or TOOLRACK_REGISTRY_BASENAME
• invoking python -m toolrack from the repo's virtualenv

Defaults:

• CLI name: inferred from the wrapper filename, else toolrack
• registry basename: .toolrack
• repo discovery also accepts legacy .attic during walk-up

Security Model

toolrack runs the scripts a repository registers, with your privileges and environment. Treat a scripts repository like any code you execute:

• Only run toolrack (or its wrapper) inside repositories you trust. Invoking a command in a cloned, unreviewed repository executes that repository's scripts.
• Registry entries in .toolrack are not confined to the repository: a path such as ../x.sh or an absolute path is run as written. Review registry changes in pull requests.
• Argument values are passed to scripts verbatim; scripts must validate their own input.
• .toolrack.cache.json is a machine-local cache containing absolute paths. Keep it out of version control (core init adds it to .gitignore).

Core Commands

toolrack provides a small management surface under core:

• core register <script_path>: add one script to the registry
• core auto-register: scan scripts/ for sidecar-backed scripts
• core unregister <script_path>: remove one script from the registry
• core reregister: validate every registered script and sidecar
• core refresh-cache: rebuild the cached command metadata next to .toolrack
• core list: show registry entries and sidecar status
• core install-completion <shell>: print shell completion source
• core validate [--strict]: read-only repository check for CI (see Testing Your Tools)
• core init [path] [--no-example]: create a minimal scripts repository; never overwrites existing files
• core doctor: diagnose the installation and setup (legacy toolrack install shadowing toolrack-cli, missing bash or psql for registered scripts, command not on PATH, validation errors); exits 1 on failures

Starting without the template: pip install toolrack-cli, then python -m toolrack core init my-tools and cd my-tools.

Scripts Repo Contract

A scripts repository normally contains:

• scripts/: runnable .py, .sh, .bash, or .sql files
• .toolrack: registry of enabled scripts
• .toolrack.cache.json: cached command metadata used for faster startup and completion
• aliases.cfg: optional group aliases, stored next to .toolrack
• bin/: wrappers that export the toolrack environment variables

aliases.cfg lets you shorten first-level or nested group paths without renaming folders. For example:

[groups]
environments = env
refactoring = refactor

That turns your-tools environments get-status into your-tools env get-status.

Sidecars live next to scripts as <script>.yml. See SIDECAR_SPEC.md.

For a starter repository layout, use toolrack-template.

Fast --help

--help and --version are also answered from the cache without importing Click (about 95 ms instead of 190 ms, close to Python's own startup time). The output is identical to Click's; help it cannot reproduce exactly (very long descriptions that wrap, env or epilog sections, non-ASCII text, core subcommands) is rendered by Click as before. Set TOOLRACK_NO_FAST_HELP=1 to bypass the fast path, for example when debugging a sidecar.

Mistyped commands get a suggestion: my exmaple prints No such command 'exmaple'. Did you mean 'example'?.

Shell completion

Completion is answered from .toolrack.cache.json without importing Click, so stays fast; anything the fast path does not model exactly falls back to Click. core install-completion <shell> prints a script that calls the Python interpreter directly and falls back to the wrapper if that interpreter moves. This applies to bash, zsh and fish; the zsh and fish scripts are covered by string-level tests only, not exercised in a real zsh or fish, so report any problems.

Development

python -m pip install -e ".[dev]"
python -m pytest
python -m pytest -m integration -s