Initial setup

git clone <url>
git <project> init --worktree

Add convenience substitution variables

Configured values may name other configured values. Inside a worktree, {path} is the worktree’s path and {worktree} is its name. So a build directory written once gives every worktree its own:

git <project> config srcdir "{path}"
git <project> config builddir "{srcdir}/build/{worktree}"
git <project> config make "make -C {srcdir} BUILDDIR={builddir} {build}"

The Substitution and Scopes sections of the git-project documentation describe the names, and how a worktree’s values override the project’s: https://pypi.org/project/git-project/

Adding custom commands

git <project> run --make-alias configure
git <project> run --make-alias build
git <project> run --make-alias install

git <project> add configure debug "mkdir -p {builddir} && cd {builddir} && cmake -DCMAKE_BUILD_TYPE=Debug {srcdir}"
git <project> configure debug

artifact

The artifact command adds or removes associations between git config objects and file-system objects.

Summary:

git <project> artifact add <subsection> <path>
git <project> artifact rm <subsection> [<path>]

<subsection> is a git config section which will appear under the <project>.artifact section. Artifacts look up objects associated with <subsection> and perform substitutions on paths to yield the final associated file-system object. The artifact rm command simply removes an artifact association, it does not remove the artifact itself. A <subsection> may hold several paths. With <path>, artifact rm removes only that association. <path> is a regular expression that git matches against each path as it was added, before substitution. Escape characters such as { and . to match them literally. A <path> that matches more than one path removes none of them.

For example:

git <project> artifact add worktree.myworktree /path/to/artifact

Presumably, /path/to/artifact is in some way created in association with myworktree, for example by the run command. When we delete myworktree, the artifact association causes /path/to/artifact to also be removed.

Removal runs without a shell. A leading ~ and $VAR references are expanded. A path that exists is removed as written. Otherwise it is expanded as a glob, and each match is removed. A symbolic link is removed, not its target.

Removal refuses the root, the home directory, the current worktree and the repository’s git directory, and any directory that contains one of them. A refused path stops the command before anything is removed.

Substitutions can make artifact associations easier to manage:

git <project> artifact add worktree /path/to/{worktree}/artifact

Notice that we’ve added the artifact under the more general worktree subsection instead of naming a worktree explicitly as before. Because the {worktree} substitution appears in the artifact path, deleting any worktree will cause the worktree’s name to be substituted into the artifact path, forming a unique artifact path to remove. If a worktree also has its own worktree.<name> artifact, removing that worktree uses only its own, not the general one.

We may make this even more general:

git <project> config srcdir "{path}"
git <project> config builddir "{srcdir}/build/{worktree}"
git <project> config make "make -C {srcdir} BUILDDIR={builddir} {build}"
git <project> run --make-alias build
git <project> add build debug "{make}"
git <project> add build release "{make}"
git <project> add build check "{make}"
git <project> artifact add worktree "{builddir}"

We’ve added a single artifact association that will handle any worktree and all of our different build types. When we delete the worktree, all artifacts related to debug, release and check builds will also be removed.

See also:

config
run
worktree

branch

The branch command queries the status of branches against the project’s branches and prunes branches that are finished.

Summary:

git <project> branch status (--all | --all-user | <pattern>) [<target>]
git <project> branch prune [--force] [--no-ask] [--keep-remote-branch] (--all-user | <pattern>)

A <pattern> matches every branch whose name starts with it, and a name without refs/ is looked up under refs/heads/. –all-user matches the branches under user/<login>.

branch status prints a table of the matching branches, with whether each is merged to a project branch and whether it is pushed to a project remote. –all reports on every local branch. With <target>, merged means merged to <target> instead. For example:

git <project> config --add branch release
git <project> branch status mybranch

The project branch defaults to the repository’s main branch, main or master, when none is configured. config –add adds more.

branch prune asks whether to delete each matching branch that is merged to a project branch, locally or on a project remote. If the answer is yes, it deletes the local branch and its copy on each project remote. A pattern matches project branches too, so check the list before you answer.

With –force, branch prune offers every matching branch, merged or not. With –no-ask, it deletes without asking. With –keep-remote-branch, it deletes only the local branch and leaves the remote copies in place. worktree rm takes the same option, and it matters more here, because branch prune acts on every branch that matches the pattern.

See also:

config
worktree

clone

The clone command clones a repository.

Summary:

git <project> clone <url> [<path>] [--bare]

By itself clone does a basic clone, then sets the project’s defaults in the new repository. With no <path> it clones into the last component of <url> under the current directory, keeping any .git suffix, where git clone drops it. It offers no ssh key, so an ssh url fails to authenticate.

Plugins add options to clone. For example, the worktree command adds a –worktree option to have clone create a worktree layout.

See also:

worktree

config

The config command manages git config settings under the <project> section.

Summary:

git <project> config [--add] [--unset] <name> [<value>]

The config command operates much like git’s built-in config command, except all configuration keys are prefixed with <project>, keeping values under a single project namespace. This is a convenient way to store parameters for other commands. For example:

git <project> config builddir /path/to/build
git <project> add run build "make BUILDDIR={builddir} all"

A value may name other values as {name}. The commands that use a value, such as run, substitute it when they run, not when it is set. For example, inside a worktree {worktree} is that worktree’s name:

git <project> config builddir /path/to/{worktree}/build

The Substitution and Scopes sections of the git-project documentation describe the names and how a worktree’s values override the project’s: https://pypi.org/project/git-project/

The config command also adds a config subcommand to worktree, which sets a value for one worktree and takes that worktree’s name first:

git <project> worktree config <ident> <name> [<value>]

See also:

run
worktree

help

The help command displays tutorial-style help for commands.

Summary:

git <project> add help [--manpage] <subsection> <text>
git <project> rm help [--manpage] <subsection>
git <project> help [<command> | <section>... <name>]

With no argument, help shows git-project’s general help. help <command> shows the command’s manual. help <section>... <name> shows the manpage stored for that subsection, so help run build shows the one stored for run.build.

Users may add help to any project config section. For example:

git <project> add help run.build "Perform a build"
git <project> add help run.check "Run tests"

All help is stored under a <project>.help config sub-section. If a command supports it, such help may appear in the command’s own help output by querying the appropriate <project>.help sub-section:

git <project> run --help

<standard help text>

Available runs:
    check                - Run tests
    build                - Perform a build

In this way projects can self-document their configurations. Normally <text> is stored in <project>.help.<subsection>.short. With –manpage, <text> is stored in <project>.help.<subsection>.manpage. Commands may reference short help or manpages in various ways to present help.

See also:

run

init

The init command initializes project state.

Summary:

git <project> init

init itself changes nothing. Any command run in a repository sets the project’s branch to the main branch and its remote to origin, when they are not set already.

Plugins add options to init. For example the worktree command adds a –worktree option to convert an existing local clone to a worktree layout.

See also:

worktree

run

The run command executes commands via a shell.

Summary:

git <project> add run <name> <command>
git <project> rm run <name>
git <project> run --make-alias <name>
git <project> run <name> [<option>...]

Full shell substitution is supported, as well as config {key} substitution, where the text {key} is replaced by key’s value. The command is printed after substitution. The words given after <name> on the command line are shell-quoted, so the shell sees each as one argument and never runs it. In those words only a plain {name} is substituted, and other text in braces is passed on as it is.

The add run command associates a command string with a name, and rm run removes it. The run command itself invokes the command string via a shell. With –make-alias, run registers a new command, such as build, that works like run. For example:

git <project> run --make-alias build
git <project> add build all "make -C {git_workdir} all"
git <project> build all

Each such command keeps its own list of names. So in the example above this fails, because all was added as a build, not a run:

git <project> run all

In this way we may use the same <name> for different commands, which can be convenient:

git <project> build all
git <project> check all

The Substitution section of the git-project documentation lists the names a command may use, such as {git_workdir} and {branch}: https://pypi.org/project/git-project/

The run command adds its own. {run}, or the alias name such as {build}, gives the name of the command being run. An example will make this more clear:

git <project> config cmd           "make -C {git_workdir} BLDDIR=/path/to/{build} {build}"
git <project> add build all "{cmd}"
git <project> add build some "{cmd}"

We have configured two different build flavors, each which place build results in separate directories and invoke different targets. Substitution proceeds as follows:

git <project> build all -> make -C /cur/workarea BLDDIR=/path/to/all all
git <project> build some -> make -C /cur/workarea BLDDIR=/path/to/some some

Extra options passed to the run command may be referenced in the command string:

git <project> add build extra "echo {options}"
git <project> build extra hello world! -> echo hello world!

Individual options may also be referenced:

git <project> add build extra "echo {options_1} {options_0}"
git <project> build extra hello world! -> echo world! hello

A special dash-separated “key” string composed of option values may be generated:

git <project> add build any "make {option_key}"
git <project> build any target debug -> make target-debug

A special {option_keysep} substitution will result in a - if there are options and the empty string otherwise:

git <project> add build any "make my{option_keysep}{option_key}"
git <project> build any target debug -> make my-target-debug
git <project> build any -> make my

Other substitutions may be used in options:

git <project> add build branch "make {options}"
git <project> build branch {branch} -> make dev  # On the dev branch

A special substitution {option_names} gives the options separated by spaces, with their braces removed, so they are not substituted:

git <project> add build branch "make {option_names} {options}"
git <project> build branch {branch} -> make branch dev  # On the dev branch

Inside a worktree, a value set for that worktree overrides the project’s value of the same name. See the Scopes section of the git-project documentation.

See also:

config
worktree

worktree

The worktree command manages worktrees and connects them to projects.

Summary:

git <project> worktree add [-b <branch>] <name-or-path> [<committish>]
git <project> worktree rm [-f] [--keep-branch] [--keep-remote-branch]
                          <name>
git <project> worktree config <ident> [--add] [--unset] <name> [<value>]
git <project> worktree migrate [--apply]

worktree add creates a worktree at <name-or-path> and checks out a branch in it. The worktree’s name is the last component of <name-or-path>. The branch is named after <name-or-path>: a simple name gives a branch of that name, and a relative path such as ../user/topic gives user/topic, the part after any ... That branch is created at <committish>, or at HEAD, when it does not exist yet. An existing branch is checked out as it is, and <committish> is then ignored. With -b <branch>, the new branch <branch> is created at <committish> or HEAD and checked out instead.

A relative path that does not start with .. is placed under the root of the current worktree, or under the current directory in a bare repository. A path that starts with .. is taken from the current directory. So worktree add ../topic, run in the main worktree, puts the new worktree beside it, and worktree add topic puts it inside.

To keep things simple, we’ll usually always name worktrees similarly (or identically) to the branches they reference, though it is not strictly necessary to do so.

worktree rm removes a worktree and its workarea. It first removes the paths that artifact associates with the worktree, and if one of those is refused, nothing is removed. The branch is deleted too, locally and on each project remote, unless a flag says otherwise: --keep-branch leaves the branch alone everywhere, and --keep-remote-branch deletes only the local copy. A branch the project configures is never deleted, whatever the flags say. Removing a worktree whose branch is unmerged requires -f, unless --keep-branch means the branch survives anyway.

The key idea behind project worktrees is that they are connected to various artifacts. Worktrees are managed together with these artifacts to provide a project-level view of various tasks. For example, a run command can create artifacts associated with a worktree. Removing the worktree implicitly removes these artifacts, making build cleanups easy and convenient. Commands may use the {worktree} substitution to create worktree-unique artifacts. Other substitutions may also reference {worktree} in a recursive manner.

Here is a concrete example:

git <project> config srcdir "{path}"
git <project> config builddir "{srcdir}/build/{worktree}"
git <project> config make "make -C {srcdir} BUILDDIR={builddir} {build}"
git <project> run --make-alias build
git <project> add build release "{make}"

Assuming the build system uses BUILDDIR to determine where build artifacts go, each worktree will get a unique set of build artifacts, via the {builddir} and, recursively, {worktree} substitutions. When we delete the worktree, we’ll also delete the associated build directory.

We associate artifacts with worktrees via the artifact commands.

Another important benefit of worktrees and associated builds is that switching to work on a new worktree (by simply editing sources in a different worktree directory) will not result in build artifacts from the previous worktree being overwritten. Thus we avoid the rebuild the world problems of switching branches within the same workarea. Generally, each created branch will have its own worktree and we will rarely, if ever, switch branches within a worktree.

A worktree layers a config scope on top of the global project scope, so that configuring a key in the worktree with the same name as a key in the project will cause the worktree key’s value to override the project key’s value:

git <project> config buildwidth 16
git <project> worktree config myworktree buildwidth 32

The worktree to configure is named explicitly, so myworktree gets buildwidth=32 while the project keeps 16.

Inside myworktree, wherever {buildwidth} appears (say, in a run command), 32 is substituted instead of 16. Outside myworktree, or in a worktree with no buildwidth of its own, {buildwidth} gives 16. The Scopes section of the git-project documentation describes the rule: https://pypi.org/project/git-project/

The worktree plugin also adds a –worktree option to the clone and init commands. Both set up the worktree layout described in the package documentation. The bare repository is a hidden child directory named for the last component of the remote url, such as .myrepo.git. The top-level directory holds it alongside the worktrees.

clone --worktree clones bare, then rewrites the fetch refspec and sets the main branch to track its remote branch, so fetch and pull behave as they do in a regular clone. The refs/heads and refs/remotes namespaces remain, and every other local branch is deleted. Add --bare to skip the refspec rewrite. The clone still gets the .git file and the main worktree. With no <path>, clone --worktree uses the current directory itself as the top-level directory, where a plain clone makes a new one.

init --worktree converts an existing clone in place. The workarea must be clean. The conversion deletes every file in the top-level directory except the git directory, so preserve anything there that is not part of the repository. Only the main branch gets a worktree, so a different checked-out branch gets none, though one is easy to add afterward. To convert a clone you work in, with its branches and linked worktrees, use worktree migrate instead.

On a repository that is already bare, init --worktree deletes nothing. It refuses to run while a branch other than the main one exists, because it cannot know which remote each branch should go to.

worktree migrate converts a flat clone, one whose .git is a directory, to the worktree layout. Every branch, ref, stash and config value stays. Each linked worktree moves into the top-level directory. The files of the main worktree, ignored ones too, move into a worktree named for the main branch. Without --apply it prints each step and changes nothing.

The main branch is the first branch the project configures that exists locally. Failing that, it is main, then master, then the only local branch. The main clone must have it checked out. A linked worktree named <top>-<rest>, where <top> is the name of the top-level directory, moves to <rest>. Any other moves to a directory named for its branch, with each / turned into -. So proj-topic beside proj becomes proj/topic, and a worktree on user/feature becomes proj/user-feature.

It refuses when a worktree has changes or untracked files, is locked, prunable or missing, or has a detached HEAD. It refuses during a merge, cherry-pick, revert, rebase or bisect. It refuses a worktree with submodules, a sparse checkout, or assume-unchanged or skip-worktree files. It refuses a linked worktree inside <top> or on another filesystem, and a mount point in <top>. It refuses when two worktrees get the same target directory or worktree name, and when a target directory exists. It refuses when core.worktree is set, or when the main worktree’s config.worktree sets core.bare. It refuses when extensions.relativeWorktrees or worktree.useRelativePaths is true, because the repair writes absolute links. It also refuses when the project remote has no url, when GIT_DIR or a variable like it is set, with git older than 2.36, and on a clone already converted. It reports every reason it finds and makes no change of its own. Some basic checks, such as the git version, stop it before the rest run.

Each worktree’s config moves with it. A worktreepath entry for an old path stays and is reported, so remove it by hand. A project value that holds an old path, such as <project>.builddir set to <old>/build, gets a warning that names its key and not its value. The warning does not stop the migration, so fix the value by hand. A tool that records absolute paths, such as a Python virtualenv, may need to be made again.

worktree migrate also converts a store that is already bare at <top>/.git, with its worktrees inside <top>. It renames the store to the hidden name, writes the .git file and repairs each worktree. A worktree’s commondir file that holds an absolute path is rewritten as ../.., because the repair leaves it naming the old store. No worktree moves and no config changes. The checks above apply to each worktree, except the sparse checkout, assume-unchanged, skip-worktree and filesystem checks. Those matter only when files move or an index is reset. It also refuses a worktree outside <top> or inside the store, and a store with no worktrees. It refuses a store with an index. It looks in <top> and in each directory between <top> and a worktree, and refuses an entry there that holds no worktree, unless its name starts with .. An index or such an entry can mean a checkout that core.bare hides. It refuses a commondir that names another repository. A core.bare in the store’s config.worktree is allowed, since git sparse-checkout puts it there.

For a flat clone or a bare store, the plan lists each local branch with commits its upstream lacks, and how many. A commit the upstream has under another id, as after a rebase or cherry-pick, does not count. A branch with no upstream, or an upstream that is gone, is compared with the main branch. A bare store with no main branch lists only branches with an upstream. The list is for information and does not stop the migration.

After a successful --apply of either kind, migrate runs the command in <project>.postmigrate, if it is set, in <top>. The value is split into words as a shell would split it, but no shell runs it, so pipes, redirections and variables do not work. The command runs in its own session and gets no standard input. It has the number of seconds in <project>.postmigratetimeout to finish, 600 when that is unset. On a timeout or a Ctrl-C, migrate sends SIGTERM to the command’s process group, then SIGKILL after five seconds. A process that starts its own session or process group is not stopped. Only a Ctrl-C or the timeout stops the command. A hangup or a SIGTERM to migrate does not reach it. The result goes in post.status in the manifest, ok or the reason it failed. When the command fails or is interrupted, the migration stays done and migrate exits with a failure status. The dry run shows the step and the config file that sets the command, but does not run it. Migrate refuses a command it cannot split, an empty one, and a timeout that is not a whole number from 1 to 86400.

--apply writes the plan to git-project-migrate.json in the git directory. Before each step runs, its name goes in current_step. When the step finishes, it is added to the steps_done list. The migration stops at the first failure and names the step in failed_step. A hard stop, such as a kill, leaves no failed_step, so read current_step. The error names the failed step, its reason and the manifest. After a stop in rename_store the manifest is at either <top>/.git or <store_new>. When verify finds a worktree that is not clean, it names the first 10 changed or untracked paths and counts the rest.

Manual rollback. Undo the steps that steps_done lists, in the order below. The failed step, or the current step after a hard stop, may be fully done, partly done, or not done. Check it as its comment below says, and undo it when it is done or partly done. Take <top>, <store_new>, <main> and the other values from the manifest. <main> is main.new and each <new> and <old> is a linked entry. The steps verify, repair, reset_index and copy_main_state need nothing:

# write_pointer: done when <top>/.git is a file
rm <top>/.git
# rename_store: done when <store_new> exists
mv <store_new> <top>/.git
git -C <top> worktree repair <new>...
# detach_head: undo it either way
git --git-dir=<top>/.git symbolic-ref HEAD <head_ref>
# set_bare: done when this prints true
git --git-dir=<top>/.git config core.bare
# set_bare, when core_bare is not null
git --git-dir=<top>/.git config core.bare <core_bare>
# set_bare, when core_bare is null
git --git-dir=<top>/.git config --unset core.bare
# each move_entry <entry>: done when <main>/<entry> exists
mv <main>/<entry> <top>/<entry>
# add_main: done when <main> exists
rm -f <main>/.git
rmdir <main>
git -C <top> worktree prune
# each move_linked: done when <new> exists
git -C <top> worktree repair <new>
git -C <top> worktree move <new> <old>
# then, if any move_linked was undone
git -C <top> worktree repair <old>...
# write_config: undo it either way. Set each config_writes key
# back to its prior value, or unset it when prior is null.
# last, remove the manifest
rm -f <top>/.git/git-project-migrate.json
rm -f <top>/.git/git-project-migrate.json.tmp

A bare store at <top>/.git has a manifest with form set to bare_at_root. Each <path> is a linked entry. The verify step needs no undo. The repair command below undoes repair. The rewrite_commondir steps need no undo, since ../.. names the store at either path:

# write_pointer: done when <top>/.git is a file
rm <top>/.git
# rename_store: done when <store_new> exists
mv <store_new> <top>/.git
git --git-dir=<top>/.git worktree repair <path>...
# last, remove the manifest
rm -f <top>/.git/git-project-migrate.json
rm -f <top>/.git/git-project-migrate.json.tmp

See also:

artifact
config
run