git-externals 0.1.0

Provide an SVN external mechanism for git

git-externals

You are tired of git-submodules? For some us cases, we all were tempted to scream at git submodules, since they do not work as svn:externals work. git-external supports git, svn, and git-svn remotes as externals.

Installation

git-externals can be installed via pip or an equivalent via:

    pip install git-externals

Usage

Add an externals repo to current repo

    git externals add URL PATH

Update all externals or specific externals

    git externals update <PATH>

Project Lore

This is an alternative to Christian Dietrich's alternative to the alternative to git submodule written by Daniel Cestary.

Christian really like Daniel's concept, but could not force his users to install a Ruby gem. Therefore he crammed (his words) his reimplementation of git-external into a single python file, intended to be commited into your project repository as a ./init script.

I happened upon this project while searching for a solution to incrementally migrate a large number of svn repos that abused svn:externals as a makeshift dependency management system to git without breaking any builds. However, I have strong opinions on some design choices:

• Copying this script into every repo and having it self-update is messy. It is better to package & distribute it as a python wheel. Naming the project.script to conform with git's automatic extension detection and adding it to the shell's PATH should be a more robust approach.
• Might as well modernize the build tooling to match the python dev's recommendations.

Why?

git-external is a tool for including external repositories inside of a super repository or main repository. Is very similar to git submodule since it helps keeping some dependencies or sub-projects in different repositories and reuse them in many projects.

git external intends to mimic the behavior of svn:externals, providing an updatable reference to an external repository which the main repository doesn't need to know about.

Why not git submodule?

With git submodule, git supports to import other repositories as modules, , all of them are a git repository on their own.

The problem is each module has a commit id associated with it and every time the module is updated (by issuing a git pull inside the module) this causes a change on the supermodule therefor forcing you to make a new commit only for updating what is sometimes a dependency.

git submodule is a great tool, but it's not for everyone, neither is git-external.

How?

git-externalstores a file in the root of the repository called .gitexternals with a format very similar to the .gitmodules from git submodule (if not almost identical). This file keeps the information of the externals (meaning path and url). If you are interested in the format, look at the .gitexternals in this repository.

Each external is really a clone of the repository specified to git external add so you can do everything you would do on a normal git repository inside an external.

The path where the external's clone resides is added to the .gitignore file in order to keep it out of your way while you get your work done.

With the only attribute you prevent git-external to clone the repository by default. For these externals, the explicit clone command has to be used.

[external "repo"]
	path = local-path
	url = [e-mail]:remote/repo.git
	branch = master
	only = update
	vcs = git

Branches

Within the branch = configuration, you cannot only give branch names, but also tag names. We use git checkout to guess what you want to checkout.

On update, branches are not switched, even if the external definition has changed. If you want to be sure to switch to the configured branch, use ./git-external clone.

Further Attributes:

• auto = false: Exclude externals from automatic cloning

    [external "exams"]
         auto = false
  

• vcs = none: Only create a symlink without any VCS (e.g to symlink Nexcloud folders)

    [external]
        ibrvsscloud = "/home/...."
    
    [external "foo"]
            url = "${ibrvsscloud}/foo"	
            path = "foo"
            vcs = none
  

• script = none: Execute a script located at repository root after cloning the external. Note: not supported on Windows (including git-bash).

    [external "foo"]
            script = run.sh
  

• cloneArgs: Additional arguments/options when clone a repository

    [external "foo"]
           ...
           cloneArgs = "--sparse --depth=1"
  

• updateArgs: Additional arguments/options when clone a repository (e.g. --sparse)

    [external "foo"]
           ...
           updateArgs = --sparse
  

• sparseCheckout: Enable and configure sparse checkout. If this option is not present or is empty, sparse checkout is disabled. Note, --sparse option to cloneArgs or updateArgs is not required for this to work.

    [external "foo"]
           ...
           sparseCheckout = "bar baz"
  

Overrides

You can overide the settings for externals by putting an external section into your global git config. For example, if you want all possible forms of Github urls that are used in externals you can match up the external defnitions with this override in our ~/.gitconfig:

[external "bib-override"]
	match-url = "*github.com*luhsra/bib*"
	url = "[e-mail]:luhsra/bib"

The match-* attributes are regular shell globs and matche against the corresponding attribute. All other keys override settings in the original external definition. A prominent example of such an override is to use always git-svn instead of svn:

[external "override-svn"]
     match-vcs = svn
     vcs = git-svn

Another problem that comes up with git-externals is often that you have multiple copies of the same repository, over and over again. However, git-external provides the possibility to symlink an already existing repository instead of cloning a new instance. This can also be employed by overrides:

[external "bib-override"]
   match-url = "*github.com*luhsra/bib*"
   symlink = ~/proj/SRA/bib

Instead of cloning always a new instance of the bib repository there exists only one instance of it that is always symlinked.