Skip to main content

Clone/update all user/organization GitHub repositories

Project description


It’s a script to clone/update all repos for a user/organization from GitHub.

Target audience: maintainers of large collections of projects (for example, ZopeFoundation members).



Clone all ZopeFoundation repositories:

mkdir ~/src/zf
cd ~/src/zf
./ --init --org ZopeFoundation

Clone all mgedmin’s vim plugins:

mkdir ~/src/vim-plugins
cd ~/src/vim-plugins
./ --init --user mgedmin --pattern '*.vim'


What it does:

  • clones repositories you don’t have locally

  • pulls changes for repositories you already have locally

  • warns you about local changes and other unexpected situations:

    • unknown files in the tree (in –verbose mode only)

    • staged but not committed changes

    • uncommitted (and unstaged changes)

    • non-master branch checked out

    • committed changes that haven’t been pushed to master

    • remote URL pointing to an unexpected location (in –verbose mode only)

You can speed up the checks for local unpublished changes by running ./ -n: this will skip the git pull/git clone.


Other command-line options:

$ ./ --help
usage: [-h] [--version] [-c CONCURRENCY] [-n] [-v]
                   [--start-from REPO] [--organization ORGANIZATION]
                   [--user USER] [--pattern PATTERN] [--init]
                   [--http-cache DBNAME] [--no-http-cache]

Clone/update all user/org repositories from GitHub.

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
                        set concurrency level
  -n, --dry-run         don't pull/clone, just print what would be done
  -v, --verbose         perform additional checks
  --start-from REPO     skip all repositories that come before REPO
  --organization ORGANIZATION
                        specify the GitHub organization
  --user USER           specify the GitHub user
  --pattern PATTERN     specify repository name pattern to filter
  --init                create a .cloneallrc from command-line arguments
  --http-cache DBNAME   cache HTTP requests on disk in an sqlite database
                        (default: .httpcache)
  --no-http-cache       disable HTTP disk caching

Configuration file

The script looks for .cloneallrc in the current working directory, which should look like this:

# Provide either github_user or github_org, but not both
# github_org = ZopeFoundation
github_user = mgedmin
pattern = *.vim

You can create one with ./ --init --{user,org} X [--pattern Y].


For best results configure SSH persistence to speed up git pulls – in your ~/.ssh/config:

ControlMaster auto
ControlPersist yes
ControlPath ~/.ssh/control-%r@%h-%p

It takes about 1 minute to run git pull on all 339 ZopeFoundation repos on my laptop with this kind of setup.


1.5 (2016-12-29)

  • Released to PyPI as ghcloneall

  • Added Python 2.7 support

1.4 (2016-12-28)

  • Command line args: –user, –pattern, –init

  • Load (some) options from a .cloneallrc

  • Stop using --organization=ZopeFoundation by default, require an explicit option (or config file)

  • Rename to

1.3 (2016-12-28)

  • Command line args: -c

  • Show progress while fetching the list of repositories from GitHub

  • Update repositories concurrently by default

  • Highlight items in progress

  • Highlight failed items in red

  • Tweak progress bar style from [=== ] to [###..]

  • Clear the progress bar on ^C

  • Handle git errors nicely

  • Bugfix: -vv could fail with NameError if unknown files were present in a working tree

  • Bugfix: correctly show progress when using –start-from

  • Bugfix: script would hang (for 10 minutes) if you didn’t already have an SSH control master process running

  • Bugfix: –dry-run didn’t show which repos were new

1.2 (2016-11-09)

  • Command line args: –dry-run, –verbose

  • Cache HTTP responses on disk for 10 minutes to avoid GitHub API rate limits

  • Report about forgotten uncommitted and staged changes

  • Warn about local (unpushed) commits too

  • Warn about other branches being checked out

  • Default to SSH URLs again (faster when using SSH’s ControlPersist)

1.1 (2015-11-07)

  • Command line args: –version, –start-from, –organization

  • Output formatting: shorter repository names, totals at the end

  • Use ANSI colors to indicate changes

  • Don’t print tracebacks on ^C

  • Default to HTTPS URLs

1.0 (2015-11-07)

  • Moved from a gist to a proper GitHub repository.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ghcloneall-1.5.tar.gz (12.0 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page