checkoutmanager 3.0.2

Gives you overview and control over your git/hg/bzr/svn checkouts/clones.

Checkoutmanager

Makes bzr/hg/git/svn checkouts in several places according to a .checkoutmanager.cfg config file (in your home directory).

The advantage: you’ve got one command with which you can update all your checkouts. And with which you can ask for a list of uncommitted changes. And you can rebuild your entire checkout structure on a new machine just by copying the config file (this was actually the purpose I build it for: I had to change laptops when I switched jobs…).

Checkoutmanager works on linux, osx and windows.

Starting to use it

Starting is easy. Just pip install checkoutmanager and run checkoutmanager.

• The first time, you’ll get a sample configuration you can copy to .checkoutmanager.cfg in your home directory.

• The second time, you’ll get a usage message. (You’ll want to do checkoutmanager co to grab your initial checkouts).

Generic usage

What I normally do every morning when I get to work is checkoutmanager up. This grabs the latest versions of all my checkouts from the server(s). So an svn up for my subversion checkouts, a hg pull -u for mercurial and so on.

From time to time, I’ll do a checkoutmanager st to show if I’ve got some uncommitted files lying around somewhere. Very handy if you’ve worked in several directories throughout the day: it prevents you from forgetting to check in that one bugfix for a whole week.

A new project means I add a single line to my config file and run checkoutmanager co.

Checkoutmanager allows you to spread your checkouts over multiple directories. It cannot mix version control systems per directory, however. As an example, I’ve got a ~/buildout/ directory with my big svn website projects checked out there. And a directory with my svn work python libraries. And a ~/hg/ dir with my mercurial projects. And I’ve made checkouts of several config directories in my home dir, such as ~/.emacs.d, ~/.subversion and so on. Works just fine.

Commands

Available commands:

exists

Print whether checkouts are present or missing

up

Grab latest version from the server.

st

Print status of files in the checkouts

co

Grab missing checkouts from the server

missing

Print directories that are missing from the config file

out

Show changesets you haven’t pushed to the server yet

in

Show incoming changesets that would be pulled in with ‘up’. For some version control systems, this depends on the English output of the respective commands and is therefore inherently fragile.

Hidden commands

A few commands are hidden because they are seldom used and are only useful for subversion.

upgrade

This upgrades the working copy to the new subversion 1.7 layout of the .svn directory. This should be done once after you have upgraded your subversion to 1.7. Note that when you accidentally run this twice you get an error, but nothing breaks. Since this command is so rarely needed, it is not advertised in the command line help.

info

Display the svn info for the remote url. This is useful when your svn program has been updated and the security mechanisms on your OS now require you to explictly allow access to the stored credentials. The other commands either do not access the internet or are non-interactive (like command up). In fact, the reason for adding this command is that a non-interactive ‘svn update’ will fail when you have not granted access to your credentials yet for this new svn program. This has happened a bit too often for me (Maurits).

Output directory naming

If you don’t specify an output directory name for your checkout url, it just takes the last part. To make life easier, we do have some adjustments we make:

• https://xxx/yyy/product/trunk becomes product instead of trunk. (Handy for subversion).

• https://xxx/yyy/product/branches/experiment becomes product_experiment instead of experiment (Handy for subversion).

• https://xxx/customername/buildout/trunk becomes customername instead of “trunk” or “buildout”. (Old convention we still support).

• Bzr checkouts that start with “lp:” (special launchpad urls) get their “lp:” stripped.

• Git checkouts lose the “.git” at the end of the url.

• If you want to preserve the directory configuration of your version control system, add the preserve_tree option to a group. It should contain one or more base checkout urls (one per line). If the checkout url starts with one of the preserve_tree urls, the folder structure after it is preserved.

  With a preserve_tree of https://github.com, https://github.com/reinout/checkoutmanager becomes reinout/checkoutmanager instead of checkoutmanager. Also handy for subversion, which often has nested directories.

  If the preserve_tree base url isn’t found, the standard rules are used, so you won’t get an error.

If you want different behaviour from the defaults above, just specify a directory name (separated by a space) in the configuration file after the url. So https://github.com/reinout/checkoutmanager bla_bla becomes bla_bla instead of checkoutmanager

Custom commands

You can write your own custom commands. To do that you need to create a Python package and register an entry point in your setup.py for the checkoutmanager.custom_actions target.

A test command is included with checkoutmanager and can serve as an example. It is registered like this in checkoutmanager’s own setup.py:

entry_points={
    'checkoutmanager.custom_actions': [
        'test = checkoutmanager.tests.custom_actions:test_action'
     ]
}

The entry point function must take one positional argument which will be the checkoutmanager.dirinfo.DirInfo instance associated with the directroy for which the command is being executed. The function can also take optional keyword arguments. See checkoutmanager.tests.custom_actions.test_action for an example.

Arguments are passed to the custom command using the following syntax:

checkoutmanager action:arg1=val1,arg2=val2

Config file

Sample configuration file:

# Sample config file.  Should be placed as .checkoutmanager.cfg in your home
# directory.
#
# There are different sections per base location and version control
# system.
#
# ``checkoutmanager co`` checks them all out (or clones them).
# ``checkoutmanager up`` updates them all.
# ``checkoutmanager st`` to see if there are uncommitted changes.
# ``checkoutmanager out`` to see if there are unpushed git/hg commits.


[git-example]
vcs = git
basedir = ~/example/git
checkouts =
    https://github.com/reinout/checkoutmanager
    [e-mail]:django/django.git


[recipes]
# Buildout recipes I work on.
vcs = svn
basedir = ~/example/svn
checkouts =
    http://svn.zope.org/repos/main/z3c.recipe.usercrontab/trunk


[hg-example]
vcs = hg
basedir = ~/example/utilities
checkouts =
    https://bitbucket.org/reinout/eolfixer
    https://bitbucket.org/reinout/createcoverage


# [dotfolders]
# # Advanced usage!
# # Folders that end up as dotted configfolders in my home dir.
# # Note that there's a directory name behind the repository
# # location, separated by a space.
# vcs = bzr
# basedir = ~
# checkouts =
#     lp:emacsconfig/trunk .emacs.d
#     sftp://somewhere/subversion/trunk .subversion
# # By ignoring everything, we do not find missing import files but also
# # don't get warnings for every subdirectory in our home dir
# ignore =
#     *
#     .*

TODO

• Wait for feedback so that I can improve the documentation.

• Perhaps make a better sample config (one that actually works instead of the current one that’s structured for benefit of the automated tests).

• Modernize test setup (pytest instead of zope’s testrunner + doctests).

Credits

• Created by Reinout van Rees.

• “out” command by Dmitrii Miliaev.

• Git support by Robert Kern.

• Globbing support for the ignores by Patrick Gerken.

• Custom commands support by Rafael Oliveira.

• Parallelism code by Morten Lied Johansen.

Source code is on github at https://github.com/reinout/checkoutmanager .

Bugs and feature requests can be reported at https://github.com/reinout/checkoutmanager/issues .

Changelog of checkoutmanager

3.0.2 (2024-01-11)

• Fixed super() call on the MultiExecutor that I messed up when modernizing the python 2.7 code :-)

3.0.1 (2024-01-11)

• Cleaned up left-over TODO comments in readme.

3.0 (2024-01-11)

• Added github action for basic testing.

• Removed ye olde buildout setup, including z3c.testsetup. Using pytest now.

• Removed python 2.7 support, we’re on 3.8+ now.

• Achieved compatibility with python 3.12: Safeconfigparser->ConfigParser.

2.7 (2021-09-28)

• More robust error handling. [mortenlj]

2.6.1 (2019-09-23)

• Fixed small, but essential, README error.

2.6 (2019-09-10)

• Updated the setup (mostly: buildout version pins) so that the project can be developed/tested again.

• The exists and co command used to check only if a directory existed. Now it also checks if the dot directory (.git, .svn) exists. This way an empty directory also will get filled with a checkout.

2.5 (2016-11-07)

• Fix #19: sometimes git remote changes were seen where there were none. [reinout]

2.4.1 (2015-09-10)

• Bugfix for the 2.4-introduced run_one() function. [chintal]

2.4 (2015-09-09)

• Added in command that reports incoming changes (so: the changes you’d get by running checkoutmanager up). Due to differences between versions of git/svn/hg/bzr, the reporting might not be entirely accurate. It is very hard to get right. So: please report an issue if something is not quite right. [chintal]

• Added better support for using checkoutmanager as a library. Provided you first load a config file, you can now programmatically run actions on individual directories or urls. See the source code for the checkoutmanager.runner.run_one() function. [chintal]

2.3 (2015-09-08)

• Added a preserve_tree option to config files to allow structured checkouts mirroring the repository tree. [chintal]

2.2 (2015-08-24)

• Checkoutmanager now also runs on python 3! [reinout]

• Moved from bitbucket (https://bitbucket.org/reinout/checkoutmanager) to github (https://github.com/reinout/checkoutmanager). [reinout]

2.1 (2015-08-18)

• Fixed missing command: do not swallow the output when looking for not yet checked out items. Fixes issue #24. [maurits]

2.0 (2015-03-25)

• Huge speed increase because commands are now run in parallel instead of sequentially. Great fix by Morten Lied Johansen. For me, “checkoutmanager up” now takes 19 seconds instead of 105 seconds!

1.17 (2015-02-06)

• Added support for custom commands: now you can write an extension for checkoutmanager so that you can run checkoutmanager your_custom_command. See the README for documentation. Patch by Rafael Oliveira.

1.16 (2015-01-02)

• Added globbing support for ignores.

1.15 (2013-09-27)

• Handle corner case in determining directory name for a git clone.

1.14 (2013-08-12)

• Added --force-interactive to svn info for svn version 1.8 and higher. This is for the “hidden” instancemanager info command that is handy for updating your repositories when you’ve switched svn versions. (See the changelog entry for 1.10). Patch by Maurits.

1.13 (2012-07-20)

• Not using the sample config file as the test config file anymore. This means there’s a much nicer and more useful sample config file now.

  (Thanks Craig Blaszczyk for his pull request that was the basis for this!)

1.12 (2012-04-14)

• For bzr, the “out” command uses the exit code instead of the command output now. This is more reliable and comfortable. Fix by Jendrik Seipp, thanks!

1.11 (2012-03-20)

• Allow more than one vcs in a directory. This was already possible before, but now known you no longer need to list all the checkouts of the competing vcs in the ignore option. Also, items that are ignored in one section are now also ignored in other sections for the same directory. Fixes #11. [maurits]

1.10 (2012-01-16)

• Using –mine-only option to bzr missing to only show our outgoing changesets when running checkoutmanager’s “out” command for bzr.

• Copying sample .cfg file if it doesn’t exist instead of only suggesting the copy. Fixes #12.

• Added hidden info command. Should be only useful for subversion if your svn program is updated and your OS requires you to give svn access to your stored credentials again, for each repository. [maurits]

1.9 (2011-11-08)

• Added upgrade command that upgrades your subversion checkouts to the new 1.7 layout of the .svn directory. [maurits]

1.8 (2011-10-13)

• Using git push --dry-run now to detect not-yet-pushed outgoing changes with checkoutmanager out. Fixes #9 (reported by Maurits van Rees).

1.7 (2011-10-06)

• Added –configfile option. Useful when you want to use checkoutmanager to manage checkouts for something else than your regular development projects. In practice: I want to use it for an ‘sdistmaker’ that works with git.

1.6 (2010-12-27)

• Full fix for #7: checkoutmanager doesn’t stop on the first error, but continues. And it reports all errors afterwards. This helps when just one of your svn/hg/whatever servers is down: the rest will just keep working.

• Partial fix for #7: svn up runs with --non-interactive now, so conflict errors errors are reported instead of pretty much silently waiting for interactive input that will never come.

1.5 (2010-09-14)

• Using except CommandError, e instead of except CommandError as e for python2.4 compatibility.

1.4 (2010-08-17)

• Added git support (patch by Robert Kern: thanks!) Fixes issue #6.

1.3 (2010-08-09)

• Added new “out” action that shows changesets not found in the default push location of a repository for a dvcs (hg, bzr). The action doesn’t make sense for svn, so it is ignored for svn checkouts. Fixes issue #1. Thanks Dmitrii Miliaev for this fix!

1.2.1 (2010-08-06)

• Bugfix: when reporting an error, the os.getcwd method itself would get printed instead of the output of os.getcwd()…

1.2 (2010-08-04)

• If the config file doesn’t exist, just print the config file hints instead of the generic usage info.

• Fixed issue #4: the generic ‘buildout’ name is stripped from the path. svn://somewhere/customername/buildout/trunk is a common pattern.

• Added -v option that prints the commands and the directory where you execute them. Fixes issue #3.

• Reporting on not yet checked out items when running “checkoutmanager missing”. Fixes issue #2.

• Checking return code from executed commands. On error, the command and working directory is printed and also the output. And the script stops right away. Fixes #5.

• Updated the documentation, for instance by mentioning the config file name and location.

1.1 (2010-08-02)

• Switched from “commands” module to “subprocesses” for windows compatibility.

1.0 (2010-08-01)

• Small fixes. It works great in practice.

• Moved from bzr to hg and made it public on bitbucket.org.

• Big documentation update as I’m going to release it.

0.1 (2010-05-07)

• First reasonably working version.

• Initial library skeleton created by thaskel.