jarn.mkrelease 2.0.1

Release sdist eggs

jarn.mkrelease is a no-frills Python egg releaser. It was created to take the cumber out of building and distributing Python eggs.

Motivation

Here at Jarn, we have switched to zc.buildout and pinned egg versions for customer deployments. This means that for every update, we have to make proper egg releases of all modified packages.

Turns out it’s quite a bit of work to put a new egg on a distribution server! After preparing a package for release (update version strings, etc.), we typically have to:

1. Commit modified files.

2. Tag the release in SCM.

3. Package up an egg.

4. Distribute the egg via scp or upload it to an index server.

Now multiply by the number of packages needing a release, and the moment of I gotta script this approaches at warp 9.

Enter jarn.mkrelease.

Installation

jarn.mkrelease requires Python 2.6 for its improved distutils support. Use easy_install jarn.mkrelease to install the mkrelease script. Then put it on your system PATH by e.g. symlinking it to /usr/local/bin.

Usage

mkrelease [options] [scm-url|scm-sandbox]

Options

-C, --skip-checkin

Do not checkin modified files from the sandbox.

-T, --skip-tag

Do not tag the release in SCM.

-S, --skip-upload

Do not upload the release to dist-location.

-D, --dry-run

Dry-run; equivalent to -CTS.

--svn, --hg, --git

Select the SCM type. Only required if the SCM type cannot be guessed from the argument.

-d dist-location, --dist-location=dist-location

An scp destination specification, or an index server configured in ~/.pypirc, or an alias name for either. This option may be specified more than once.

-s, --sign

Sign the release with GnuPG.

-i identity, --identity=identity

The GnuPG identity to sign with.

-p, --push

Push changes upstream (Mercurial and Git).

-e, --develop

Allow version number extensions (i.e. don’t ignore respective setup.cfg options).

-q, --quiet

Suppress output of setuptools commands.

-k, --keep-temp

Keep the temporary build directory.

-h, --help

Print the help message and exit.

-v, --version

Print the version string and exit.

scm-url

The URL of a remote SCM repository.

scm-sandbox

A local SCM sandbox; defaults to the current working directory.

Examples

Release my.package, using version information from the package’s setup.py, and distribute it to the default location:

$ mkrelease https://svn.jarn.com/public/my.package/trunk

The same as above, but release the contents of the SCM sandbox in src/my.package:

$ mkrelease src/my.package

Release my.package and distribute it via scp to jarn.com:/var/dist/public:

$ mkrelease -d jarn.com:/var/dist/public src/my.package

Release my.package and upload it to PyPI:

$ mkrelease -d pypi src/my.package

Configuration

mkrelease reads available index servers from the distutils configuration file ~/.pypirc. How this file must look is documented elsewhere.

mkrelease furthermore reads its own configuration file ~/.mkrelease. Here’s an example:

[defaults]
distbase =
distdefault = public

[aliases]
public =
  jarn.com:/var/dist/public
customerA =
  jarn.com:/var/dist/customerA
world =
  public
  pypi

(Note that pypi refers to the index server pypi as configured in ~/.pypirc.)

Armed with this configuration we can shorten example 3 to:

$ mkrelease -d public src/my.package

And, because public is the default location, we can omit -d entirely:

$ mkrelease src/my.package

Working with scp

The simplest distribution location is a server directory shared through Apache. Releasing eggs means scp-ing them to the server:

$ mkrelease -d jarn.com:/var/dist/public src/my.package

We have a distribution point for every project, so customer A does not see customer B’s releases:

$ mkrelease -d jarn.com:/var/dist/customerB src/my.package

Typing the full destination every time is tedious, even setting up an alias for each and every customer is, so we configure distbase instead:

[defaults]
distbase = jarn.com:/var/dist
distdefault = public

[aliases]
world =
  public
  pypi

The distbase is prepended if an scp destination does not contain a host part. We can now write:

$ mkrelease -d public src/my.package
$ mkrelease -d customerB src/my.package

Working with index servers

In the Plone world, it is common practice to upload packages to plone.org and PyPI. For this to work, we first need a ~/.pypirc file:

[distutils]
index-servers =
  pypi
  ploneorg

[pypi]
username = fred
password = secret

[ploneorg]
repository = http://plone.org/products
username = fred
password = secret

We can now type:

$ mkrelease -d pypi -d ploneorg src/my.package

Next, we define an alias in ~/.mkrelease:

[aliases]
plone =
  pypi
  ploneorg

Which allows us to write:

$ mkrelease -d plone src/my.package

Releasing a tag

Release my.package from an existing tag:

$ mkrelease -T https://svn.jarn.com/public/my.package/tags/1.0

Only Subversion allows us to specify a branch or tag to check out. With Mercurial and Git we need a local working copy, switched to the tag we want to release:

$ git checkout -f 1.0
$ mkrelease -T

Using GnuPG

Release my.package and sign the archive with GnuPG (the gpg command must be available on the system PATH):

$ mkrelease -s -i [e-mail] -d pypi src/my.package

For convenience, and to support mandatory-signing scenarios, defaults for -s and -i may be configured in ~/.mkrelease:

[defaults]
sign = yes
identity = [e-mail]

Requirements

The following commands must be available on the system PATH:

• svn

• hg

• git

• scp

Limitations

Subversion

The release tag can only be created if the package follows the standard Subversion repository layout: package.name/trunk, package.name/branches, and package.name/tags. If you have a non-standard repository, you must tag by hand and run mkrelease with the -T option.

Git

Giving the -p option results in git push origin and git push origin tag <tagid> respectively. If this does not fit your setup, avoid -p and push manually.

Git’s short-hand notation for ssh:// URLs is not supported.

Changelog

2.0.1 - 2009-07-24

• Fixed bug which could cause mkrelease to issue eggs with faulty manifest files (Symptom: data files not installed). [stefan]

• The -e flag now implies -T. We never want to tag a development release. [stefan]

2.0 - 2009-07-16

• Allow command line options to appear after the argument. As in: mkrelease src/my.package -q -d pypi. [stefan]

2.0b2 - 2009-07-09

• Improve user feedback in the SCM-detection part. [stefan]

• Document the -e flag. [stefan]

• Drop global configuration file for YAGNI. [stefan]

• Allow to set default values for -s and -i in ~/.mkrelease. [stefan]

2.0b1 - 2009-07-03

• By default, ignore all version number extensions (dev-r12345) that may be configured in setup.cfg. Passing the -e flag disables this safeguard. [witsch, stefan]

• Delete any existing signature file before signing anew. This keeps GnuPG from complaining about existing (but left-over) files. [stefan]

2.0a2 - 2009-06-27

• Drop configurable Python and use sys.executable. This also means we now require Python 2.6. [stefan]

• Force setuptools to only use file-finders for the selected SCM type. This is required to support multi-SCM sandboxes (think git-svn). [stefan]

• Treat Subversion sandboxes just like the others and avoid the temporary checkout step. [stefan]

• Remove the -u flag for being pointless. [stefan]

2.0a1 - 2009-06-14

• Added support for Mercurial and Git. [stefan]

• Added 250+ unit tests. [stefan]

1.0.2 - 2009-06-13

• Documented long options. [stefan]

• Print a “Tagging …” line before tagging. [stefan]

1.0 - 2009-05-14

• Print help and version to stdout, not stderr. [stefan]

1.0b4 - 2009-04-30

• Since distutils commands may return 0, successful or not, we must check their output for signs of failure. [stefan]

• Allow to pass argument list to main(). [stefan]

1.0b3 - 2009-03-23

• No longer depend on grep. [stefan]

• Use subprocess.Popen instead of os.system and os.popen. [stefan]

• Protect against infinite alias recursion. [stefan]

• Drop -z option and always create zip files from now on. [stefan]

1.0b2 - 2009-03-19

• Checkin everything that’s been modified, not just “relevant” files. [stefan]

• Expand aliases recursively. [stefan]

1.0b1 - 2009-03-18

• The distbase and distdefault config file options no longer have default values. [stefan]

• Read index servers from ~/.pypirc and allow them to be used with -d. [stefan]

• The -d option may be specified more than once. [stefan]

• Dropped -p option. Use -d pypi instead. [stefan]

• Dropped -c option. If your have non-standard SVN repositories you must tag by hand. [stefan]

0.19 - 2009-02-23

• Absolute-ize the temp directory path. [stefan]

0.18 - 2009-01-26

• Include README.txt and CHANGES.txt in long_description. [stefan]

• Rid unused imports and locals. [stefan]

0.17 - 2009-01-23

• Add -c option to enable codespeak support. The codespeak.net repository uses branch and tag instead of branches and tags. [gotcha, stefan]

0.16 - 2009-01-13

• Fold regex construction into find and make find a method. [stefan]

• Update README.txt. [stefan]

0.15 - 2009-01-13

• Support for reading default options from a config file. [fschulze, stefan]

0.14 - 2009-01-08

• Add -s and -i options for signing PyPI uploads with GnuPG. [stefan]

• Stop execution after any failing step. [stefan]

0.13 - 2009-01-05

• Stop execution when the checkin step fails. [stefan]

0.12 - 2009-01-02

• setup.cfg may not exist. [stefan]

0.11 - 2008-12-02

• Add setup.cfg to list of files we check in. [stefan]

0.10 - 2008-10-21

• Don’t capitalize GetOptError messages. [stefan]

0.9 - 2008-10-16

• Add -v option to print the script version. [stefan]

0.8 - 2008-10-16

• Lift restriction where only svn trunk could be released. [stefan]

0.7 - 2008-10-09

• Fix PyPI upload which must happen on the same command line as sdist. [stefan]

0.6 - 2008-10-08

• Update README.txt. [stefan]

0.5 - 2008-10-08

• Also locate and checkin HISTORY.txt to support ZopeSkel’ed eggs. [stefan]

0.4 - 2008-10-08

• Use svn checkout instead of svn export because it makes a difference to setuptools. [stefan]

• Add -p option for uploading to PyPI instead of dist-location. [stefan]

0.3 - 2008-10-06

• Also locate and checkin version.txt. [stefan]

0.2 - 2008-10-01

• Add -z option to create zip archives instead of the default tar.gz. [stefan]

0.1 - 2008-10-01

• Initial release [stefan]