Skip to main content

Software releasing made easy and repeatable

Project description

Package releasing made easy

zest.releaser is collection of command-line programs to help you automate the task of releasing a software project. It’s particularly helpful with Python package projects, but it can also be used for non-Python projects. For example, it’s used to tag buildouts - a project only needs a version.txt file to be used with zest.releaser.

It will help you to automate:

  • Updating the version number. The version number can either be in setup.py or version.txt. For example, 0.3.dev0 (current) to 0.3 (release) to 0.4.dev0 (new development version).

  • Updating the history/changes file. It logs the release date on release and adds a new section for the upcoming changes (new development version).

  • Tagging the release. It creates a tag in your version control system named after the released version number.

  • Uploading a source release to PyPI. It will only do this if the package is already registered there (else it will ask, defaulting to ‘no’); the Zest Releaser is careful not to publish your private projects! It can also check out the tag in a temporary directory in case you need to modify it.

Installation

Getting a good installation consists of two steps: getting the zest.releaser commands, and setting up your environment so you can upload releases to pypi (if you want that).

Get the zest.releaser commands

Just a simple pip zest.releaser or easy_install zest.releaser is enough.

Alternatively, buildout users can install zest.releaser as part of a specific project’s buildout, by having a buildout configuration such as:

[buildout]
parts = releaser

[releaser]
recipe = zc.recipe.egg
eggs = zest.releaser

Prepare for pypi distribution

Of course you must have a version control system installed. zest.releaser currently supports:

  • Subversion (svn)

  • Mercurial (hg)

  • Git (git)

  • Bazaar (bzr)

Others could be added if there are volunteers.

When the (full)release command tries to upload your package to a pypi server, zest.releaser basically just executes the command python setup.py register sdist --formats=zip upload. The python here is the same python that was used to install zest.releaser. If that command would fail when you try it manually (for example because you have not configured a .pypirc file yet), then zest.releaser does not magically make it work. This means that you may need to have some extra python packages installed:

  • setuptools or distribute (when using subversion 1.5 or higher you need setuptools 0.6c11 or higher or any distribute version)

  • setuptools-git (Setuptools plugin for finding files under Git version control)

  • setuptools_hg (Setuptools plugin for finding files under Mercurial version control)

  • setuptools_bzr (Setuptools plugin for finding files under Bazaar version control)

  • collective.dist (when using python2.4, depending on your ~/.pypirc file)

  • setuptools_subversion (Setuptools plugin for finding files under Subversion version control.) You probably need this when you upgrade to the recent subversion 1.7. If you suddenly start missing files in the sdists you upload to PyPI you definitely need it. Alternatively: set up a proper MANIFEST.in as that method works with any version control system.

The setuptools plugins are mostly so you do not miss files in the generated sdist that is uploaded to pypi.

For more info, see the section on Uploading to pypi server(s).

In general, if you are missing files in the uploaded package, the best is to put a proper MANIFEST.in file next to your setup.py. See zest.pocompile for an example.

Running

Zest.releaser gives you four commands to help in releasing python packages. They must be run in a version controlled checkout. The commands are:

  • prerelease: asks you for a version number (defaults to the current version minus a ‘dev’ or so), updates the setup.py or version.txt and the HISTORY.txt/CHANGES.txt/CHANGES with this new version number and offers to commit those changes to subversion (or bzr or hg or git)

  • release: copies the the trunk or branch of the current checkout and creates a version control tag of it. Makes a checkout of the tag in a temporary directory. Offers to register and upload a source dist of this package to PyPI (Python Package Index). Note: if the package has not been registered yet, it will not do that for you. You must register the package manually (python setup.py register) so this remains a conscious decision. The main reason is that you want to avoid having to say: “Oops, I uploaded our client code to the internet; and this is the initial version with the plaintext root passwords.”

  • postrelease: asks you for a version number (gives a sane default), adds a development marker to it, updates the setup.py or version.txt and the HISTORY.txt/CHANGES.txt with this and offers to commit those changes to version control. Note that with git and hg, you’d also be asked to push your changes (since 3.27). Otherwise the release and tag only live in your local hg/git repository and not on the server.

  • fullrelease: all of the above in order.

There are two additional tools:

  • longtest: small tool that renders a setup.py’s long description and opens it in a web browser. This assumes an installed docutils (as it needs rst2html.py).

  • lasttagdiff: small tool that shows the diff of the currently committed trunk with the last released tag. Handy for checking whether all the changes are adequately described in the HISTORY.txt/CHANGES.txt.

Details

Current assumptions

Zest.releaser originated at Zest software so there are some assumptions build-in that might or might not fit you. Lots of people are using it in various companies and open source projects, so it’ll probably fit :-)

  • If you are using svn, your svn is structured with /trunk, /tags (or /tag) and optionally /branches (or /branch). Both a /trunk or a /branches/something checkout is ok.

  • There’s a version.txt or setup.py in your project. The version.txt has a single line with the version number (newline optional). The setup.py should have a single version = '0.3' line somewhere. You can also have it in the actual setup() call, on its own line still, as `` version = ‘0.3’,``. Indentation and the comma are preserved. If you need something special, you can always do a version=version and put the actual version statement in a zest.releaser-friendly format near the top of the file. Reading (in Plone products) a version.txt into setup.py works great, too.

  • The history file (either HISTORY.txt, CHANGES.txt or CHANGES) restriction is probably the most severe at the moment. zest.releaser searches for a restructuredtext header with parenthesis. So something like:

    Changelog for xyz
    =================
    
    0.3 (unreleased)
    ----------------
    
    - Did something
    
    0.2 (1972-12-25)
    ----------------
    
    - Reinout was born.

    That’s just the style we started with. Pretty clear and useful. It also supports the current zopeskel style with 0.3 - unreleased.

  • If using Python 2.4 you don’t want to have tar.gz eggs due to an obscure bug on python

Development notes, bug tracker

The source code can be found on github: https://github.com/zestsoftware/zest.releaser

If you are going to do a fix or want to run the tests, please see the DEVELOPERS.txt file in the root of the package.

Bugs can be added to https://bugs.launchpad.net/zest.releaser .

Note that there are alternative release scripts available, for instance http://pypi.python.org/pypi/collective.releaser which installs itself as a setuptools command (“python setup.py release”), so it “only” works with setuptools projects.

Uploading to pypi server(s)

Like noted earlier, for safety reasons zest.releaser will only offer to upload your package to http://pypi.python.org when the package is already registered there. If this is not the case yet, you can go to the directory where zest.releaser put the checkout (or make a fresh checkout yourself. Then with the python version of your choice do:

python setup.py register sdist --formats=zip upload

For this to work you will need a .pypirc file in your home directory that has your pypi login credentials like this:

[server-login]
username:maurits
password:secret

Since python 2.6, or in earlier python versions with collective.dist, you can specify multiple indexes for uploading your package in .pypirc:

[distutils]
index-servers =
  pypi
  local

[pypi]
#pypi.python.org
username:maurits
password:secret

[local]
repository:http://localhost:8080/test/products/
username:maurits
password:secret
# You may need to specify the realm, which is the domain the
# server sends back when you do a challenge:
#realm:Zope

See http://pypi.python.org/pypi/collective.dist for more info.

When all this is configured correctly, zest.releaser will first reregister and upload at the official pypi (if the package is registered there already). Then it will offer to upload to the other index servers that you have specified in .pypirc.

Note that since version 3.15, zest.releaser also looks for this information in the setup.cfg if your package has that file. One way to use this, is to restrict the servers that zest.releaser will ask you upload to. If you have defined 40 index-servers in your pypirc but you have the following in your setup.cfg, you will not be asked to upload to any server:

[distutils]
index-servers =

Note that after creating the tag we still ask you if you want to checkout that tag for tweaks or pypi/distutils server upload. We could add some extra checks to see if that is really needed, but someone who does not have index-servers listed, may still want to use an entry point like gocept.zestreleaser.customupload to do uploading, or do some manual steps first before uploading.

Some people will hardly ever want to do a release on PyPI but in 99 out of 100 cases only want to create a tag. They won’t like the default answer of ‘yes’ to that question of whether to create a checkout of the tag. So since version 3.16 you can influence this default answer. You can add some lines to the .pypirc file in your home directory to change the default answer for all packages, or change it for individual packages in their setup.cfg file. The lines are this:

[zest.releaser]
release = no

You can use no/false/off/0 or yes/true/on/1 as answers; upper, lower or mixed case are all fine.

Entrypoints documentation

Warning: entry points were added in 3.0, I’m reserving the right to make backwards-incompatible changes to the entry point mechanism in the next couple of releases. It is a major new piece of functionality for zest.releaser and getting all the details right at the first attempt isn’t guaranteed.

A zest.releaser entrypoint gets passed a data dictionary and that’s about it. You can do tasks like generating documentation. Or downloading external files you don’t want to store in your repository but that you do want to have included in your egg.

Every release step (prerelease, release and postrelease) has three points where you can hook in an entry point:

before

Only the workingdir and name are available in the data dictionary, nothing has happened yet.

middle

All data dictionary items are available and some questions (like new version number) have been asked. No filesystem changes have been made yet.

after

The action has happened, everything has been written to disk or uploaded to pypi or whatever.

For the release step it made sense to create one extra entry point:

after_checkout

The middle entry point has been handled, the tag has been made, a checkout of that tag has been made and we are now in that checkout directory. Of course, when the user chooses not to do a checkout, this entry point never triggers.

Note that an entry point can be specific for one package (usually the package that you are now releasing) or generic for all packages. An example of a generic one is gocept.zestreleaser.customupload, which offers to upload the generated distribution to a chosen destination (like a server for internal company use). If your entry point is specific for the current package only, you should add an extra check to make sure it is not run while releasing other packages; something like this should do the trick:

def my_entry_point(data):
    if data['name'] != 'my.package':
        return
    ...

Entry point specification

An entry point is configured like this in your setup.py:

entry_points={
    #'console_scripts': [
    #    'myscript = my.package.scripts:main'],
    'zest.releaser.prereleaser.middle': [
        'dosomething = my.package.some:some_entrypoint',
        ]},

Replace prereleaser and middle in zest.releaser.prereleaser.middle with prerelease/release/postrelease and before/middle/after where needed.

See the setup.py of zest.releaser itself for some real world examples.

You’ll have to make sure that the zest.releaser scripts know about your entry points, for instance by placing your egg (with entry point) in the same zc.recipe.egg section in your buildout as where you placed zest.releaser. Or, if you installed zest.releaser globally, your egg-with-entrypoint has to be globally installed, too.

Comments about data dict items

Your entry point gets a data dictionary: the items you get in that dictionary are documented below. Some comments about them:

  • Not all items are available. If no history/changelog file is found, there won’t be any data['history_lines'] either.

  • Items that are templates are normal python string templates. They use dictionary replacement: they’re actually passed the same data dict. For instance, prerelease’s data['commit_message'] is by default Preparing release %(new_version)s. A “middle” entry point could modify this template to get a different commit message.

Prerelease data dict items

commit_msg

Message template used when committing

history_file

Filename of history/changelog file (when found)

history_header

Header template used for 1st history header

history_lines

List with all history file lines (when found)

name

Name of the project being released

new_version

New version (so 1.0 instead of 1.0dev)

original_version

Version before prereleasing (e.g. 1.0dev)

today

Date string used in history header

workingdir

Original working directory

Release data dict items

name

Name of the project being released

tag_already_exists

Internal detail, don’t touch this :-)

tagdir

Directory where the tag checkout is placed (if a tag checkout has been made)

version

Version we’re releasing

workingdir

Original working directory

Postrelease data dict items

commit_msg

Message template used when committing

dev_version

New development version with dev marker (so 1.1.dev0)

dev_version_template

Template for dev version number

history_header

Header template used for 1st history header

name

Name of the project being released

new_version

New development version (so 1.1)

nothing_changed_yet

First line in new changelog section

workingdir

Original working directory

To do

Credits

  • Reinout van Rees (Nelen & Schuurmans) is the originator and main author.

  • Maurits van Rees (Zest Software) added a heapload of improvements.

  • Kevin Teague (Canada’s Michael Smith Genome Sciences Center) added support for multiple version control systems, most notable Mercurial.

  • Wouter vanden Hove (University of Gent) added support for uploading to multiple servers, using collective.dist.

  • Godefroid Chapelle (BubbleNet) added /tag besides /tags for subversion.

Changelog for zest.releaser

3.34 (2012-03-20)

  • In the warning about a missing MANIFEST.in file, also suggest to install setuptools_subversion/git, etc. Fixes issue #4. [maurits]

3.33 (2012-03-20)

  • Fix python 2.4 issues with tarfile by always creating a zip file. Formerly we would only do this when using python2.4 for doing the release, but a tarball sdist created by python2.6 could still break when the end user is using python 2.4. [kiorky]

3.32 (2012-03-09)

3.31 (2012-02-23)

  • Fixed test for unadvised egg_info commands on tag, which could result in a ConfigParser error. [maurits]

3.30 (2011-12-27)

  • Added some more PyPI classifiers. Tested with Python 2.4, 2,4, 2.6, and 2.7. [maurits]

  • Moved changes of 3.15 and older to docs/HISTORY.txt. [maurits]

  • Added GPL license text in the package. [maurits]

  • Updated README.txt. Added MANIFEST.in. [maurits]

3.29 (2011-12-27)

  • In postrelease create a version number like 1.0.dev0. See http://www.python.org/dev/peps/pep-0386 [maurits]

  • Offer to cleanup setup.cfg on the tag when releasing. You do not want tag_build or tag_svn_revision options in a release usually. [maurits]

  • For convenience also print the tag checkout location when only doing a release (instead of a fullrelease). [maurits]

3.28 (2011-11-18)

  • Git: in pre/postrelease only check for uncommitted changes in files that are already tracked. [maurits]

3.27 (2011-11-12)

  • Postrelease now offers (=asks) to push your changes to the server if you’re using hg or git.

  • Support for some legacy projects, often converted from CVS, have multiple subprojects under a single trunk. The trunk part from the top level project isn’t erroneously stripped out anymore. Thanks to Marc Sibson for the fix.

3.26 (2011-11-01)

  • Added sanity check before doing a prerelease so you are warned when you are about to commit on a tag instead of a branch (or trunk or master). [maurits]

3.25 (2011-10-28)

  • Removed special handling of subversion lower than 1.7 when searching for the history/changes file. In corner cases it may be that we find a wrong HISTORY.txt or CHANGES.txt file when you have it buried deep in your directory structure. Please move it to the root then, which is the proper place for it. [maurits]

  • Fixed finding a history/changes file that is in a sub directory when using subversion 1.7 or higher or bazaar. [maurits]

3.24 (2011-10-19)

  • Note: you may need to install setuptools_subversion when you use subversion 1.7. If you suddenly start missing files in the sdists you upload to PyPI you definitely need it. Alternatively: set up a proper MANIFEST.in as that method works with any version control system. [maurits]

  • Made compatible with subversion 1.7 (the only relevant change is in the code that checks if a tags or tag directory already exists). Earlier versions of subversion are of course still supported. [maurits]

  • Code repository moved to github: https://github.com/zestsoftware/zest.releaser [maurits]

3.23 (2011-09-28)

For older changes see HISTORY.txt in the docs directory.

Project details


Release history Release notifications | RSS feed

This version

3.34

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

zest.releaser-3.34.zip (112.8 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page