zc.recipe.deployment 0.7.1

ZC Buildout recipe for Unix deployments

Contents

• Changes

  • 0.7.1 (2010-03-05)

  • 0.7.0 (2010-02-01)

  • 0.6 (2008-02-01)

  • 0.5 (Mar 23, 2007)

  • 0.4 (Mar 22, 2007)

    • Features Added

    • Bugs Fixed

  • 0.3 (Feb 14, 2007)

  • 0.2.1 (Feb 13, 2007)

  • 0.2 (Feb 7, 2007)

• Detailed Documentation

  • Deployment Name

  • Configuration files

  • Cron support

• Download

The zc.recipe.deployment recipe provides support for deploying applications with multiple processes on Unix systems. (Perhaps support for other systems will be added later.) It creates directories to hold application instance configuration, log and run-time files. It also sets or reads options that can be read by other programs to find out where to place files:

crontab-directory

The name of the directory in which cron jobs should be placed. This is /etc/cron.d.

etc-directory

The name of the directory where configuration files should be placed. This is /etc/NAME, where NAME is the deployment name.

log-directory

The name of the directory where application instances should write their log files. This is /var/log/NAME, where NAME is the deployment name.

logrotate-directory

The name of the directory where logrotate configuration files should be placed, typically, /etc/logrotate.d.

run-directory

The name of the directory where application instances should put their run-time files such as pid files and inter-process communication socket files. This is /var/run/NAME, where NAME is the deployment name.

rc-directory

The name of the directory where run-control scripts should be installed. This is /etc/init.d.

The etc, log, and run directories are created in such a way that the directories are owned by the user specified in the user option and are writable by the user and the user’s group.

Changes

0.7.1 (2010-03-05)

Bugs fixed

• Fixed a serious bug that cause buildouts to fail when using new versions of the deployment recipe with older buildouts.

• Made uninstall more tolerant of directories it’s about to delete already being deleted.

0.7.0 (2010-02-01)

Features Added

You can now specify a user for crontab entries that is different than a deployment user.

0.6 (2008-02-01)

Features Added

Added the ability to specify a name independent of the section name. Also, provide a name option for use by recipes.

Important note to recipe authors: Recipes should use the deployment name option rather than the deployment name when computing names of generated files.

0.5 (Mar 23, 2007)

Features Added

Added recipe for generating crontab files in /etc/cron.d.

0.4 (Mar 22, 2007)

Features Added

• Added setting for the logrotate configuration directories.

Bugs Fixed

• The documentation gave the wrong name for the crontab-directory option.

0.3 (Feb 14, 2007)

Features Added

• Added a configuration recipe for creating configuration files.

0.2.1 (Feb 13, 2007)

• Fixed bug in setup file.

0.2 (Feb 7, 2007)

Bugs Fixed

• Non-empty log and run directories were deleated in un- and re-install.

Detailed Documentation

Using the deployment recipe is pretty simple. Jusr specify a deployment name, specified via the part name, and a deployment user.

Let’s add a deployment to a sample buildout:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = foo
...
... [foo]
... prefix = %s
... recipe = zc.recipe.deployment
... user = %s
... etc-user = %s
... ''' % (sample_buildout, user, user))

>>> print system(join('bin', 'buildout')), # doctest: +NORMALIZE_WHITESPACE
Installing foo.
zc.recipe.deployment:
    Creating 'PREFIX/etc/foo',
    mode 755, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Creating 'PREFIX/var/log/foo',
    mode 755, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Creating 'PREFIX/var/run/foo',
    mode 750, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Creating 'PREFIX/etc/cron.d',
    mode 755, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Creating 'PREFIX/etc/init.d',
    mode 755, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Creating 'PREFIX/etc/logrotate.d',
    mode 755, user 'USER', group 'GROUP'

Note that we are providing a prefix and an etc-user here. These options default to ‘/’ and ‘root’, respectively.

Now we can see that directories named foo in PREFIX/etc, PREFIX/var/log and PREFIX/var/run have been created:

>>> import os
>>> print ls(os.path.join(sample_buildout, 'etc/foo'))
drwxr-xr-x USER GROUP PREFIX/etc/foo

>>> print ls(os.path.join(sample_buildout, 'var/log/foo'))
drwxr-xr-x USER GROUP PREFIX/var/log/foo

>>> print ls(os.path.join(sample_buildout, 'var/run/foo'))
drwxr-x--- USER GROUP PREFIX/var/run/foo

By looking at .installed.cfg, we can see the options available for use by other recipes:

>>> cat('.installed.cfg') # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
[buildout]
...
[foo]
__buildout_installed__ =
...
crontab-directory = PREFIX/etc/cron.d
etc-directory = PREFIX/etc/foo
etc-user = USER
log-directory = PREFIX/var/log/foo
logrotate-directory = PREFIX/etc/logrotate.d
name = foo
prefix = PREFIX
rc-directory = PREFIX/etc/init.d
recipe = zc.recipe.deployment
run-directory = PREFIX/var/run/foo
user = USER

If we uninstall, then the directories are removed.

>>> print system(join('bin', 'buildout')+' buildout:parts='),
Uninstalling foo.
Running uninstall recipe.
zc.recipe.deployment: Removing 'PREFIX/etc/foo'
zc.recipe.deployment: Removing 'PREFIX/etc/cron.d'.
zc.recipe.deployment: Removing 'PREFIX/etc/init.d'.
zc.recipe.deployment: Removing 'PREFIX/etc/logrotate.d'.
zc.recipe.deployment: Removing 'PREFIX/var/log/foo'.
zc.recipe.deployment: Removing 'PREFIX/var/run/foo'.

>>> import os
>>> os.path.exists(os.path.join(sample_buildout, 'etc/foo'))
False
>>> os.path.exists(os.path.join(sample_buildout, 'var/log/foo'))
False
>>> os.path.exists(os.path.join(sample_buildout, 'var/run/foo'))
False

The log and run directories are only removed if they are empty. To see that, we’ll put a file in each of the directories created:

>>> print system(join('bin', 'buildout')),
... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
Installing foo.
zc.recipe.deployment:
    Creating 'PREFIX/etc/foo',
    mode 755, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Creating 'PREFIX/var/log/foo',
    mode 755, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Creating 'PREFIX/var/run/foo',
    mode 750, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Creating 'PREFIX/etc/cron.d',
    mode 755, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Creating 'PREFIX/etc/init.d',
    mode 755, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Creating 'PREFIX/etc/logrotate.d',
    mode 755, user 'USER', group 'GROUP'

>>> write(os.path.join(sample_buildout, 'etc/foo/x'), '')
>>> write(os.path.join(sample_buildout, 'var/log/foo/x'), '')
>>> write(os.path.join(sample_buildout, 'var/run/foo/x'), '')

And then uninstall:

>>> print system(join('bin', 'buildout')+' buildout:parts='),
Uninstalling foo.
Running uninstall recipe.
zc.recipe.deployment: Removing 'PREFIX/etc/foo'
zc.recipe.deployment: Removing 'PREFIX/etc/cron.d'.
zc.recipe.deployment: Removing 'PREFIX/etc/init.d'.
zc.recipe.deployment: Removing 'PREFIX/etc/logrotate.d'.
zc.recipe.deployment: Can't remove non-empty directory 'PREFIX/var/log/foo'.
zc.recipe.deployment: Can't remove non-empty directory 'PREFIX/var/run/foo'.

>>> os.path.exists(os.path.join(sample_buildout, 'etc/foo'))
False

>>> print ls(os.path.join(sample_buildout, 'var/log/foo'))
drwxr-xr-x USER GROUP PREFIX/var/log/foo

>>> print ls(os.path.join(sample_buildout, 'var/run/foo'))
drwxr-x--- USER GROUP PREFIX/var/run/foo

Here we see that the var and run directories are kept. The etc directory is discarded because only buildout recipes should write to it and all of its data are expendible.

If we reinstall, remove the files, and uninstall, then the directories are removed:

>>> print system(join('bin', 'buildout')), # doctest: +NORMALIZE_WHITESPACE
Installing foo.
zc.recipe.deployment:
    Creating 'PREFIX/etc/foo',
    mode 755, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Updating 'PREFIX/var/log/foo',
    mode 755, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Updating 'PREFIX/var/run/foo',
    mode 750, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Creating 'PREFIX/etc/cron.d',
    mode 755, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Creating 'PREFIX/etc/init.d',
    mode 755, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Creating 'PREFIX/etc/logrotate.d',
    mode 755, user 'USER', group 'GROUP'

>>> os.remove(os.path.join(sample_buildout, 'var/log/foo/x'))
>>> os.remove(os.path.join(sample_buildout, 'var/run/foo/x'))

>>> print system(join('bin', 'buildout')+' buildout:parts='),
Uninstalling foo.
Running uninstall recipe.
zc.recipe.deployment: Removing 'PREFIX/etc/foo'
zc.recipe.deployment: Removing 'PREFIX/etc/cron.d'.
zc.recipe.deployment: Removing 'PREFIX/etc/init.d'.
zc.recipe.deployment: Removing 'PREFIX/etc/logrotate.d'.
zc.recipe.deployment: Removing 'PREFIX/var/log/foo'.
zc.recipe.deployment: Removing 'PREFIX/var/run/foo'.

>>> os.path.exists('' + os.path.join(sample_buildout, 'PREFIX/etc/foo'))
False
>>> os.path.exists('' + os.path.join(sample_buildout, 'PREFIX/var/log/foo'))
False
>>> os.path.exists('' + os.path.join(sample_buildout, 'PREFIX/var/run/foo'))
False

Deployment Name

The deployment name is used for naming generated files and directories. The deployment name defaults to the section name, but the deployment name can be specified explicitly:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = foo
...
... [foo]
... recipe = zc.recipe.deployment
... prefix = %s
... name = bar
... user = %s
... etc-user = %s
... ''' % (sample_buildout, user, user))

>>> print system(join('bin', 'buildout')), # doctest: +NORMALIZE_WHITESPACE
Installing foo.
zc.recipe.deployment:
    Creating 'PREFIX/etc/bar',
    mode 755, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Creating 'PREFIX/var/log/bar',
    mode 755, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Creating 'PREFIX/var/run/bar',
    mode 750, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Creating 'PREFIX/etc/cron.d',
    mode 755, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Creating 'PREFIX/etc/init.d',
    mode 755, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Creating 'PREFIX/etc/logrotate.d',
    mode 755, user 'USER', group 'GROUP'

>>> print ls(os.path.join(sample_buildout, 'etc/bar'))
drwxr-xr-x USER GROUP PREFIX/etc/bar

>>> print ls(os.path.join(sample_buildout, 'var/log/bar'))
drwxr-xr-x USER GROUP PREFIX/var/log/bar

>>> print ls(os.path.join(sample_buildout, 'var/run/bar'))
drwxr-x--- USER GROUP PREFIX/var/run/bar

>>> cat('.installed.cfg') # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
[buildout]
installed_develop_eggs =
parts = foo
<BLANKLINE>
[foo]
__buildout_installed__ =
...
crontab-directory = PREFIX/etc/cron.d
etc-directory = PREFIX/etc/bar
etc-user = USER
log-directory = PREFIX/var/log/bar
logrotate-directory = PREFIX/etc/logrotate.d
name = bar
prefix = PREFIX
rc-directory = PREFIX/etc/init.d
recipe = zc.recipe.deployment
run-directory = PREFIX/var/run/bar
user = USER

Note (here and earlier) that the options include the name option, which defaults to the part name. Other parts that use the deployment name should use the name option rather than the part name.

Configuration files

Normally, configuration files are created by specialized recipes. Sometimes, it’s useful to specify configuration files in a buildout configuration file. The zc.recipe.deployment:configuration recipe can be used to do that.

Let’s add a configuration file to our buildout:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = foo x.cfg
...
... [foo]
... recipe = zc.recipe.deployment
... prefix = %s
... user = %s
... etc-user = %s
...
... [x.cfg]
... recipe = zc.recipe.deployment:configuration
... text = xxx
...        yyy
...        zzz
... ''' % (sample_buildout, user, user))

>>> print system(join('bin', 'buildout')), # doctest: +NORMALIZE_WHITESPACE
Uninstalling foo.
Running uninstall recipe.
zc.recipe.deployment: Removing 'PREFIX/etc/bar'
zc.recipe.deployment: Removing 'PREFIX/etc/cron.d'.
zc.recipe.deployment: Removing 'PREFIX/etc/init.d'.
zc.recipe.deployment: Removing 'PREFIX/etc/logrotate.d'.
zc.recipe.deployment: Removing 'PREFIX/var/log/bar'.
zc.recipe.deployment: Removing 'PREFIX/var/run/bar'.
Installing foo.
zc.recipe.deployment:
    Creating 'PREFIX/etc/foo',
    mode 755, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Creating 'PREFIX/var/log/foo',
    mode 755, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Creating 'PREFIX/var/run/foo',
    mode 750, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Creating 'PREFIX/etc/cron.d',
    mode 755, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Creating 'PREFIX/etc/init.d',
    mode 755, user 'USER', group 'GROUP'
zc.recipe.deployment:
    Creating 'PREFIX/etc/logrotate.d',
    mode 755, user 'USER', group 'GROUP'
Installing x.cfg.

By default, the configuration is installed as a part:

>>> cat('parts', 'x.cfg')
xxx
yyy
zzz

If a deployment is specified, then the file is placed in the deployment etc directory:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = foo x.cfg
...
... [foo]
... recipe = zc.recipe.deployment
... prefix = %s
... user = %s
... etc-user = %s
...
... [x.cfg]
... recipe = zc.recipe.deployment:configuration
... text = xxx
...        yyy
...        zzz
... deployment = foo
... ''' % (sample_buildout, user, user))

>>> print system(join('bin', 'buildout')),
Uninstalling x.cfg.
Updating foo.
Installing x.cfg.

>>> os.path.exists(join('parts', 'x.cfg'))
False

>>> cat(os.path.join(sample_buildout, 'etc/foo/x.cfg'))
xxx
yyy
zzz

We can read data from a file rather than specifying in the configuration:

>>> write('x.in', '1\n2\n3\n')

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = foo x.cfg
...
... [foo]
... recipe = zc.recipe.deployment
... prefix = %s
... user = %s
... etc-user = %s
...
... [x.cfg]
... recipe = zc.recipe.deployment:configuration
... file = x.in
... deployment = foo
... ''' % (sample_buildout, user, user))

>>> print system(join('bin', 'buildout')),
Uninstalling x.cfg.
Updating foo.
Installing x.cfg.

>>> cat(os.path.join(sample_buildout, 'etc/foo/x.cfg'))
1
2
3

The recipe sets a location option that can be used by other recipes:

>>> cat('.installed.cfg') # doctest: +ELLIPSIS
[buildout]
...
[x.cfg]
...
location = PREFIX/etc/foo/x.cfg
...

Cron support

The crontab recipe provides support for creating crontab files. It uses a times option to specify times to run the command and a command option containing the command.

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = foo cron
...
... [foo]
... recipe = zc.recipe.deployment
... prefix = %s
... user = %s
... etc-user = %s
...
... [cron]
... recipe = zc.recipe.deployment:crontab
... times = 30 23 * * *
... command = echo hello world!
... deployment = foo
... ''' % (sample_buildout, user, user))

>>> print system(join('bin', 'buildout')),
Uninstalling x.cfg.
Updating foo.
Installing cron.

This example creates PREFIX/etc/cron.d/foo-cron

>>> open(os.path.join(sample_buildout, 'etc/cron.d/foo-cron')).read()
'30 23 * * *\tUSER\techo hello world!\n'

The crontab recipe gets its user from the buildout’s deployment by default, but it doesn’t have to.

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = foo cron
...
... [foo]
... recipe = zc.recipe.deployment
... name = bar
... prefix = %s
... user = %s
... etc-user = %s
...
... [cron]
... recipe = zc.recipe.deployment:crontab
... times = 30 23 * * *
... user = bob
... command = echo hello world!
... deployment = foo
... ''' % (sample_buildout, user, user))

>>> print system(join('bin', 'buildout')), # doctest: +NORMALIZE_WHITESPACE
Uninstalling cron.
Updating foo.
Installing cron.

>>> open('etc/cron.d/bar-cron').read()
'30 23 * * *\tbob\techo hello world!\n'

Download