zc.zodbrecipes

ZC Buildout recipes for ZODB

These details have not been verified by PyPI

Project links

Homepage

GitHub Statistics

View statistics for this project via Libraries.io, or by using our public dataset on Google BigQuery

Project description

Recipes for working with ZODB.

Changes

0.4 (2008-02-18)

New Features

The server recipe now honors the deployment name option.

0.3.1 (2008-01-03)

Bugs Fixed

Shell-scripts using su weren’t generated correctly.

0.3 (2008-01-03)

New Features

You can now specify an email address in a packing crontab file to control where errors are sent.
If you install a general ZEO script, using a zdaemon part:
```
[zdaemon]
recipe = zc.recipe.egg:script
eggs = zdaemon
```
you can request that shell scripts be generated that use the generic zdaemon script. This can make updating software for deployed systems easier since instance-specific scripts don’t depend on paths, like egg names, that are likely to change.

Bugs Fixed

ZODB3 and most of its requirements were spurious dependencies of the recipes. This caused ZODB3 to be installed using the Python used to run the buildout, which caused problems in some circumstances.

0.2.1 (2007-04-23)

Bugs Fixed

crontab and logrotate configuration files were being generates incorrectly.

Detailed Documentation

Defining ZEO Storage Servers

The zc.zodbrecipes:server recipe can be used to define ZEO storage servers. To define a storage server, you define a part for the server and specify configuration data.

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = server
...
... [server]
... recipe = zc.zodbrecipes:server
... zeo.conf =
...    <zeo>
...       address 8100
...       monitor-address 8101
...       transaction-timeout 300
...    </zeo>
...    %import foo
...    <foo main>
...       path /databases/Data.fs
...    </foo>
... ''')

Here we specified a minimal configuration using a “foo” storage. We can use aby kind of storage we want. Here we ised an import statement to import the schema definition that defined the foo section. Any imports are simply copied to the generated configuration file. When we run the buildout:

>>> print system(buildout),
Installing server.
zc.zodbrecipes:
A runzeo script couldn't be found at:
<BLANKLINE>
  '/sample-buildout/bin/runzeo'
<BLANKLINE>
You may need to generate a runzeo script using the
zc.recipe.eggs:script recipe and the ZODB3 egg, or you may need
to specify the location of a script using the runzeo option.
<BLANKLINE>
Generated script '/sample-buildout/bin/server'.

We got a warning because the recipe expects there to be a runzeo script and we haven’t created one. This is done using the zc.recipe.egg:script recipe:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = zodb server
...
... [zodb]
... recipe = zc.recipe.egg:script
... eggs = ZODB3
...
... [server]
... recipe = zc.zodbrecipes:server
... zeo.conf =
...    <zeo>
...       address 8100
...       monitor-address 8101
...       transaction-timeout 300
...    </zeo>
...    %import foo
...    <foo main>
...       path /databases/Data.fs
...    </foo>
... ''')

>>> print system(buildout),
Installing zodb.
Generated script '/sample-buildout/bin/mkzeoinst'.
Generated script '/sample-buildout/bin/fstail'.
Generated script '/sample-buildout/bin/zeopack'.
Generated script '/sample-buildout/bin/runzeo'.
Generated script '/sample-buildout/bin/zeopasswd'.
Generated script '/sample-buildout/bin/zeoctl'.
Generated script '/sample-buildout/bin/fsdump'.
Generated script '/sample-buildout/bin/fsrefs'.
Generated script '/sample-buildout/bin/repozo'.
Generated script '/sample-buildout/bin/fsoids'.
Updating server.

We get 2 things. We get a directory in parts containing ZEO and zdaemon configuration files:

>>> ls('parts', 'server')
-  zdaemon.conf
-  zeo.conf

Let’s look at the configuration files:

>>> cat('parts', 'server', 'zeo.conf')
%import foo
<BLANKLINE>
<zeo>
  address 8100
  monitor-address 8101
  transaction-timeout 300
</zeo>
<BLANKLINE>
<foo main>
  path /databases/Data.fs
</foo>
<BLANKLINE>
<eventlog>
  <logfile>
    path STDOUT
  </logfile>
</eventlog>

We see the same data we input with the addition of an eventlog section that directs logging to standard out. In production, we’ll use zdaemon’s transacript log to capture this logging output in a file. If we wish, we can specify a log file ourselves:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = zodb server
...
... [zodb]
... recipe = zc.recipe.egg:script
... eggs = ZODB3
...
... [server]
... recipe = zc.zodbrecipes:server
... zeo.conf =
...    <zeo>
...       address 8100
...       monitor-address 8101
...       transaction-timeout 300
...    </zeo>
...    %import foo
...    <foo main>
...       path /databases/Data.fs
...    </foo>
...    <eventlog>
...      <logfile>
...        path /var/log/zeo.log
...      </logfile>
...    </eventlog>
... ''')

>>> print system(buildout),
Uninstalling server.
Updating zodb.
Installing server.
Generated script '/sample-buildout/bin/server'.

>>> cat('parts', 'server', 'zeo.conf')
%import foo
<BLANKLINE>
<zeo>
  address 8100
  monitor-address 8101
  transaction-timeout 300
</zeo>
<BLANKLINE>
<foo main>
  path /databases/Data.fs
</foo>
<BLANKLINE>
<eventlog>
  <logfile>
    path /var/log/zeo.log
  </logfile>
</eventlog>

But we’ll stick with the default:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = zodb server
...
... [zodb]
... recipe = zc.recipe.egg:script
... eggs = ZODB3
...
... [server]
... recipe = zc.zodbrecipes:server
... zeo.conf =
...    <zeo>
...       address 8100
...       monitor-address 8101
...       transaction-timeout 300
...    </zeo>
...    %import foo
...    <foo main>
...       path /databases/Data.fs
...    </foo>
... ''')

>>> print system(buildout),
Uninstalling server.
Updating zodb.
Installing server.
Generated script '/sample-buildout/bin/server'.

Let’s look at the zdaemon log file:

>>> cat('parts', 'server', 'zdaemon.conf')
<runner>
  daemon on
  directory /sample-buildout/parts/server
  program /sample-buildout/bin/runzeo -C /sample-buildout/parts/server/zeo.conf
  socket-name /sample-buildout/parts/server/zdaemon.sock
  transcript /sample-buildout/parts/server/zeo.log
</runner>
<BLANKLINE>
<eventlog>
  <logfile>
    path /sample-buildout/parts/server/zeo.log
  </logfile>
</eventlog>

We run the runzeo script with the zeo.conf file. Log and run-time files are places in the server part directory. We use a transcript log to provide the ZEO server log. I like to use the transacriot log because it captures program output, such as start-up exceptions that aren’t captured in a program’s logs.

And we get a control script generated in our bin directory:

>>> cat('bin', 'server')
#!/usr/local/bin/python2.4
<BLANKLINE>
import sys
sys.path[0:0] = [
  '/sample-buildout/eggs/zdaemon-2.0-py2.4.egg',
  '/sample-buildout/eggs/setuptools-0.6-py2.4.egg',
  '/sample-buildout/eggs/ZConfig-2.4-py2.4.egg',
  ]
<BLANKLINE>
import zdaemon.zdctl
<BLANKLINE>
if __name__ == '__main__':
    zdaemon.zdctl.main([
        '-C', '/sample-buildout/parts/server/zdaemon.conf',
        ]+sys.argv[1:]
        )

This is a zdaemon script. We can use this to control the ZEO server process.

Unix Deployment support

The server recipe works with the zc.recipe.deployment. In particular, if a deployment option is specified, it names a part or section that defines the following uptions:

crontab-directory: A directory for crontab files.
etc-directory: A directory for configuration files.
log-directory: A directory for log files.
logrotate-directory: A directory for logrotate configuration files.
rc-directory: A directory for run-control scripts.
run-directory: A directory for run-time files.
user: The user the server process should run as

Let’s create some directories and add a deployment section to our buildout:

>>> for d in 'cron', 'etc', 'log', 'rotate', 'rc', 'run':
...     mkdir(d)
...     globals()[d] = join(sample_buildout, d)

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = zodb server
...
... [zodb]
... recipe = zc.recipe.egg:script
... eggs = ZODB3
...
... [server]
... recipe = zc.zodbrecipes:server
... zeo.conf =
...    <zeo>
...       address 8100
...       monitor-address 8101
...       transaction-timeout 300
...    </zeo>
...    %%import foo
...    <foo main>
...       path /databases/Data.fs
...    </foo>
... deployment = demo
...
... [demo]
... crontab-directory = %(cron)s
... etc-directory = %(etc)s
... log-directory = %(log)s
... logrotate-directory = %(rotate)s
... rc-directory = %(rc)s
... run-directory = %(run)s
... user = bob
... ''' % globals())

>>> print system(buildout),
Uninstalling server.
Updating zodb.
Installing server.
Generated script '/sample-buildout/rc/demo-server'.

Now, the parts directory and the control script will be gone:

>>> import os
>>> os.path.exists(join('parts', 'server'))
False
>>> os.path.exists(join('bin', 'server'))
False

Instead, the control script will be in the rc directory:

>>> cat('rc', 'demo-server')
#!/usr/local/bin/python2.4
<BLANKLINE>
import sys
sys.path[0:0] = [
  '/sample-buildout/eggs/zdaemon-2.0-py2.4.egg',
  '/sample-buildout/eggs/setuptools-0.6-py2.4.egg',
  '/sample-buildout/eggs/ZConfig-2.4-py2.4.egg',
  ]
<BLANKLINE>
import zdaemon.zdctl
<BLANKLINE>
if __name__ == '__main__':
    zdaemon.zdctl.main([
        '-C', '/sample-buildout/etc/server-zdaemon.conf',
        ]+sys.argv[1:]
        )

The run-control script name now combines the deployment name and the script name.

and the configuration files will be in the etc directory:

>>> ls('etc')
-  server-zdaemon.conf
-  server-zeo.conf

In additional we’ll get a logrotate configuration file:

>>> cat('rotate', 'demo-server')
/sample-buildout/log/server-zeo.log {
  rotate 5
  weekly
  postrotate
    /sample-buildout/rc/demo-server -C /sample-buildout/etc/server-zdaemon.conf reopen_transcript
  endscript
}

This will rotate the zeo log file once a week.

If we look at the zdaemon configuration file, we can see that it reflects the deployment locations:

>>> cat('etc', 'server-zdaemon.conf')
<runner>
  daemon on
  directory /sample-buildout/run
  program /sample-buildout/bin/runzeo -C /sample-buildout/etc/server-zeo.conf
  socket-name /sample-buildout/run/server-zdaemon.sock
  transcript /sample-buildout/log/server-zeo.log
  user bob
</runner>
<BLANKLINE>
<eventlog>
  <logfile>
    path /sample-buildout/log/server-zeo.log
  </logfile>
</eventlog>

Note that different file names are used. Since a deployment may be (and usually is) shared by multiple parts, files are prefixed with their part names. Also note that the deployment user is set in the zdaemon configuration.

Packing cron job

We can request definition of a cron job to pack the databases by specifying a pack option. This option takes 5 to 7 values. The first 5 values are the time and date fields defined by Unix crontab files. The sixth field is the number of days in the past to pack to and defaults to 1.

Let’s add a pack option:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = zodb server
...
... [zodb]
... recipe = zc.recipe.egg:script
... eggs = ZODB3
...
... [server]
... recipe = zc.zodbrecipes:server
... zeo.conf =
...    <zeo>
...       address 8100
...       monitor-address 8101
...       transaction-timeout 300
...    </zeo>
...    %%import foo
...    <foo main>
...       path /databases/Data.fs
...    </foo>
... deployment = demo
... pack = 1 1 * * 0 3
...
... [demo]
... crontab-directory = %(cron)s
... etc-directory = %(etc)s
... log-directory = %(log)s
... logrotate-directory = %(rotate)s
... rc-directory = %(rc)s
... run-directory = %(run)s
... user = bob
... ''' % globals())

>>> print system(buildout+' -D'),
Uninstalling server.
Updating zodb.
Installing server.
Generated script '/sample-buildout/rc/demo-server'.

Now, we’ll get a crontab file:

>>> cat(cron, 'pack-demo-server')
1 1 * * 0 bob /sample-buildout/bin/zeopack -p 8100 -S main -d 3

In this example, we’ll pack the databases every Sunday at 1:01 to 3 days.

We can add an email address to the pack option. If we do, then a MAILTO variable will be added to the cron file:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = zodb server
...
... [zodb]
... recipe = zc.recipe.egg:script
... eggs = ZODB3
...
... [server]
... recipe = zc.zodbrecipes:server
... zeo.conf =
...    <zeo>
...       address 8100
...       monitor-address 8101
...       transaction-timeout 300
...    </zeo>
...    %%import foo
...    <foo main>
...       path /databases/Data.fs
...    </foo>
... deployment = demo
... pack = 1 1 * * 0 3 jim@zope.com
...
... [demo]
... crontab-directory = %(cron)s
... etc-directory = %(etc)s
... log-directory = %(log)s
... logrotate-directory = %(rotate)s
... rc-directory = %(rc)s
... run-directory = %(run)s
... user = bob
... ''' % globals())

>>> print system(buildout+' -D'),
Uninstalling server.
Updating zodb.
Installing server.
Generated script '/sample-buildout/rc/demo-server'.

>>> cat(cron, 'pack-demo-server')
MAILTO=jim@zope.com
1 1 * * 0 bob /sample-buildout/bin/zeopack -p 8100 -S main -d 3

This causes email to be sent to the given address, rather than to the user specified in the crontab file.

shell start scripts

By default, the startup scripts are generated Python scripts that use the zdaemon module. Sometimes, this is inconvenient. In particular, when deploying software, generated Python scripts may break after a software update because they contain pasths to software eggs. We can request shell scripts that invoke a generic zdaemon script. The shell script only depends on the path to the zdaemon script, which generally doesn’t change when updating softawre.

To request a shell script, add a shell-script option with a true value. We also need to cause a zdaemon script to be generated:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = zodb zdaemon server
...
... [zodb]
... recipe = zc.recipe.egg:script
... eggs = ZODB3
...
... [zdaemon]
... recipe = zc.recipe.egg:script
... eggs = zdaemon
...
... [server]
... recipe = zc.zodbrecipes:server
... zeo.conf =
...    <zeo>
...       address 8100
...       monitor-address 8101
...       transaction-timeout 300
...    </zeo>
...    %%import foo
...    <foo main>
...       path /databases/Data.fs
...    </foo>
... deployment = demo
... pack = 1 1 * * 0 3 jim@zope.com
... shell-script = true
...
... [demo]
... name = Demo
... crontab-directory = %(cron)s
... etc-directory = %(etc)s
... log-directory = %(log)s
... logrotate-directory = %(rotate)s
... rc-directory = %(rc)s
... run-directory = %(run)s
... user = bob
... ''' % globals())

>>> print system(buildout+' -D'),
Uninstalling server.
Updating zodb.
Installing zdaemon.
Generated script '/sample-buildout/bin/zdaemon'.
Installing server.
zc.zodbrecipes: Generated shell script '/sample-buildout/rc/Demo-server'.

>>> cat('rc', 'Demo-server')
#!/bin/sh
su bob -c \
  "/sample-buildout/bin/zdaemon -C '/sample-buildout/etc/server-zdaemon.conf' $*"

Download

Project details

These details have not been verified by PyPI

Project links

Homepage

GitHub Statistics

View statistics for this project via Libraries.io, or by using our public dataset on Google BigQuery

Release history Release notifications | RSS feed

3.0

Feb 7, 2023

2.0.0

Feb 12, 2014

0.6.2

Oct 31, 2012

0.6.1

May 25, 2010

0.6.0

Dec 3, 2009

0.5.0

Nov 3, 2008

This version

0.4.0

Feb 18, 2008

0.3.1

Jan 3, 2008

0.3.0

Jan 3, 2008

0.2.1

Apr 23, 2007

0.2

Apr 18, 2007

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

zc.zodbrecipes-0.4.0.tar.gz (12.2 kB view hashes)

Uploaded Feb 18, 2008 Source

Hashes for zc.zodbrecipes-0.4.0.tar.gz

Hashes for zc.zodbrecipes-0.4.0.tar.gz
Algorithm	Hash digest
SHA256	`ba8259483b7c4c4bd3311113b77c284a0cc26a69521532d33cf695884d0273e5`
MD5	`3e145c625f920219cb5bc071fb0dd2f8`
BLAKE2b-256	`7a5a6cd19bd467ca4673d5dd61df3c3f2d3df95d8757ad06932d1b4518397370`