lovely.recipe 1.0.0

Set of helper recipies for zc.buildout

Filesystem Buildout Recipe

Creating Directories

>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = data-dir
... find-links = http://download.zope.org/distribution
...
... [data-dir]
... recipe = lovely.recipe:mkdir
... path = ${buildout:directory}/mystuff
... """)
>>> print system(buildout),
Installing data-dir.
data-dir: Creating directory /sample-buildout/mystuff

>>> ls(sample_buildout)
-  .installed.cfg
d  bin
-  buildout.cfg
d  develop-eggs
d  eggs
d  mystuff
d  parts

If we change the directory name the old directory (‘mystuff’) is not deleted.

>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = data-dir
... find-links = http://download.zope.org/distribution
...
... [data-dir]
... recipe = lovely.recipe:mkdir
... path = ${buildout:directory}/otherdir
... """)
>>> print system(buildout),
Uninstalling data-dir.
Installing data-dir.
data-dir: Creating directory /sample-buildout/otherdir

>>> ls(sample_buildout)
-  .installed.cfg
d  bin
-  buildout.cfg
d  develop-eggs
d  eggs
d  mystuff
d  otherdir
d  parts

We can also create a full path.

>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = data-dir
... find-links = http://download.zope.org/distribution
...
... [data-dir]
... recipe = lovely.recipe:mkdir
... path = ${buildout:directory}/with/subdir
... """)
>>> print system(buildout),
Uninstalling data-dir.
Installing data-dir.
data-dir: Cannot create /sample-buildout/with/subdir. /sample-buildout/with is not a directory.
While:
  Installing data-dir.
Error: Invalid Path

But we need to activate this function explicitely.

>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = data-dir
... find-links = http://download.zope.org/distribution
...
... [data-dir]
... recipe = lovely.recipe:mkdir
... createpath = True
... path = ${buildout:directory}/with/subdir
... """)
>>> print system(buildout),
Installing data-dir.
data-dir: Creating parent directory /sample-buildout/with
data-dir: Creating directory /sample-buildout/with/subdir

>>> ls(sample_buildout)
-  .installed.cfg
d  bin
-  buildout.cfg
d  develop-eggs
d  eggs
d  mystuff
d  otherdir
d  parts
d  with
>>> ls(sample_buildout + '/with')
d  subdir

There is no update method so the install method is used upon update and the directories get recreated.

>>> rmdir(sample_buildout + '/with')
>>> print system(buildout),
Updating data-dir.
The recipe for data-dir doesn't define an update method. Using its install method.
data-dir: Creating parent directory /sample-buildout/with
data-dir: Creating directory /sample-buildout/with/subdir

We can change the owner of the created directory if run as root. This is tested in mkdir-root.txt.

If not run as root, setting the owner is an error:

>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = data-dir
... find-links = http://download.zope.org/distribution
...
... [data-dir]
... recipe = lovely.recipe:mkdir
... createpath = True
... path = ${buildout:directory}/another/with/subdir
... owner = nobody
... """)
>>> print system(buildout),
While:
  Installing.
  Getting section data-dir.
  Initializing part data-dir.
Error: Only root can change the owner to nobody.

It is an error when the user does not exist:

>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = data-dir
... find-links = http://download.zope.org/distribution
...
... [data-dir]
... recipe = lovely.recipe:mkdir
... createpath = True
... path = ${buildout:directory}/another/with/subdir
... owner = someuser
... """)
>>> print system(buildout),
While:
  Installing.
  Getting section data-dir.
  Initializing part data-dir.
Error: The user someuser does not exist.

Creating Files

The mkfile recipe creates one or more files with a given path, content and permissions.

Note that the parent directory needs to exist, otherwise a user error is raised.

>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = script
...
... [script]
... recipe = lovely.recipe:mkfile
... path = /x/y/file.sh
... content = hoschi
... mode = 0755
... """)
>>> print system(buildout)
Uninstalling data-dir.
Installing script.
script: Cannot create file /x/y/file.sh. /x/y is not a directory.
While:
  Installing script.
Error: Invalid path

>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = script
...
... [script]
... recipe = lovely.recipe:mkfile
... path = file.sh
... content = hoschi
... mode = 0755
... """)
>>> print system(buildout)
Installing script.
script: Writing file /sample-buildout/file.sh
<BLANKLINE>

>>> ls(sample_buildout)
-  .installed.cfg
d  bin
-  buildout.cfg
d  develop-eggs
d  eggs
-  file.sh
d  mystuff
d  otherdir
d  parts
d  with

The content is written to the file.

>>> cat(sample_buildout, 'file.sh')
hoschi

And the mode is set.

>>> import os, stat
>>> path = os.path.join(sample_buildout, 'file.sh')
>>> oct(stat.S_IMODE(os.stat(path)[stat.ST_MODE]))
'0755'

If we change the filename the old file is deleted.

>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = script
...
... [script]
... recipe = lovely.recipe:mkfile
... path = newfile.sh
... content = hoschi
... mode = 0755
... """)
>>> print system(buildout)
Uninstalling script.
Installing script.
script: Writing file /sample-buildout/newfile.sh
<BLANKLINE>

>>> ls(sample_buildout)
-  .installed.cfg
d  bin
-  buildout.cfg
d  develop-eggs
d  eggs
d  mystuff
-  newfile.sh
d  otherdir
d  parts
d  with

We can also specify to create the path for the file.

>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = script
...
... [script]
... recipe = lovely.recipe:mkfile
... createpath = On
... path = subdir/for/file/file.sh
... content = hoschi
... mode = 0755
... """)
>>> print system(buildout)
Uninstalling script.
Installing script.
script: Creating directory /sample-buildout/subdir/for/file
script: Writing file /sample-buildout/subdir/for/file/file.sh

>>> ls(sample_buildout + '/subdir/for/file')
-  file.sh

File Variations

A common use-case is to have variations of a file, for example if init scripts have to be created. As an example we create two files with variations “1” and “2”. These variations can be used in the file path and in the content of the file via normal string formatting notation.

>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = script
...
... [script]
... recipe = lovely.recipe:mkfile
... variations = 1 2
... path = prod_%(variation)s.ini
... content = hoschi variation %(variation)s
... mode = 0755
... """)
>>> print system(buildout)
Uninstalling script.
Installing script.
script: Writing file ...sample-buildout/prod_1.ini
script: Writing file ...sample-buildout/prod_2.ini

>>> cat(sample_buildout, 'prod_1.ini')
hoschi variation 1
>>> cat(sample_buildout, 'prod_2.ini')
hoschi variation 2

Egg Box Buildout Recipe

This recipe is derivd from zc.recipe.egg, but instead of just creating paths, it generates a directory structure for each top-level namespace. It is also possible to automatically zip the generated directories which is espacially usefull if used in Google Appengine environments. The recipies path option is filled with the created path so it can be referenced by other buildout sections which may want to use the recipe.

>>> import os
>>> lovely_recipy_loc = os.path.dirname(os.path.dirname(os.path.dirname(
...     os.path.dirname(os.path.dirname(__file__)))))

>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... develop = %(loc)s
... parts = packages
... find-links = %(server)s
... index = %(server)s/index
...
... [packages]
... recipe = lovely.recipe:eggbox
... eggs = demo
...        lovely.recipe
... interpreter = py
... """ % dict(loc=lovely_recipy_loc, server=link_server))

>>> print system(buildout)
Develop: '...lovely.recipe'
Getting distribution for 'demo'.
Got demo 0.4c1.
Getting distribution for 'demoneeded'.
Got demoneeded 1.2c1.
Installing packages.
Generated script '...sample-buildout/bin/demo'.
Generated interpreter '...sample-buildout/bin/py'.

We now have a zip file for each top-level directory. Note that the zip-files are ending with .egg for pkg_resources compatibility.

>>> ls(sample_buildout + '/parts/packages')
-  easy_install.py.egg
-  eggrecipedemo.py.egg
-  eggrecipedemoneeded.py.egg
-  lovely.egg
-  pkg_resources.py.egg
-  setuptools.egg
-  zc.egg

The generated interpreter now has the demo zip file in the path.

>>> cat(sample_buildout + '/bin/py')
#!...
sys.path[0:0] = [
  '/sample-buildout/parts/packages/easy_install.py.egg',
  '/sample-buildout/parts/packages/eggrecipedemo.py.egg',
  '/sample-buildout/parts/packages/eggrecipedemoneeded.py.egg',
  '/sample-buildout/parts/packages/lovely.egg',
  '/sample-buildout/parts/packages/pkg_resources.py.egg',
  '/sample-buildout/parts/packages/setuptools.egg',
  '/sample-buildout/parts/packages/zc.egg',
  ]...

It is possible to disable zipping. And also to exclude or include patterns of files. So for example we can strip down the uneeded setuptools egg. We can also create a script.

>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... develop = %(loc)s
... parts = packages test
... find-links = %(server)s
... index = %(server)s/index
...
... [packages]
... zip = False
... recipe = lovely.recipe:eggbox
... eggs = demo
...        lovely.recipe
... excludes = ^setuptools/.*
...            ^easy_install.*
...            ^pkg_resources.*
...
... [test]
... recipe = zc.recipe.egg:scripts
... eggs = lovely.recipe
... extra-paths = ${packages:path}
... interpreter = py
... """ % dict(loc=lovely_recipy_loc, server=link_server))
>>> print system(buildout),
Develop: '/Users/bd/sandbox/lovely.recipe'
Uninstalling packages.
Installing packages.
Generated script '/sample-buildout/bin/demo'.
Installing test.
Generated interpreter '/sample-buildout/bin/py'.

Note that we still have the same directory structure as the zipped version with a directory for each top-level namespace. The ‘lovely’ directory is not in he packages directory because it is a develop egg and we have set zipped to false, therefore it is only added to the python path.

>>> ls(sample_buildout + '/parts/packages')
d  eggrecipedemo.py
d  eggrecipedemoneeded.py
d  zc

>>> print system(join(sample_buildout, 'bin', 'py') + \
...        ' -c "import lovely.recipe; print lovely.recipe.__file__"')
/.../src/lovely/recipe/__init__.py...

The test section uses the path of our packages section. Note that due, to the development path of lovely.recipe this path is actually included twice because the script recipe does not check duplicates.

>>> cat(sample_buildout + '/bin/py')
#!...
sys.path[0:0] = [
  '/...lovely.recipe/src',
  ...
  '/.../lovely.recipe/src',
  '/sample-buildout/parts/packages/eggrecipedemo.py',
  '/sample-buildout/parts/packages/eggrecipedemoneeded.py',
  '/sample-buildout/parts/packages/zc',
  ]...

i18n Tools Recipe

This recipe allows to create i18n tools to extract and merge po files.

Creating The Tools

>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = i18n
... index = http://download.zope.org/zope3.4
... offline = true
...
... [i18n]
... recipe = lovely.recipe:i18n
... package = lovely.recipe
... domain = recipe
... location = src/somewhere
... output = locales
... maker = z3c.csvvocabulary.csvStrings
... """)
>>> print system(buildout),
Installing i18n.
i18n: setting up i18n tools
Generated script 'bin/i18nextract'.
Generated script 'bin/i18nmergeall'.
Generated script 'bin/i18nstats'.

>>> import os
>>> ls(os.path.join(sample_buildout, 'bin'))
-  buildout
-  i18nextract
-  i18nmergeall
-  i18nstats

The i18n Extractor

>>> cat('bin', 'i18nextract')
#!...
<BLANKLINE>
import sys
sys.path[0:0] = [
...
  ]
<BLANKLINE>
import lovely.recipe.i18n.i18nextract
<BLANKLINE>
if __name__ == '__main__':
    lovely.recipe.i18n.i18nextract.main(['i18nextract', '-d', 'recipe', '-s', '/sample-buildout/parts/i18n/configure.zcml', '-p', 'src/somewhere', '-o', 'locales', '-m', 'z3c.csvvocabulary.csvStrings'])

We have a configure.zcml created.

>>> cat('parts', 'i18n', 'configure.zcml')
<configure xmlns='http://namespaces.zope.org/zope'>
  <include package="lovely.recipe" />
</configure>

i18n Merge

>>> cat('bin', 'i18nmergeall')
#!...
<BLANKLINE>
import sys
sys.path[0:0] = [
...
  ]
<BLANKLINE>
import lovely.recipe.i18n.i18nmergeall
<BLANKLINE>
if __name__ == '__main__':
    lovely.recipe.i18n.i18nmergeall.main(['i18nmergeall', '-l', 'src/somewhere/locales'])

i18n Stats

>>> cat('bin', 'i18nstats')
#!...
<BLANKLINE>
import sys
sys.path[0:0] = [
...
  ]
<BLANKLINE>
import lovely.recipe.i18n.i18nstats
<BLANKLINE>
if __name__ == '__main__':
    lovely.recipe.i18n.i18nstats.main(['i18nstats', '-l', 'src/somewhere/locales'])

Tool Names

The created tools are named after the section name. If the section for the recipe is named ‘translation’ then the tools are named ‘translationextract’ and ‘translationmergeall’.

>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... index = http://download.zope.org/zope3.4
... parts = translation
...
... offline = true
...
... [translation]
... recipe = lovely.recipe:i18n
... package = lovely.recipe
... domain = recipe
... location = src/somewhere
... output = locales
... maker = z3c.csvvocabulary.csvStrings
... """)
>>> print system(buildout),
Uninstalling i18n.
Installing translation.
translation: setting up i18n tools
Generated script 'bin/translationextract'.
Generated script 'bin/translationmergeall'.
Generated script 'bin/translationstats'.

Adding a custom configure.zcml

The created configure.zcml includes the package an assumes that the package contains a configure.zcml. If this is not the case or if additional package includes are needed then the zcml parameter can be used to define the content of the generated configure.zcml.

>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = i18n
...
... offline = true
...
... [i18n]
... recipe = lovely.recipe:i18n
... package = lovely.recipe
... domain = recipe
... location = src/somewhere
... output = locales
... maker = z3c.csvvocabulary.csvStrings
... zcml =
...    <include package='zope.component' file='meta.zcml' />
...    <include package='lovely.recipe' />
...
... """)

>>> print system(buildout),
Uninstalling translation.
Installing i18n.
i18n: setting up i18n tools
Generated script 'bin/i18nextract'.
Generated script 'bin/i18nmergeall'.
Generated script 'bin/i18nstats'.

>>> cat('bin', 'i18nextract')
#!...
<BLANKLINE>
import sys
sys.path[0:0] = [
...
  ]
<BLANKLINE>
import lovely.recipe.i18n.i18nextract
<BLANKLINE>
if __name__ == '__main__':
    lovely.recipe.i18n.i18nextract.main(['i18nextract', '-d', 'recipe', '-s', '/sample-buildout/parts/i18n/configure.zcml', '-p', 'src/somewhere', '-o', 'locales', '-m', 'z3c.csvvocabulary.csvStrings'])

And the generated configure-zcml contains our extra code.

>>> cat('parts', 'i18n', 'configure.zcml')
<configure xmlns='http://namespaces.zope.org/zope'
           xmlns:meta="http://namespaces.zope.org/meta"
           >
<BLANKLINE>
<BLANKLINE>
<include package='zope.component' file='meta.zcml' />
<include package='lovely.recipe' />
<BLANKLINE>
</configure>

Importchecker Recipe

This recipe creates an importchecker instance in the bin directory.

Creating The Script

>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = checker
...
... offline = true
...
... [checker]
... recipe = lovely.recipe:importchecker
... path = src/lovely
... """)
>>> print system(buildout),
Installing checker.
checker: setting up importchecker
Generated script 'bin/importchecker'.

>>> import os
>>> ls(os.path.join(sample_buildout, 'bin'))
-  buildout
-  importchecker

>>> cat('bin', 'importchecker')
#!...
<BLANKLINE>
import sys
sys.path[0:0] = [
  ...
  ]
<BLANKLINE>
import lovely.recipe.importchecker.importchecker
<BLANKLINE>
if __name__ == '__main__':
    lovely.recipe.importchecker.importchecker.main(['importchecker', 'src/lovely'])