minitage.recipe.scripts 1.56

zc.buildout recipes to compile and install software or python packages and generate scripts or configuration files sponsored by Makina Corpus.

Contents

• Introduction

• Makina Corpus sponsored software

• minitage.recipe.scripts

  • Abstract

  • Specific options

  • Detailled documentation

    • Generating all scripts

    • Selecting scripts to install

    • Declaring entry-points

    • Adding initialization code

    • Adding arguments

    • Generating a python interpreter

    • Generating an envrionment file

• CHANGELOG

  • 1.56 (2013-02-13)

  • 1.55 (2011-03-11)

  • 1.50

  • 1.47

  • 1.45

  • 1.44

  • 1.41

  • 1.39

  • 1.37

  • 1.36

  • 1.35

Introduction

This is a collection of recipe which can be use inside or outside a minitage environment. What is interresting in using them in minitage is that you ll have all your system dependencies in the build environment automaticly.

The egg has those entry point:

• cmmi: install configure/make/make install softwares

• fetch: fetch something, somewhere, with git, http, frp, static, hg, svn or bzr.

• egg: install python eggs / packages ‘setuptoolisables’

• printer: print or dump to a file all versions needed to achieve eggs requirements (versions.cfg made easy)

• scripts: install scripts from an egg and install egg dependencies if they are not already in the cache

• wsgi: Make a Python paste configuration file eatable by mod_wsgi with all the eggs dependencies you need.

The reasons why i have rewrite yet another buildout recipes builder are:

• Support for downloading stuff

• Do not rely on easy_install dependency system

• Support on the fly patchs for eggs and other distribution.

• Support multiple hooks at each stage of the build system.

• Support for distutils

• Robust offline mode

• We like pypi, but offer a mode to scan for eggs without need to check the index,

• Support malformed or not indexed distributions. In other terms, we provide an url, and the recipe builds it, that’s all.

• All recipes must support automaticly minitage dependencies and rpath linking.

You can browse the code on minitage’s following resources:

• http://git.minitage.org/git/minitage/eggs/minitage.recipe/

• http://www.minitage.org/trac/browser/minitage/eggs/minitage.recipe

You can migrate your buldouts without any effort with buildout.minitagificator:

• http://pypi.python.org/pypi/buildout.minitagificator

Makina Corpus sponsored software

• Planet Makina Corpus

• Contact us

minitage.recipe.scripts

Abstract

• This recipe intends to install eggs and python software and on top of installed stuff, generating scripts and envrionment files.

• This recipe inherit from minitage;recipe:egg.

• Its heavilly inspired by zc.recipe.eggs* and try to completly replace it whereas be API compatbile.

• You can use it in conjunction with the buildout.minitagificator extension which monkey patch zc.buildout to use minitage recipes.

• What we can do that zc.recipe.egg wouldnt do, either at all or not in the way we want to:

  • All scripts support initialisation code

  • The ‘scripts’ egg metadata is also handled

• You can use it as you would use minitage.recipe:egg, use patch facility and etc.

• Ths recipe is also declared under those entry points:

  • minitage.recipe:eggs

  • minitage.recipe:script

Specific options

• All the shared options and the options from minitage.recipe:egg +

• scripts

  • Scripts to generate, if empty, generate scripts for all the working set.

  • If your egg have an old ‘scripts’ metadata, and old scripts where you want wrappers to be generated, just add the egg name to the scripts entry.

  • If you want to rename a script, just enter something like entrypoint|scriptname=NewName:

    s=NewName

• zap If you do not want to a script, just enter a line separated list of not wanted scripts

• entry-points

  A list of entry-point identifiers of the form::

  name=module:attrs

  where name is a script name, module is a dotted name resolving to a module name, and attrs is a dotted name resolving to a callable object within a module. This option is useful when working with distributions that don’t declare entry points, such as distributions not written to work with setuptools.

• interpreter

  The name of a script to generate that allow access to the Python interpreter with the PYTHONPATH set with all the working set entries.

• dependent-scripts When set to true, scripts will be generated for all required eggs in addition to the eggs named specifically. This idea came from two forks of this recipe, repoze.recipe.egg and pylons_sandbox, but the option name is spelled with a dash instead of underscore and it defaults to false.

• (script-name|interpretername-)arguments

  Specify some arguments to be passed to entry points as Python source.

• (script-name|interpretername-)initialization

  Python code to run prior to call the entry point

• env_initialization

  run code after initialization of env (shell)

• template-replacements

  • replacements for variables values in templates to override buildout’s strip mecanism (initialization, env_initialization, arguments)

  • Spaces before and after => are important:

  • Pattern are in the form:

    re-pattern => value

Detailled documentation

Generating all scripts

Thus by not specifying any scripts entry in the buildout part.

>>> init_env()
>>> index_url = start_server(os.path.sep.join(tempdir))
>>> data = """
... [buildout]
... eggs-directory =${buildout:directory}/eggs
... download-cache=${buildout:directory}/dl
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... eggs=foo
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> noecho = [remove(os.path.join('eggs', egg)) for egg in os.listdir('eggs') if 'foo' in egg]
>>> sh('bin/buildout -vvvvv install')
b...
minitage.recipe: Generated scripts: 's', 'toto.py'...

Look at what have been generated.

>>> cat('bin', 'toto.py')
#!...
# ! GENERATED BY minitage.recipe !
import os
import sys
import subprocess...
sys.path[0:0] = ['/tmp/buildout.test/eggs/foo-1.0-py....egg' ]...
# EXEC ORGINAL CODE WITHOUT SHEBANG
__doc__  = 'I am generated by minitage.recipe.script recipe'...
os.environ['PYTHONPATH'] = ':'.join(sys.path + os.environ.get('PYTHONPATH', '').split(':'))
sys.argv.pop(0)
sys.exit(
    subprocess.Popen(
        [sys.executable, '/tmp/buildout.test/eggs/foo-1.0-py....egg/EGG-INFO/scripts/toto.py']+sys.argv,
        env=os.environ
    ).wait()
)...
>>> cat('bin', 's')
#!...
#!!! #GENERATED VIA MINITAGE.recipe !!!...
import sys...
sys.path[0:0] = [ '/tmp/buildout.test/eggs/foo-1.0-py....egg' ]...
import toto.toto...
if __name__ == '__main__':
    toto.toto.f()...

Selecting scripts to install

Installing only s.

>>> init_env()
>>> index_url = start_server(os.path.sep.join(tempdir))
>>> data = """
... [buildout]
... eggs-directory =${buildout:directory}/eggs
... download-cache=${buildout:directory}/dl
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... scripts =
...     s
... eggs=foo
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install')
b...
minitage.recipe: Generated scripts: 's'...

Installing only toto.py.

>>> data = """
... [buildout]
... eggs-directory =${buildout:directory}/eggs
... download-cache=${buildout:directory}/dl
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... scripts =
...     toto.py
... eggs=foo
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install')
b...
minitage.recipe: Generated scripts: 'toto.py'....

Declaring entry-points

We ll add an entry point ‘t’ to be generated.

>>> init_env()
>>> index_url = start_server(os.path.sep.join(tempdir))
>>> data = """
... [buildout]
... eggs-directory =${buildout:directory}/eggs
... download-cache=${buildout:directory}/dl
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... entry-points=t=toto.toto:f
... eggs=foo
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install')
b...
minitage.recipe: Generated scripts: 't'...
>>> cat('bin', 't') #doctest: +REPORT_NDIFF
#!...
#!!! #GENERATED VIA MINITAGE.recipe !!!
<BLANKLINE>
import sys
<BLANKLINE>
<BLANKLINE>
<BLANKLINE>
sys.path[0:0] = [ '/tmp/buildout.test/eggs/foo-1.0-...
...]
<BLANKLINE>
<BLANKLINE>
<BLANKLINE>
import toto.toto
<BLANKLINE>
if __name__ == '__main__':
    toto.toto.f()
<BLANKLINE>

Adding initialization code

What about adding environment variables for gis env.

>>> init_env()
>>> index_url = start_server(os.path.sep.join(tempdir))
>>> data = """
... [buildout]
... eggs-directory =${buildout:directory}/eggs
... download-cache=${buildout:directory}/dl
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... entry-points=t=toto.toto:f
... eggs=foo
... initialization = import os;os.environ.set('GDAL', 'TRUE')
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install')
b...

>>> "import os;os.environ.set('GDAL', 'TRUE')" in open(os.path.join('bin', 't')).read()
True

Adding arguments

What about adding arguments to our launchers.

>>> init_env()
>>> index_url = start_server(os.path.sep.join(tempdir))
>>> data = """
... [buildout]
... eggs-directory =${buildout:directory}/eggs
... download-cache=${buildout:directory}/dl
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... eggs = foo
... entry-points=t=toto.toto:f
... arguments = ['a', 'b']
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install')
b...

>>> "toto.toto.f(['a', 'b'])" in open(os.path.join('bin', 't')).read()
True

Generating a python interpreter

Here is how you can generate a specific python interpreter will all the environement of the working set.

>>> init_env()
>>> index_url = start_server(os.path.sep.join(tempdir))
>>> data = """
... [buildout]
... eggs-directory =${buildout:directory}/eggs
... download-cache=${buildout:directory}/dl
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... interpreter = mypy
... arguments = ['a', 'b']
... eggs=foo
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install')
b...
minitage.recipe: Generated scripts: 'mypy',...

>>> cat('bin', 'mypy')
#!...
#!!! #GENERATED VIA MINITAGE.recipe !!!...
sys.path[0:0] = [ '/tmp/buildout.test/eggs/foo-1.0-py....egg' ]...
if _interactive:
    import code
    code.interact(banner="", local=globals())...

Generating an envrionment file

Here is how you can generate a specific envrionment file that you can source from to get the PYTHONPATH populated with eggs that you have configured.

>>> init_env()
>>> index_url = start_server(os.path.sep.join(tempdir))
>>> data = """
... [buildout]
... download-cache=${buildout:directory}/dl
... eggs-directory =${buildout:directory}/eggs
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... env-file = mypy
... eggs=foo
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install')
b...
>>> cat('bin', 'mypy')
#!/usr/bin/env sh
<BLANKLINE>
PYTHONPATH="/tmp/buildout.test/eggs/foo-1.0..."
export PYTHONPATH
<BLANKLINE>
<BLANKLINE>
<BLANKLINE>
<BLANKLINE>

CHANGELOG

1.56 (2013-02-13)

• Keep return code from scripts

1.55 (2011-03-11)

• Use os.execvpe to spawn processes, not subprocess. Thx to Dave Foster (daf)

• Split tests

• Add initialization for interpreter script

• Fix type ‘list’and pattern replacement code

1.50

• workaround to install when there are conflicts from versions pinned in setup.py and different ones in buildout

1.47

• Added the dependent-scripts option to stick to zc.recipe.egg

1.45

• better handle of entrypoints/console script conflict options (cs. prefix, see code)

• add per script options (initialization, argurments, python)

1.44

• when both EntryPoint and Script have the same name, it’s the EP which wins.

1.41

• fix a special case where all scripts were filtered out

1.39

• do not preerve PYTHONPATH on env.file generation

1.37

• Decorator helper

1.36

• fix develop link

1.35

• splitted out from minitage.recipe