minitage.recipe.scripts 1.46

zc.buildout recipes to compile and install software or python packages and generate scripts or configuration files.

Contents

• Introduction

• minitage.recipe.scripts

  • Abstract

  • Specific options

  • Detailled documentation

    • Initializing test env.

    • Generating all scripts

    • Selecting scripts to install

    • Declaring entry-points

    • Adding initialization code

    • Adding arguments

    • Generating a python interpreter

    • Generating an envrionment file

• CHANGELOG

  • 1.45

  • 1.44

  • 1.41

  • 1.39

  • 1.37

  • 1.36

  • 1.35

Introduction

This is a collection of recipe which can be use inside or outside a minitage environment. What is interresting in using them in minitage is that you ll have all your system dependencies in the build environment automaticly.

The egg has those entry point:

• cmmi: install configure/make/make install softwares

• fetch: fetch something, somewhere, with git, http, frp, static, hg, svn or bzr.

• egg: install python eggs / packages ‘setuptoolisables’

• printer: print or dump to a file all versions needed to achieve eggs requirements (versions.cfg made easy)

• scripts: install scripts from an egg and install egg dependencies if they are not already in the cache

• wsgi: Make a Python paste configuration file eatable by mod_wsgi with all the eggs dependencies you need.

The reasons why i have rewrite yet another buildout recipes builder are:

• Support for downloading stuff

• Do not rely on easy_install dependency system

• Support on the fly patchs for eggs and other distribution.

• Support multiple hooks at each stage of the build system.

• Support for distutils

• Robust offline mode

• We like pypi, but offer a mode to scan for eggs without need to check the index,

• Support malformed or not indexed distributions. In other terms, we provide an url, and the recipe builds it, that’s all.

• All recipes must support automaticly minitage dependencies and rpath linking.

You can browse the code on minitage’s following resources:

• http://git.minitage.org/git/minitage/eggs/minitage.recipe/

• http://www.minitage.org/trac/browser/minitage/eggs/minitage.recipe

You can migrate your buldouts without any effort with buildout.minitagificator:

• http://pypi.python.org/pypi/buildout.minitagificator

minitage.recipe.scripts

Abstract

• This recipe intends to install eggs and python software and on top of installed stuff, generating scripts and envrionment files.

• This recipe inherit from minitage;recipe:egg.

• Its heavilly inspired by zc.recipe.eggs* and try to completly replace it whereas be API compatbile.

• You can use it in conjunction with the buildout.minitagificator extension which monkey patch zc.buildout to use minitage recipes.

• What we can do that zc.recipe.egg wouldnt do, either at all or not in the way we want to:

  • All scripts support initialisation code

  • The ‘scripts’ egg metadata is also handled

• You can use it as you would use minitage.recipe:egg, use patch facility and etc.

• Ths recipe is also declared under those entry points:

  • minitage.recipe:eggs

  • minitage.recipe:script

Specific options

• All the shared options and the options from minitage.recipe:egg +

• scripts

  • Scripts to generate, if empty, generate scripts for all the working set.

  • If your egg have an old ‘scripts’ metadata, and old scripts where you want wrappers to be generated, just add the egg name to the scripts entry.

  • If you want to rename a script, just enter something like entrypoint|scriptname=NewName:

    s=NewName

• zap If you do not want to a script, just enter a line separated list of not wanted scripts

• entry-points

  A list of entry-point identifiers of the form::

  name=module:attrs

  where name is a script name, module is a dotted name resolving to a module name, and attrs is a dotted name resolving to a callable object within a module. This option is useful when working with distributions that don’t declare entry points, such as distributions not written to work with setuptools.

• interpreter

  The name of a script to generate that allow access to the Python interpreter with the PYTHONPATH set with all the working set entries.

• dependent-scripts

  compatibility option, has no effect

• (script-name|interpretername-)arguments

  Specify some arguments to be passed to entry points as Python source.

• (script-name|interpretername-)initialization

  Python code to run prior to call the entry point

• env_initialization

  run code after initialization of env (shell)

• template-replacements

  • replacements for variables values in templates to override buildout’s strip mecanism (initialization, env_initialization, arguments)

  • Spaces before and after => are important:

  • Pattern are in the form:

    re-pattern => value

Detailled documentation

Let’s create a buildout configuration file:

>>> rmdir(tempdir)
>>> mkdir(tempdir)
>>> cd(tempdir)
>>> a = [mkdir(d) for d in ('eggs', 'develop-eggs', 'bin', 'src')]
>>> install_develop_eggs(['minitage.recipe.scripts'])
>>> install_eggs_from_pathes(['zc.buildout'], sys.path)
>>> touch('buildout.cfg')
>>> sh('buildout -o bootstrap')
buildout -o bootstrap...
>>> index_url = start_server(os.path.sep.join(tempdir))

Initializing test env.

>>> if os.path.exists('foo'): rmdir(dl)
>>> mkdir('dl')
>>> if os.path.exists('foo'): rmdir(foo)
>>> mkdir('foo')
>>> mkdir('foo/src/toto')
>>> touch('foo/setup.py', data="""
... from setuptools import setup, find_packages
... setup(name='foo', version='1.0',
...     packages=find_packages('src'),
...     package_dir = {'': 'src'},
...     include_package_data=True,
...     scripts=['src/toto/toto.py'],
...     entry_points={'console_scripts': ['s=toto.toto:f']},
...     )
... """)
>>> touch('foo/src/toto/__init__.py')
>>> touch('foo/src/toto/toto.py', data="""
... def f():
...     print "foo"
... if __name__ == '__main__' :
...     print 'called'
...
... """)
>>> noecho = [os.remove(d) for d in os.listdir('.') if '.tar.gz' in d]
>>> os.chdir('foo')
>>> sh('python setup.py sdist')
p...
>>> noecho = [shutil.copy(os.path.join('dist', d), os.path.join('..', d)) for d in os.listdir('dist')]
>>> os.chdir('..')

Generating all scripts

Thus by not specifying any scripts entry in the buildout part.

>>> data = """
... [buildout]
... download-cache=${buildout:directory}/dl
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... eggs=foo
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> noecho = [remove(os.path.join('eggs', egg)) for egg in os.listdir('eggs') if 'foo' in egg]
>>> sh('bin/buildout -vvvvv install')
b...
minitage.recipe: Got foo 1.0.
minitage.recipe: Picked: foo = 1.0
minitage.recipe: All egg dependencies seem to be installed!
minitage.recipe: Generated scripts: 's', 'toto.py'...

Look at what have been generated.

>>> cat('bin', 'toto.py')
#!...
# ! GENERATED BY minitage.recipe !
import os
import sys
import subprocess...
sys.path[0:0] = ['/tmp/buildout.test/eggs/foo-1.0-py....egg' ]...
# EXEC ORGINAL CODE WITHOUT SHEBANG
__doc__  = 'I am generated by minitage.recipe.script recipe'...
os.environ['PYTHONPATH'] = ':'.join(sys.path + os.environ.get('PYTHONPATH', '').split(':'))
sys.argv.pop(0)
sys.exit(
    subprocess.Popen(
        [sys.executable, '/tmp/buildout.test/eggs/foo-1.0-py....egg/EGG-INFO/scripts/toto.py']+sys.argv,
        env=os.environ
    ).wait()
)...
>>> cat('bin', 's')
#!...
#!!! #GENERATED VIA MINITAGE.recipe !!!...
import sys...
sys.path[0:0] = [ '/tmp/buildout.test/eggs/foo-1.0-py....egg' ]...
import toto.toto...
if __name__ == '__main__':
    toto.toto.f()...

Selecting scripts to install

Installing only s.

>>> data = """
... [buildout]
... download-cache=${buildout:directory}/dl
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... scripts =
...     s
... eggs=foo
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install')
b...
minitage.recipe: Generated scripts: 's'....

Installing only toto.py.

>>> data = """
... [buildout]
... download-cache=${buildout:directory}/dl
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... scripts =
...     toto.py
... eggs=foo
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install')
b...
minitage.recipe: Generated scripts: 'toto.py'....

Declaring entry-points

We ll add an entry point ‘t’ to be generated.

>>> data = """
... [buildout]
... download-cache=${buildout:directory}/dl
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... entry-points=t=toto.toto:f
... eggs=foo
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install')
b...
minitage.recipe: Generated scripts: 't'....

Adding initialization code

What about adding environment variables for gis env.

>>> data = """
... [buildout]
... download-cache=${buildout:directory}/dl
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... entry-points=t=toto.toto:f
... eggs=foo
... initialization = import os;os.environ.set('GDAL', 'TRUE')
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install')
b...

>>> "import os;os.environ.set('GDAL', 'TRUE')" in open(os.path.join('bin', 't')).read()
True

Adding arguments

What about adding arguments to our launchers.

>>> data = """
... [buildout]
... download-cache=${buildout:directory}/dl
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... eggs = foo
... entry-points=t=toto.toto:f
... arguments = ['a', 'b']
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install')
b...

>>> "toto.toto.f(['a', 'b'])" in open(os.path.join('bin', 't')).read()
True

Generating a python interpreter

Here is how you can generate a specific python interpreter will all the environement of the working set.

>>> data = """
... [buildout]
... download-cache=${buildout:directory}/dl
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... interpreter = mypy
... arguments = ['a', 'b']
... eggs=foo
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install')
b...
minitage.recipe: Generated scripts: 'mypy'....

>>> cat('bin', 'mypy')
#!...
#!!! #GENERATED VIA MINITAGE.recipe !!!...
sys.path[0:0] = [ '/tmp/buildout.test/eggs/foo-1.0-py....egg' ]...
if _interactive:
    import code
    code.interact(banner="", local=globals())...

Generating an envrionment file

Here is how you can generate a specific envrionment file that you can source from to get the PYTHONPATH populated with eggs that you have configured.

>>> data = """
... [buildout]
... download-cache=${buildout:directory}/dl
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... env-file = mypy
... eggs=foo
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install')
b...
minitage.recipe: Generated scripts: '/tmp/buildout.test/bin/mypy'....

>>> cat('bin', 'mypy')
#!/usr/bin/env sh
<BLANKLINE>
PYTHONPATH="/tmp/buildout.test/eggs/foo-1.0-py....egg:$PYTHONPATH"
export PYTHONPATH
<BLANKLINE>
<BLANKLINE>

CHANGELOG

1.45

• better handle of entrypoints/console script conflict options (cs. prefix, see code)

• add per script options (initialization, argurments, python)

1.44

• when both EntryPoint and Script have the same name, it’s the EP which wins.

1.41

• fix a special case where all scripts were filtered out

1.39

• do not preerve PYTHONPATH on env.file generation

1.37

• Decorator helper

1.36

• fix develop link

1.35

• splitted out from minitage.recipe