appfy.recipe.gae 0.9.10

Buildout recipes for App Engine development.

appfy.recipe.gae

appfy.recipe.gae provides a series of zc.buildout recipes to help with Google App Engine development. It is inspired by rod.recipe.appengine, but using a different layout and with extended functionalities. It is also split in different recipes. Currently appfy.recipe.gae has 3 recipes:

appfy.recipe.gae:app_lib:

Downloads libraries from PyPi and installs in the app directory.

appfy.recipe.gae:sdk:

Downloads and installs the App Engine SDK.

appfy.recipe.gae:tools:

Installs a python executable and several SDK scripts in the buildout directory: appcfg, bulkload_client, bulkloader, dev_appserver and remote_api_shell. It also allows to set default values to start the dev_appserver.

Source code and issue tracker can be found at https://github.com/prmtl/appfy.recipe.gae.

For an example of how appfy makes distribution of App Engine apps easy and nice, see Moe installation instructions.

appfy.recipe.gae:app_lib

Downloads libraries from PyPi and installs in the app directory. This recipe extends zc.recipe.egg.Scripts, so all the options from that recipe are also valid.

Options

eggs:

Package names to be installed.

lib-directory:

Destination directory for the libraries. Default is distlib.

use-zipimport:

If true, a zip file with the libraries is created instead of a directory. The zip filename will be the value of lib-directory plus .zip.

ignore-globs:

A list of glob patterns to not be copied from the library.

ignore-packages:

A list of top-level package names or modules to be ignored. This is useful to ignore dependencies that won’t be used. Some packages may install distribute, setuptools or pkg_resources but these are not very useful on App Engine, so you can set them to be ignored, for example.

delete-safe:

If true, always move lib-directory to a temporary directory inside the parts dir as a backup when building, instead of deleting it. This is to avoid accidental deletion if lib-directory is badly configured. Default to true.

Example

[app_lib]
# Sets the library dependencies for the app.
recipe = appfy.recipe.gae:app_lib
lib-directory = app/distlib
use-zipimport = false

# Define the libraries.
eggs =
    tipfy

# Don't copy files that match these glob patterns.
ignore-globs =
    *.c
    *.pyc
    *.pyo
    */test
    */tests
    */testsuite
    */django
    */sqlalchemy

# Don't install these packages or modules.
ignore-packages =
    distribute
    setuptools
    easy_install
    site
    pkg_resources

appfy.recipe.gae:sdk

Downloads and installs the App Engine SDK in the buildout directory.

Options

url:

URL to the App Engine SDK file. Default is to download the latest version from storage.googleapis.com.

destination:

Destination of the extracted SDK. Default is the parts directory.

clear-destination:

If true, deletes the destination dir before extracting the download. Default is true.

Example

[gae_sdk]
# Dowloads and extracts the App Engine SDK.
recipe = appfy.recipe.gae:sdk
url = http://googleappengine.googlecode.com/files/google_appengine_1.3.5.zip
destination = ${buildout:parts-directory}
hash-name = false
clear-destination = true

appfy.recipe.gae:tools

Installs a python executable and several SDK scripts in the buildout directory: appcfg, bulkload_client, bulkloader, dev_appserver and remote_api_shell.

It also allows to set default values to start the dev_appserver.

This recipe extends zc.recipe.egg.Scripts, so all the options from that recipe are also valid.

Options

sdk-directory:

Path to the App Engine SDK directory. It can be an absolute path or a reference to the appfy.recipe.gae:sdk destination option. Default is ${buildout:parts-directory}/google_appengine.

appcfg-script:

Name of the appcfg script to be installed in the bin directory.. Default is appcfg.

bulkload_client-script:

Name of the bulkloader script to be installed in the bin directory. Default is bulkload_client.

bulkloader-script:

Name of the bulkloader script to be installed in the bin directory. Default is bulkloader.

dev_appserver-script:

Name of the dev_appserver script to be installed in the bin directory. Default is dev_appserver.

remote_api_shell-script:

Name of the remote_api_shell script to be installed in the bin directory. Default is remote_api_shell.

config-file:

Configuration file with the default values to use in scripts. Default is gaetools.cfg.

extra-paths:

Extra paths to include in sys.path for generated scripts.

initialization:

Allows to specify some Python code to be included in the scripts.

Example

[gae_tools]
# Installs appcfg, dev_appserver and python executables in the bin directory.
recipe = appfy.recipe.gae:tools
sdk-directory = ${gae_sdk:destination}/google_appengine
# add extra code
initialization =
  import dev_appserver
  dev_appserver.fix_sys_path()
# Add these paths to sys.path in the generated scripts.
extra-paths =
    app/lib
    app/distlib

Note that this example references an gae_sdk section from the appfy.recipe.gae:sdk example. An absolute path could also be used.

To set default values to start the dev_appserver, create a section dev_appserver in the defined configuration file (gaetools.cfg by default). For example:

[dev_appserver]
# Set default values to start the dev_appserver. All options from the
# command line are allowed. They are inserted at the beginning of the
# arguments. Values are used as they are; don't use variables here.
recipe = appfy.recipe.gae:tools
defaults =
    --datastore_path=var/data.store
    --history_path=var/history.store
    --blobstore_path=var/blob.store
    app

Each option should be set in a separate line, as displayed above. Options provided when calling dev_appserver will override the default values.

Versions

Version 0.9.10 - February 21, 2015

• Wait to check for an empty sdk url until install time (Joshua Johnston)

Version 0.9.9 - November 20, 2014

• Fix problem with 401 error code when fetching latest SDK

Version 0.9.8 - October 26, 2014

• Fetch the latest SDK version if ‘url’ is not specified (Oliver Brook)

• PEP8 and code style checks added

Version 0.9.7 - July 22, 2014

• Added fixes to allow run multiple yaml files

Version 0.9.6 - June 02, 2014

• Added endpointscfg script needed to work with Google Cloud Endpoints

Version 0.9.5 - May 11, 2014

• Append any initialization options to the generated initialization output (Josh Johnston)

Version 0.9.4 - December 01, 2013

• Moved package to Github

• Moved from z3c.recipe.egg to zc.recipe.egg

Version 0.9.3 - December 03, 2010

• Changed requirements:

  zc.buildout >= 1.5.2
  z3c.recipe.scripts >= 1.0.1
  zc.recipe.egg >= 1.3.2

Version 0.9.1 - November 27, 2010

• Readded option ‘clear-destination’ to the dowanlod recipe, true by default. It was causing an error when the files existed, and ‘ignore-existing’ wasn’t avoiding it.

Version 0.9 - November 23, 2010

• Scripts now extends z3c.recipe.scripts.scripts.Scripts, for better compatibility with buildout 1.5.2.

• appfy.recipe.gae.sdk accepts a sha1sum option, to check the SDK checksum as provided by Google.

• removed hexagonit.recipe.download as it was not flexible enough to allow the sha1 checksum check.

Version 0.8 - July 27, 2010

• Do not raise IOError when egg info is not found, and let installation proceed only emitting a warning.

Version 0.7.2 - June 18, 2010

• os.makedirs(), not os.mkdirs(). Ooops.

Version 0.7.1 - June 18, 2010

• Minor enhancement: use os.mkdirs() instead of os.mkdir() when creating the backup for app_libs (Tom Lynn).

Version 0.7 - June 17, 2010

• Added multi-line top_level support. Now it can handle eggs with multiple lines in top_level.txt. Thanks to Benjamin Kampmann for this (Issue #3).

• Added ignore-packages option, useful to ignore setuptools, distribute and other dependency packages not useful on App Engine.

• Single modules are now also matched by ignore-globs.

• ignore-globs now removes the non-related path prefix for better matching.

• Documented extra-paths, useful to add libraries directories to sys.path in scripts.

• Several refactorings and cleanups.

Version 0.6.1 - June 3, 2010

• Don’t install package if egg info is not found, instead of breaking. This was causing a problem when setuptools is declared as dependency.

Version 0.6 - June 1, 2010

• app_lib can now also install develop eggs.

Version 0.5.2 - May 27, 2010

• Single files are correctly installed.

• Namespaced packages are put in the same directory structure. This was causing an error when trying to create a directory for the second time.

Version 0.5.1 - May 17, 2010

• appfy.recipe.gae:app_lib now extends zc.recipe.egg.Scripts, so that scripts from packages are installed, as before.

Version 0.5 - May 5, 2010

• Dropped checksum checking, and now move files to a backup directory if delete-safe is true (which is the default). This makes the build faster and avoids the annoying invalid checksum errors.

Version 0.4.5 - May 5, 2010

• Use tempfile.gettempdir() to save temporary files. Thanks, tlynn.

Version 0.4.4 - May 3, 2010

• Unzip eggs by default in app_lib or we can’t use some.

Version 0.4.3 - May 3, 2010

• Only accepts multi-line configuration for gaetools.

• Fixed tmpdir in app_lib.

Version 0.4.2 - May 1st, 2010

• app_lib now extends zc.recipe.egg.Eggs and sets default eggs to an empty string, just to avoid errors.

• More small refactorings.

Version 0.4.1 - May 1st, 2010

• Removed primary-lib-directory option from app_lib.

• Code cleanup and refactoring.

Version 0.4 - April 30, 2010

• Fixed script path problem when buildout is configured to use absolute paths. Fixes issue #1. Thanks Lacrima.Maxim for the report.

• Scripts now run using alter_sys=True, so that help messages that use __doc__ are displayed correctly.

Version 0.3 - April 29, 2010

• Added config-file option to appfy.recipe.gae:tools, to define the configuration file used to set default arguments for the scripts.

Version 0.2 - April 29, 2010

• Added bulkload_client, bulkloader and remote_api_shell scripts.

• Fixed script entry-points: they were breaking on Windows; now they work.

• All recipes are also tested and working on Windows now.

Version 0.1 - April 28, 2010

Initial release