Skip to main content
This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

A Buildout recipe to install and configure Celery for Birdhouse.

Project Description

Introduction

birdhousebuilder.recipe.celery is a Buildout recipe to install and configure Celery Distributed Task Queue with Anaconda. This recipe is used by the Birdhouse project.

The recipe is based on collective.recipe.celery.

Usage

The recipe requires that Anaconda is already installed. You can use the buildout option anaconda-home to set the prefix for the anaconda installation. Otherwise the environment variable CONDA_PREFIX (variable is set when activating a conda environment) is used as conda prefix.

The recipe will install the celery package from a conda channel in a conda enviroment defined by CONDA_PREFIX. It deploys a Supervisor configuration for Celery in {{prefix}}/etc/supervisor/conf.d/celery.conf. Supervisor can be started with {{prefix}}/etc/init.d/supervisord start.

The recipe depends on birdhousebuilder.recipe.conda and birdhousebuilder.recipe.supervisor.

Supported options

anaconda-home
Buildout option pointing to the root folder of the Anaconda installation. Default: $HOME/anaconda.

Buildout options for celery:

app
The application instance to use for the celery worker.
eggs
A list of additional eggs you want to make available to Celery. Use this to add additional dependencies and the module(s) containing your task definitions.
use-celeryconfig
Generate and use the celeryconfig.py. Set to false if Celery is configured e.a. by pyramid_celery. Default: true.

Celery configuration options

The following configuration options are supported. See Celery documentation for more details.

broker-url
The url of the broker. Default: redis://localhost:6379/0
celery-result-backend
The url of the backend used to store task results. Default: redis://localhost:6379/0
celery-imports
List of modules to import when celery starts.
celeryd-concurrency
The number of concurrent worker processes/threads/green threads executing tasks. Defaults to the number of available CPUs.
loglevel
Logging level, choose between DEBUG, INFO, WARNING, ERROR, CRITICAL, or FATAL. Default: WARNING

Example usage

The following example buildout.cfg installs Celery with Anaconda and default options:

[buildout]
parts = celery

anaconda-home = /home/myself/anaconda

[celery]
recipe = birdhousebuilder.recipe.celery
app = mystuff_app
eggs = mysuff

The next example shows a configuration with pyramid_celery and enabled Flower monitoring:

[buildout]
parts = celery

anaconda-home = /home/myself/anaconda

[celery]
recipe = birdhousebuilder.recipe.celery
app = pyramid_celery.celery_app --ini default.ini
eggs = mysuff
use-celeryconfig = false

Authors

Carsten Ehbrecht ehbrecht at dkrz.de

Changes

0.2.0 (2016-07-04)

  • using bool_option from zc.buildout.
  • using zc.recipe.deployment.
  • removed flower.

0.1.3 (2016-06-13)

  • added update-conda option.
  • use flower, redis and pymongo only if necessary.

0.1.2 (2016-06-03)

  • added CELERY_MONGODB_BACKEND_SETTINGS config option

0.1.1 (2015-12-10)

  • Renamed celeryconfig.py template.

0.1.0 (2015-12-10)

  • Initial Release.
Release History

Release History

This version
History Node

0.2.0

History Node

0.1.3

History Node

0.1.2

History Node

0.1.1

History Node

0.1.0

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
birdhousebuilder.recipe.celery-0.2.0.tar.gz (7.1 kB) Copy SHA256 Checksum SHA256 Source Jul 4, 2016

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting