Easy configuration for celery app using Yaml file
Project description
Easy Configuration For Celery App Using a Yaml File
celery-yaml
is a library to inject a --yaml option to the celery worker
command in order to inject its configuration.
It also handle help to configurate this application for Pyramid application.
Usage
With Celery 4
celery worker -A my_application.module_containing_my.app --yaml development.yaml ...
With Celery 5
celery -A my_application.module_containing_my.app worker --yaml development.yaml ...
This will configure the application my_application
containing an application
app
in a submodule module_containing_my
.
The celery app must register the --yaml
using the add_yaml_option
on the
app instance, this way:
from celery import Celery
from celery_yaml import add_yaml_option
app = Celery()
add_yaml_option(app)
Yaml format
celery:
broker_url: 'amqp://guest:guest@localhost:5672//'
result_backend: 'rpc://'
imports:
- my_application.tasks
# see all settings in the celery docs:
# https://docs.celeryproject.org/en/stable/userguide/configuration.html
logging:
version: 1
# dictConfig format
# https://docs.python.org/3/library/logging.config.html#logging-config-dictschema
NOTE
The broker_url can also be override by an environment variable CELERY_BROKER_URL
to avoid password in the configuration file.
Using Celery in a Pyramid App.
The extras "pyramid" must be added to install the extras depencencies.
With poetry
[tool.poetry.dependencies]
celery-yaml = { version = "^0.1.3", extras = ["pyramid"] }
Then some entry_points have to configure, such as:
[tool.poetry.plugins."paste.app_factory"]
main = "pyramid_app.wsgi:main"
[tool.poetry.plugins."celery_yaml.app"]
main = "pyramid_app.backend:app"
[tool.poetry.plugins."plaster.loader_factory"]
"file+yaml" = "plaster_yaml:Loader"
the paste.app_factory
is used by Pyramid
itself to build the WSGI
sergivice but we add a plaster.loader_factory
to configure the usage
of a yaml file instead of an ini
file to configure it.
Then the celery_yaml.app
is used by celery-yaml
as an entrypoint to
the celery app.
Then, in the configuration file,
celery: &celery
result_backend: 'rpc://'
imports:
- pyramid_app.tasks
app:
"use": "egg:pyramid_app"
"pyramid.includes": ["celery_yaml"]
"celery":
<<: *celery
"use": "egg:pyramid_app"
More configuration
if the celery app as a method on_yaml_loaded
then the function
is called with the data and the filepath in parameter.
It may be used to get some config.
Example
from celery import Celery as CeleryBase
class Celery(CeleryBase):
def on_yaml_loaded(self, data: Dict[str, Any], config_path: str):
...
See full example in the examples directory:
Pyramid with Celery 4
https://github.com/mardiros/celery-yaml/tree/master/examples/pyramid-app
Pyramid with Celery 5
https://github.com/mardiros/celery-yaml/tree/master/examples/pyramid-app5
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for celery_yaml-1.0.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 84f4776a019879eea472481ef76f28bf2a8cabbd0ed0b9caa129b3507de23976 |
|
MD5 | bea43c9e615f4f73f1b3ee6943c9bd80 |
|
BLAKE2b-256 | be94f380e4f638cadd888169f5cd4369ce01082fefa17fd411522468b3bfc7bb |