Celery Crossover aims to make it really easy to execute tasks in another service.
Project description
Celery Crossover (Alpha)
About
Celery crossover aims to make it easier to execute celery tasks from a diffent source code base in the most simple and yet reliable way.
Quick Examples
1) Simple Alice task execution triggered by Bob
Lets suppose Bob is a service that needs Alice to execute the task plus
defined on Alice with the following code:
Alice:
- celery_config.py
# -*- coding: utf-8 -*-
from kombu import Exchange, Queue
BROKER_URL = 'redis://localhost:6379/1'
CELERY_QUEUES = [
Queue('alice_queue', Exchange('alice_queue'), routing_key='alice_queue')
]
- alice.py
# -*- encoding: utf-8 -*-
import crossover
from celery import Celery
app = Celery('tasks')
app.config_from_object('celery_config')
# The line bellow will make Alice's Tasks usable by other services.
crossover.register_router(app)
@app.task(name='plus', queue='alice_queue')
def plus(x, y):
_add = x + y
return _add
Bob:
All that Bob need to do is:
- exec_task_on_alice.py
# -*- encoding: utf-8 -*-
from crossover import Client
alice_broker = "redis://localhost:6379/1"
alice_client = Client(broker=alice_broker)
alice_client.plus(x=340, y=210)
2) Alice task execution triggered by Bob with callback (Auto Callback)
In the same scenario described above, lets suppose Bob need to be notified (have a task executed) after Alice is done with the plus
task.
For this case, all we have to do is decorate the Alice's task plus
with@crossover.callback(auto_callback=True)
to have its returned value sent back to Bob. Also, Bob have to define and send to Alice a task to be called by Alice's callback.
That way, Alice and Bob could would be:
Alice:
- alice_celery_config.py
# -*- coding: utf-8 -*-
from kombu import Exchange, Queue
BROKER_URL = 'redis://localhost:6379/1'
CELERY_QUEUES = [
Queue('alice_queue', Exchange('alice_queue'), routing_key='alice_queue')
]
- alice.py
# -*- encoding: utf-8 -*-
import crossover
from celery import Celery
app = Celery('tasks')
app.config_from_object('alice_celery_config')
# The line bellow will make Alice's Tasks usable by other services.
crossover.register_router(app)
@app.task(name='plus', queue='alice_queue')
@crossover.callback(auto_callback=True)
def plus(x, y):
_add = x + y
return _add
Bob:
- bob_celery_conf.py
# -*- coding: utf-8 -*-
from kombu import Exchange, Queue
BROKER_URL = 'redis://localhost:6379/1'
CELERY_QUEUES = [
Queue('bob_queue', Exchange('bob_queue'), routing_key='bob_queue')
]
- bob.py
# -*- encoding: utf-8 -*-
import crossover
from celery import Celery
from celery.utils.log import get_task_logger
logger = get_task_logger(__name__)
app = Celery('tasks')
app.config_from_object('bob_celery_conf')
crossover.register_router(app)
@app.task(name='plus_callback', queue='bob_queue')
def plus_callback(result):
logger.info('Got Addition callback = {0}'.format(result))
- exec_task_on_alice.py
# -*- encoding: utf-8 -*-
from crossover import Client
from bob import plus_callback
alice_broker = "redis://localhost:6379/1"
alice_client = Client(broker=alice_broker)
alice_client.plus(x=340, y=210, callback=plus_callback)
3) Alice task execution triggered by Bob with callback (No Auto Callback)
In this case, everything is the same as 2) from Bob's perspective but lets suppose Alice's task cant calculate or
determine a response right away so it needs to pass (or persist) the callback metadata for further execution.
It can be done by using de decorator @crossover.callback
with bind_callback_meta=True
which will give
to the task function the callback metadata as its first parameters. Following an example of its usage:
- alice.py
# -*- encoding: utf-8 -*-
...
@app.task(name='plus', queue='alice_queue')
@crossover.callback(bind_callback_meta=True)
def plus(callback_meta, x, y):
logger.info('Execution actual addition task.')
calculate_addition.delay(callback_meta, x, y):
@app.task(name='calculate_addition', queue='project_1')
def calculate_addition(callback_meta, x, y):
_add = x + y
logger.info('Addition {0} + {1} = {2}'.format(x, y, _add))
crossover.CallBack(callback_meta)(result=_add)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for celery_crossover-1.1.16-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 19378f8d57aeac13c93ba4be4f1ec5341353af355913fc7eae2bd94dcf436da5 |
|
MD5 | 54ad2b34cf83ad7961516547ae1329e2 |
|
BLAKE2b-256 | c76ebf941187e1ab1791649837c3aeadbe4a88eaa8f7e7b5d4e9d10523fac658 |