Skip to main content
Help us improve PyPI by participating in user testing. All experience levels needed!

Library that provides a system to generate tasks producers and consumers with ease.

Project description

build status coverage

Version:1.4.5
Status:Production/Stable
Author:José Antonio Perdiguero López

Library that provides a system to generate tasks producers and consumers with ease.

Celery is used as backend, so it’s necessary to configure it in the application where this library will be used.

To achieve a producer-consumer behavior this library provides an easy to use script for running all necessary processes:

  • Producer: Tasks in charge of generating processing units.
  • Consumer: Tasks that handle and run processing units.
  • Scheduler: Manager that runs producer tasks according to specified dates or regularity.

Quick start

  1. Install this package using pip:
pip install task-dispatcher
  1. Decorate your functions as producer and consumer tasks:
from task_dispatcher import consumer, producer


@consumer
def square(x):
    return x**2


@producer
def prod_function(n):
    for i in range(n):
        square.delay(i)
  1. Run producer, consumer and scheduler processes:
python task-dispatcher producer
python task-dispatcher consumer
python task-dispatcher scheduler

Consumer and Producer

This library provides convenient decorators for generating a task dispatcher system based on producer-consumer pattern. Decorated functions or methods acts as Celery tasks and can be called using his own syntax: Calling celery tasks. Also, it’s possible to compose these tasks using Celery Canvas.

Register

Consumer and producer tasks are registered to ease the way of access them. There is a register module that contains the task register where all tasks can be found:

from task_dispatcher import register

# Get consumers
register.consumers

# Get produces
register.producers

Also, this register provides a set of utilities, such as convert it into JSON or YAML format:

from task_dispatcher import register

yaml_register = register.to_yaml()
json_register = register.to_json()

Command Line Interface

There is a script that can be called directly through executing the task_dispatcher package itself or the command located in commands module. To show command help:

python task-dispatcher -h

This script also gives a friendly way of list all tasks registered:

python task-dispatcher list

Django

This library can be imported and used as a Django application instead of a plain library, so that the CLI script also acts as a Django command.

Settings

Celery settings can be specified through TASK_DISPATCHER_SETTINGS variable using path format indicated in Celery application configuration.

Project details


Release history Release notifications

This version
History Node

1.4.5

History Node

1.4.4

History Node

1.4.3

History Node

1.4.2

History Node

1.4.1

History Node

1.4.0

History Node

1.3.0

History Node

1.2.2

History Node

1.2.1

History Node

1.2.0

History Node

1.1.4

History Node

1.1.3

History Node

1.1.2

History Node

1.1.1

History Node

1.1.0

History Node

1.0.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
task_dispatcher-1.4.5-py2.py3-none-any.whl (13.7 kB) Copy SHA256 hash SHA256 Wheel py2.py3 Apr 24, 2018
task-dispatcher-1.4.5.tar.gz (9.4 kB) Copy SHA256 hash SHA256 Source None Apr 24, 2018

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page