worker-dispatcher 0.0.2

Developing

Python Worker Dispatcher

A flexible task dispatcher for Python with multiple threading or processing control

Features

• Tasks Dispatching to managed worker

• Elegant Interface for setup and use

---

OUTLINE

• Demonstration
• Introduction
• Requirements
• Installation
• Usage
  • Option
    • callbacks.process
    • callbacks.task

---

DEMONSTRATION

Use 20 theads concurrently to dispatch tasks for HTTP reqeusts

import worker_dispatcher
import requests

def each_task(config, task_id: int, task_data):
    response = requests.get(config['my_endpoint'] + task_data)
    return response

responses = worker_dispatcher.start({
    'task': {
        'list': ['ORD_AH001', 'ORD_KL502', '...' , 'ORD_GR393'],
        'callback': each_task,
        'config': {
            'my_endpoint': 'https://your.name/order-handler/'
        },
    },
    'worker': {
        'number': 20,
    }
})

Utilizes all CPU cores on the machine to compute tasks.

import worker_dispatcher

def each_task(config, task_id: int, task_data):
    result = sum(task_id * i for i in range(10**9))
    return result

if __name__ == '__main__':
    results = worker_dispatcher.start({
        'task': {
            'list': 10,
            'callback': each_task,
        },
        'worker': {   
            'multiprocessing': True
        }
    })

---