Skip to main content

Generic worker that performs concurrent tasks using coroutine.

Project description

coworker

Generic worker that performs concurrent tasks using coroutine.

Quick Start Tutorial

Define how a task is performed and create the worker:

from coworker import Coworker

class SquareWorker(Coworker):
    async def do_task(self, task):
        return task * task

worker = SquareWorker(max_concurrency=5)    # Only 5 tasks will run concurrently
                                            # As do_task is fast, 35,000 tasks can be done in 1 second.

To run in the background forever and add tasks:

import asyncio

async def background_worker_example():
    # Start worker / Run in background
    asyncio.ensure_future(worker.start())

    # Mulitiple tasks
    tasks = list(range(100))
    results = await asyncio.gather(*worker.add_tasks(tasks))
    print(results)  # results = [0, 1, 4, 9, ...]

    # Single task
    result = await worker.add_tasks(2)
    print(result)   # result = 4

    # Stop worker
    await worker.stop()

# Run async usage example
asyncio.get_event_loop().run_until_complete(background_worker_example())

To run for a list of tasks and stop worker when finished:

task_futures = asyncio.get_event_loop().run_until_complete(worker.start([1, 2, 3]))
print([t.result() for t in task_futures])   # [1, 4, 9]

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
coworker-1.0.1-py3-none-any.whl (4.4 kB) Copy SHA256 hash SHA256 Wheel py3
coworker-1.0.1.tar.gz (13.1 kB) Copy SHA256 hash SHA256 Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page