Skip to main content

Create asynchronous tasks from Python functions

Project description

Test for icij-worker

ICIJ's async worker library


pip install icij-worker


Create an async app and register tasks

Create asynchronous task tailored for long running Python functions:

Given the following pure Python function inside the module:

def long_running_task(greeted: str) -> str:
    greeting = f"Hello {greeted} !"
    return greeting

decorate your function with ICIJApp class and register a new task:

my_app = ICIJApp(name="my_app")

def long_running_task(greeted: str) -> str:
    greeting = f"Hello {greeted} !"
    return greeting

this will register the long_running_task function under the long_running_task task name.

Optionally add progress handlers for a better task monitoring:

def long_running_task(greeted: str, progress_handler: Callable[[float], None]) -> str:
    greeting = f"Hello {greeted} !"
    return greeting

Launch a async worker pool

Start a worker pool using:

icij-worker workers start "app.my_app"

provide worker pool options using:

icij-worker workers start -c worker_config.json -n 2 --backend multiprocessing "app.my_app"

depending on the worker configuration additional setup might be required.

Async worker implementations


In progress

  • AMQP

Worker asynchronous backends


To be implemented

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

icij_worker-0.1.3.tar.gz (26.3 kB view hashes)

Uploaded Source

Built Distribution

icij_worker-0.1.3-py3-none-any.whl (35.9 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page