ICIJ's async worker library 
Installation
With AMQP:
pip install icij-worker["amqp"]
With neo4j:
pip install icij-worker["neo4j"]
Usage
Create an async app and register tasks
Create asynchronous task tailored for long running Python functions:
Given the following pure Python function inside the app.py module:
def long_running_task(greeted: str) -> str:
greeting = f"Hello {greeted} !"
return greeting
decorate your function with ICIJApp class and register a new task:
from icij_worker import AsyncApp
my_app = AsyncApp(name="my_app")
@my_app.task
def long_running_task(greeted: str) -> str:
greeting = f"Hello {greeted} !"
return greeting
this will register the long_running_task function under the long_running_task task name.
Optionally add progress handlers for a better task monitoring:
@my_app.task
async def long_running_task(
greeted: str,
progress: Optional[Callable[[float], Awaitable]] = None
) -> str:
if progress is not None:
await progress(0.0)
greeting = f"Hello {greeted} !"
if progress is not None:
await progress(100.0)
return greeting
Launch a async worker pool
Start a worker pool using:
icij-worker workers start "app.my_app"
provide worker pool options using:
icij-worker workers start -c worker_config.json -n 2 --backend multiprocessing "app.my_app"
depending on the worker configuration additional setup might be required.
Async worker implementations
Worker asynchronous backends
Implemented
- multiprocessing for CPU bound tasks
To be implemented
- asyncio for I/O bound tasks
4 maintainers
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file icij_worker-0.20.1.tar.gz.
File metadata
- Download URL: icij_worker-0.20.1.tar.gz
- Upload date:
- Size: 59.8 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
uv/0.6.7
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
7f5c5dfde79cd9dd08157845d9e16d505cfb71c0555771435ada5f58c77455a0
|
|
| MD5 |
a00f431402b1f1c57fd911c60ee93086
|
|
| BLAKE2b-256 |
521745f7653a6a3c8bd522471534fb73c3141ed4efd7bf6dab2c83731b7a53da
|
File details
Details for the file icij_worker-0.20.1-py3-none-any.whl.
File metadata
- Download URL: icij_worker-0.20.1-py3-none-any.whl
- Upload date:
- Size: 85.3 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
uv/0.6.7
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
d5725819f3baf80ea162c5f8f37e8ee39d2727b962f38340539a74fd550ee4c4
|
|
| MD5 |
a69b2e0f1cede1041d412de1f3008af7
|
|
| BLAKE2b-256 |
bbc77c345ef04d568a3e0b32c1a8257d12098a8a30ac8ea0c52cefe4db0da42d
|
