A concurrent.futures.Executor implementation using Celery as backend
concurrent.futures.Executor implementation using Celery as backend
- Free software: Apache Software License 2.0
- Documentation: https://celery-executor.readthedocs.io.
The package provides a
CeleryExecutor implementing the interface of
>>> from celery_executor.executors import CeleryExecutor >>> executor = CeleryExecutor() >>> for result in executor.map(str.upper, ['one', 'two', 'three']): ... print(result) ONE TWO THREE
Beware that the
Executor.map() interface can yield the results out of order,
if later ones got to finish first.
This executor frees the developer to the burden of mark every single task function with the Celery decorators, and to import such tasks on the Worker beforehand. But does not frees from sending the code to the Worker.
The function sent to
CeleryExecutor.map() should be pickable on the client
.submit()) and should be unpickable on the Celery
Worker handling the "Task" sent. Is not possible to send lambdas for example.
As Celery assumes that is to the developer to put the needed code on the Worker,
be sure that the function/partial code sent to
CeleryExecutor to exist on the
To Be Done
- Document the
CeleryExecutor.__init__()nonstandard extra options
- Test behaviours of canceling a Task when canceling a Future
- Test behaviours of shutting down executors and trying to send new tasks
- Find a way to test the RUNNING state of Celery Tasks, as its events are not propagated by the test worker Celery provides
- First release on PyPI.
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Hashes for celery_executor-0.2.0-py2.py3-none-any.whl