Asynchronous and distributed scheduling of Ewoks workflows from python
Project description
ewoksjob
Utilities for job scheduling of ewoks workflows.
Ewoksjob provides an ewoks interface for asynchronous and distributed scheduling of ewoks from python.
Note that ewoksjob distributes the execution of workflows while ewoksdask distributes the execution of tasks in a workflow. So in the context of workflows, job scheduling exists on two levels.
The primary clients that need to schedule workflows are
- Ewoksserver: web backend for ewoks.
- Bliss: the ESRF beamline control system.
- Daiquiri: web backend for Bliss.
Installation
Install on the client side
pip install ewoksjob
Install on the worker side
pip install ewoksjob[worker]
Getting started
Start a worker pool that can execute ewoks graphs
ewoksjob worker
Start a workflow on the client side
from ewoksjob.client import submit
workflow = {"graph": {"id": "mygraph"}}
future = submit(args=(workflow,))
result = future.get()
Note that both environments need to be able to import celeryconfig
which
contains celery configuration (mainly the message broker and result backend URL's).
Hello world example
Clone the git repository and start a worker pool
scripts/worker.sh --sql
Submit workflows
scripts/runjobs.sh --sql
Tests
pytest --pyargs ewoksjob
To run the redis tests you need redis-server
(e.g. conda install redis-server
).
Documentation
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.