Skip to main content

SIP Processing Block Controller library.

Project description

SKA SDP SIP Processing Block Controller

Library and Celery worker image for executing Processing Block workflows within the SDP SIP prototype code. This is implemented as a set of Celery tasks which can be used to asynchronously execute SIP Processing Block workflows on a Processing Block Controller (Celery Worker) instance.

Library installation

pip install skasip-pbc

Starting a containerised PBC instance

For the Processing Block Controller to function, a Redis Database instance must be created, which provides the role of the Celery Broker and Execution Control Configuration Database. Once this is available, a Processing Block Controller (Celery Worker) can be started.

For the Redis Databasem a standard redis container can be started (eg. using the official redis:5.0.1-alpine image). The Processing Block Controller Celery worker can be started using the image skasip/processing_block_controller.

A Docker Compose file ( is provided in this folder for starting a Redis instance and a PBC Celery worker. This can be used with the following command:

docker stack deploy -c pbc

Typical usage

Note: Requires a Celery Worker, Celery Broker, and a SKA SIP Execution Control Configuration Database instance are running (see above)!

# coding: utf-8
"""Example SIP PBC usage."""
import sip_pbc

# Get the PBC version.
result = sip_pbc.version.delay()
version = result.get(timeout=1)

# Execute a PB workflow.
result = sip_pbc.execute_processing_block.delay(pb_id='...')

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

skasip-pbc-1.3.0.tar.gz (5.3 kB view hashes)

Uploaded source

Built Distribution

skasip_pbc-1.3.0-py3-none-any.whl (6.8 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page