Skip to main content

An easy-to-use Python API for Pegasus 5.0

Project description

1. Pegasus Workflow Management System Python3 API

Pegaflow, https://github.com/polyactis/pegaflow, is an easy-to-use package of the Python3 APIs for Pegasus WMS (http://pegasus.isi.edu/). It is compatible with Pegasus 5.0.0. Pegasus allows a developer to connect dependent computing jobs into a DAG (Directed Acyclic Graph) and run jobs according to the dependency.

Workflow.py is the key difference from the official Pegasus Python APIs. Inheriting Workflow.py, users can write Pegasus workflows in an Object-Oriented way. It significantly reduces the amount of coding in writing a Pegasus workflow.

Pegasus jobs do NOT support UNIX pipes while many UNIX programs can only output to stdout. A shell wrapper, pegaflow/tools/pipe2File.sh, is offered to redirect the output (stdout) of a program to a file. pegaflow/tools/ contains a few other useful shell scripts.

  • Workflow.py
  • The monitoring API
  • The Stampede database API
  • The Pegasus statistics API
  • The Pegasus plots API
  • Miscellaneous Pegasus utilities
  • The Pegasus service, including the ensemble manager and dashboard

Part of this package's source code is copied from https://github.com/pegasus-isi/pegasus, version 5.0.0,

2. Prerequisites

Pegasus and HTCondor (Condor) are only required on computers on which you intend to submit and run workflows.

On computers where only DAG yml files are outputted, no need to install Pegasus and Condor.

3. Installation

Install pegaflow:

pip3 install --upgrade pegaflow

4. Examples

Check pegaflow/example/ for examples.

20210616 Upgraded to be compatible with Pegasus 5.0.0.

20200326 An easy-to-use Python3 package for Pegasus-wms API

Pegaflow is a package of the Python3 APIs for Pegasus WMS (http://pegasus.isi.edu/), with a helper class Workflow.py.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for Pegaflow, version 5.0.4
Filename, size File type Python version Upload date Hashes
Filename, size Pegaflow-5.0.4-py3-none-any.whl (613.4 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size Pegaflow-5.0.4.tar.gz (501.4 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page