cwl-airflow-parser

About

Extends Apache-Airflow with CWL v1.0 support.

Installation

pip3.6 install -U cwl-airflow-parser

Requirements

Package has been tested on Ubuntu 16.04.3 and Mac OS X Sierra/ High Sierra. Make sure your system satisfies the following criteria:

python 3.6
docker
nodejs

Usage

from cwl_airflow_parser import CWLDAG, CWLJobDispatcher, CWLJobGatherer
from datetime import timedelta

def cwl_workflow(workflow_file):
    dag = CWLDAG(default_args={
        'owner': 'airflow',
        'email': ['my@email.com'],
        'email_on_failure': False,
        'email_on_retry': False,
        'retries': 20,
        'retry_exponential_backoff': True,
        'retry_delay': timedelta(minutes=30),
        'max_retry_delay': timedelta(minutes=60 * 4)
    },
        cwl_workflow=workflow_file)
    dag.create()
    dag.add(CWLJobDispatcher(dag=dag), to='top')
    dag.add(CWLJobGatherer(dag=dag), to='bottom')

    return dag
 
cwl_workflow("/path/to/my/workflow.cwl")

Metadata

Release files for cwl-airflow-parser 1.0.20181213213631

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for cwl-airflow-parser 1.0.20181213213631
File	Size	Uploaded
cwl-airflow-parser-1.0.20181213213631.tar.gz	13.7 kB	Dec 13, 2018	Details

Release files / cwl-airflow-parser-1.0.20181213213631.tar.gz

Download URL	cwl-airflow-parser-1.0.20181213213631.tar.gz
Size	13.7 kB
Tags	Source
SHA-256 checksum How to use checksums	`b4c3e6ac4cd86dbe2f0d7fda4c608e580f018c3608f01611713daff1250ebe25`
BLAKE2b-256 checksum How to use checksums	`06f79baaec92b2a85930e12c140e24acd41a6dd751609b07e95b0240c3d21404`
Upload date	Dec 13, 2018
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/1.11.0 pkginfo/1.4.2 requests/2.19.1 setuptools/40.0.0 requests-toolbelt/0.8.0 tqdm/4.26.0 CPython/3.6.5`