Skip to main content

Package extends Airflow functionality with CWL v1.0 support

Project description

# cwl-airflow-parser

### About
Extends **[Apache-Airflow](https://github.com/apache/incubator-airflow)** with **[CWL v1.0](http://www.commonwl.org/v1.0/)** support.

### Installation

```sh
pip3.6 install -U cwl-airflow-parser
```

### Requirements
Package has been tested on Ubuntu 16.04.3 and Mac OS X Sierra/ High Sierra. Make sure your system satisfies the following criteria:
- python 3.6
- [docker](https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/)
- nodejs

### Usage

```python
from cwl_airflow_parser import CWLDAG, CWLJobDispatcher, CWLJobGatherer
from datetime import timedelta

def cwl_workflow(workflow_file):
dag = CWLDAG(default_args={
'owner': 'airflow',
'email': ['my@email.com'],
'email_on_failure': False,
'email_on_retry': False,
'retries': 20,
'retry_exponential_backoff': True,
'retry_delay': timedelta(minutes=30),
'max_retry_delay': timedelta(minutes=60 * 4)
},
cwl_workflow=workflow_file)
dag.create()
dag.add(CWLJobDispatcher(dag=dag), to='top')
dag.add(CWLJobGatherer(dag=dag), to='bottom')

return dag

cwl_workflow("/path/to/my/workflow.cwl")
```

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

cwl-airflow-parser-1.0.20180622010727.tar.gz (9.2 kB view details)

Uploaded Source

File details

Details for the file cwl-airflow-parser-1.0.20180622010727.tar.gz.

File metadata

File hashes

Hashes for cwl-airflow-parser-1.0.20180622010727.tar.gz
Algorithm Hash digest
SHA256 3f08233750e34fb5bbb2ef9203314ad0221c26117cd27071bb93859f97dab9d7
MD5 899ae649006a6890dbbb25254104bf99
BLAKE2b-256 9500d34bb69b5de0c4ffbbbc9f22007c3cd0c99619b7d2856e7a34524c71712e

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page