Skip to main content

Package extends Airflow functionality with CWL v1.0 support

Project description

# cwl-airflow-parser

### About
Extends **[Apache-Airflow](https://github.com/apache/incubator-airflow)** with **[CWL v1.0](http://www.commonwl.org/v1.0/)** support.

### Installation

```sh
pip3.6 install -U cwl-airflow-parser
```

### Requirements
Package has been tested on Ubuntu 16.04.3 and Mac OS X Sierra/ High Sierra. Make sure your system satisfies the following criteria:
- python 3.6
- [docker](https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/)
- nodejs

### Usage

```python
from cwl_airflow_parser import CWLDAG, CWLJobDispatcher, CWLJobGatherer
from datetime import timedelta

def cwl_workflow(workflow_file):
dag = CWLDAG(default_args={
'owner': 'airflow',
'email': ['my@email.com'],
'email_on_failure': False,
'email_on_retry': False,
'retries': 20,
'retry_exponential_backoff': True,
'retry_delay': timedelta(minutes=30),
'max_retry_delay': timedelta(minutes=60 * 4)
},
cwl_workflow=workflow_file)
dag.create()
dag.add(CWLJobDispatcher(dag=dag), to='top')
dag.add(CWLJobGatherer(dag=dag), to='bottom')

return dag

cwl_workflow("/path/to/my/workflow.cwl")
```

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

cwl-airflow-parser-1.0.20180622010727.tar.gz (9.2 kB view details)

Uploaded Source

File details

Details for the file cwl-airflow-parser-1.0.20180622010727.tar.gz.

File metadata

File hashes

Hashes for cwl-airflow-parser-1.0.20180622010727.tar.gz
Algorithm Hash digest
SHA256 3f08233750e34fb5bbb2ef9203314ad0221c26117cd27071bb93859f97dab9d7
MD5 899ae649006a6890dbbb25254104bf99
BLAKE2b-256 9500d34bb69b5de0c4ffbbbc9f22007c3cd0c99619b7d2856e7a34524c71712e

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page