janis-pipelines 0.7.0

Contains classes and helpers to build a workflow, and provide options to convert to CWL / WDL

Janis (Pre-alpha)

Janis is a framework creating specialised, simple workflow definitions that are then transpiled to
Common Workflow Language or Workflow Definition Language.

Documentation is hosted here: https://janis.readthedocs.io/

v0.6.0 Backwards Compatability

NOTE: Version 0.6.0 brings large changes to simplify the workflow API.

See the example below to see how this new syntax works. Instead of creating input nodes and adding them to the graph, you add an input / output / step to the graph, and map the connections within the Step.

Inline workflows are now creating with the WorkflowBuilder class.

w.step("step_tag", Echo(inp=w.input_tag))

Introduction

WARNING: this project is work-in-progress and is provided as-is without warranty of any kind. There may be breaking changes committed to this repository without notice.

Janis gives you an API to build computational workflows and will generate a workflow description in CWL and WDL. By using Janis, you get type-safety, portability and reproducibility across all of your execution environments.

Janis requires a Python installation > 3.6, and can be installed through PIP (project page):

# Install janis and the bioinformatics tools
pip3 install janis-pipelines[bioinformatics]  

You can import Janis into your project with:

import janis as j  

Example

Further information: Simple Workflow

Below we've constructed a simple example that takes a string input, calls the echo tool and exposes the Echo tool's output as a workflow output.

import janis as j
from janis.unix.tools.echo import Echo

w = j.WorkflowBuilder("workflowId")

w.input("inputIdentifier", j.String, default="Hello, World!")
w.step("stepIdentifier", Echo(inp=w.inputIdentifier))
w.output("outputIdentifier", source=w.stepIdentifier.out)

# Will print the CWL, input file and relevant tools to the console
w.translate("cwl", to_disk=False)  # or "wdl"

We can export a CWL representation to the console using .translate("cwl"). By including the to_disk=True parameter, we can write this workflow to disk at the current location.

More examples

• Bioinformatics workflow tutorial: AlignSortedBam

• Simple unix examples: in janis/examples.

• Whole genome germline pipelines: janis-pipelines repository.

About

Further information: About

This project was produced as part of the Portable Pipelines Project in partnership with:

• Melbourne Bioinformatics (University of Melbourne)
• Peter MacCallum Cancer Centre
• Walter and Eliza Hall Institute of Medical Research (WEHI)

References:

Through conference or talks, this project has been referenced by the following titles:

• Walter and Eliza Hall Institute Talk (WEHI) 2019: Portable Pipelines Project: Developing reproducible bioinformatics pipelines with standardised workflow languages
• Bioinformatics Open Source Conference (BOSC) 2019: Janis: an open source tool to machine generate type-safe CWL and WDL workflows
• Victorian Cancer Bioinformatics Symposium (VCBS) 2019: Developing portable variant calling pipelines with Janis

Support

Contributions

Further information: Development

This project is work-in-progress and is still in developments. Although we welcome contributions, due to the immature state of this project we recommend raising issues through the Github issues page for Pipeline related issues.

Information about the project structure and more on contributing can be found within the documentation.