janis-pipelines 0.11.3

Contains classes and helpers to build a workflow, and provide options to convert to CWL / WDL

Janis (janis-pipelines) (Alpha)

Janis is a framework creating specialised, simple workflow definitions that are then transpiled to
Common Workflow Language or Workflow Definition Language.

Documentation is available here: https://janis.readthedocs.io/

Introduction

WARNING: this project is work-in-progress and is provided as-is without warranty of any kind. There may be breaking changes committed to this repository without notice.

Janis gives you an API to build computational workflows and will generate a workflow description in CWL and WDL. By using Janis, you get type-safety, portability and reproducibility across all of your execution environments.

Janis requires a Python installation > 3.6, and can be installed through PIP (project page):

# Install janis and the toolkits
pip3 install janis-pipelines 

There are two ways to use Janis:

• Build workflows (and translate to CWL / WDL)
• Run tools or workflows with CWLTool or Cromwell

Latest versions

Check the table below for the latest released versions of Janis (janis -v):

Repository	Version
Janis
janis-assistant
janis-bioinformatics
janis-core
janis-pipelines
janis-templates
janis-unix

Example workflow

Let's construct a simple example that takes a string input, calls the echo tool and exposes the Echo tool's output as a workflow output.

# write the workflow to `helloworld.py`
cat <<EOT >> helloworld.py
import janis as j
from janis_unix.tools import Echo

w = j.WorkflowBuilder("hello_world")

w.input("input_to_print", j.String)
w.step("echo", Echo(inp=w.input_to_print))
w.output("echo_out", source=w.echo.out)
EOT


# Translate workflow to WDL
janis translate helloworld.py wdl

# Run the workflow
janis run -o helloworld-tutorial helloworld.py --input_to_print "Hello, World!"

# See your output
cat helloworld-tutorial/echo_out
# Hello, World!

How to use Janis

• Tutorial 0 - Introduction to Janis
• Tutorial 1 - Building a workflow
• Tutorial 2 - Wrapping a new tool

Workshops

In addition, there are fully self-guided workshops that more broadly go through the functionality of Janis:

• Workshop 1
• Workshop 2

Examples

Sometimes it's easier to learn by examples, here are a few hand picked examples:

• Samtools View (Docs)

• WGS Germline pipeline (GATK Only) (Docs)

Toolbox

There are two toolboxes currently available on Janis:

• Unix (list of tools)
• Bioinformatics (list of tools)

About

Further information: About

This project was produced as part of the Portable Pipelines Project in partnership with:

• Melbourne Bioinformatics (University of Melbourne)
• Peter MacCallum Cancer Centre
• Walter and Eliza Hall Institute of Medical Research (WEHI)

References:

Through conference or talks, this project has been referenced by the following titles:

• Walter and Eliza Hall Institute Talk (WEHI) 2019: Portable Pipelines Project: Developing reproducible bioinformatics pipelines with standardised workflow languages
• Bioinformatics Open Source Conference (BOSC) 2019: Janis: an open source tool to machine generate type-safe CWL and WDL workflows
• Victorian Cancer Bioinformatics Symposium (VCBS) 2019: Developing portable variant calling pipelines with Janis
• GIW / ABACBS 2019: Janis: A Python framework for Portable Pipelines
• Australian BioCommons, December 2019: Portable pipelines: build once and run everywhere with Janis

Support

v0.9.0 Backwards Compatability

NOTE: Version 0.9.0 brings changes to output directories and camel case changes

• Janis watch will be incompatible with previously run workflows
• Your configs might break, as previous versions of janis were not cautious about camel case.
• Your templates might not work with unrecognised keys (try changing them to camel case instead)
• Changes to BamBai indexes, format is now .bam.bai

See the CHANGELOG for more information.

Contributions

Further information: Development

To get help with Janis, please ask a question on Gitter or raise an issue on GitHub.

If you find an issue with the tool definitions, please see the relevant issue page:

• Pipeline-bioinformatics

This project is work-in-progress and is still in developments. Although we welcome contributions, due to the immature state of this project we recommend raising issues through the Github issues page for Pipeline related issues.

Information about the project structure and more on contributing can be found within the documentation.