janis-pipelines 0.2.16

Contains classes and helpers to build a workflow, and provide options to convert to CWL / WDL

Janis

WARNING: this project is work-in-progress and is provided as-is without warranty of any kind. There may be breaking changes committed to this repository without notice.

Janis is a framework creating specialised, simple workflow definitions that are then transpiled to Common Workflow Language or Workflow Definition Language.

Documentation is hosted here: https://janis.readthedocs.io/

Introduction

Janis is designed to assist in building computational workflows to generate a runnable workflow description (CWL | WDL). It can be installed through PIP (project page) by running:

pip install janis-pipelines

You can import Janis into your project by:

import janis as j

Included tool definitions and types

Some basic unix tools have been wrapped and included as part of the base Janis module and are the basis for the examples. You can reference these unix tools through janis.unix.tools.

Bioinformatics

The Janis framework can be extended to include a suite of Bioinformatics data types and tools. These can be installed with the bioinformatics install extra option.

pip install janis-pipelines[bioinformatics]

These can be referenced by janis.bioinformatics or janis_bioinformatics, the latter might be easier due to the way nested python imports work.

Example

Further information: Simple Workflow

Below we've constructed a simple example that takes a string input, uses the echo tool to log this to stdout, and explicitly outputting this stdout to give you a basic idea of how to construct a pipeline.

import janis as j
from janis.unix.tools.echo import Echo 

w = j.Workflow("workflow_identifier")

inp = j.Input("input_identifier", j.String())
step = j.Step("step_identifier", Echo())
outp = j.Output("output_identifier")

w.add_pipe(inp, step, outp)

# Will print the CWL, input file and relevant tools to the console
w.dump_translation("cwl")

The add_pipe method is aware of the inputs and outputs of the arguments you provide it, and automatically joins the relevant non-optional parts together. More information can be found on creating edges on the "Building Connections" documentation.

Now we can an in-memory workflow, we can export a CWL representation to the console using .dump_translation("cwl").

More examples

There are some simple example pipelines that use the unix toolset in janis/examples.

Additionally there are example bioinformatics workflows that use Janis and the bioinformatics tools in the janis-examplepipelines repository.

About

This project was produced as part of the Portable Pipelines Project in partnership with:

• Melbourne Bioinformatics (University of Melbourne)
• Peter MacCallum Cancer Centre
• Walter and Eliza Hall Institute of Medical Research (WEHI)

Motivations

Given the awesome list of pipeline frameworks, languages and engines, why create another framework to generate workflow langauges.

That's a great question, and it's a little complicated. Our project goals are to have a portable workflow specification, that is reproducible across many different compute platforms. And instead of backing one technology, we thought it would be more powerful to create a technology that can utilise the community's work.

Some additional benefits we get by writing a generic framework is we sanity check connections and also add types that exist within certain domains. For example within the bioinformatics tools, there's a BamBai type that represents an indexed .bam (+ .bai) file. With this framework, we don't need to worry about pesky secondary files, or the complications that come when passing them around in WDL either, this framework can take care of that.

Related project links:

• Janis:

  • Janis Documentation: https://janis.readthedocs.io/en/latest/
  • Janis Git: https://github.com/PMCC-BioinformaticsCore/janis
  • Janis PyPi: https://pypi.org/project/janis-pipelines/
  • Janis Bioinformatics:
    • Git: https://github.com/PMCC-BioinformaticsCore/janis-bioinformatics
    • PyPi: https://pypi.org/project/illusional.wdlgen/

• Shepherd: https://github.com/PMCC-BioinformaticsCore/shepherd

• CWLGen (forked): https://github.com/illusional/python-cwlgen

• WDLGen: https://github.com/illusional/python-wdlgen

	build	docs	pypi	codecov
Janis
Bioinformatics		See Janis
Shepherd
CWL-Gen
WDL-Gen

Support

Contributions

This project is work-in-progress and is still in developments. Although we welcome contributions, due to the immature state of this project we recommend raising issues through the Github issues page for Pipeline related issues.

If you find an issue with the tool definitions, please see the relevant issue page:

• Pipeline-bioinformatics

Information about the project structure and more on contributing can be found within the documentation.

Releasing Janis

Further Information: Releasing

Releasing is automatic! Simply increment the version number in setup.py (SemVer), and tag that commit with the same version identifier:

git commit -m "Tag for v0.x.x release"
git tag -a "v0.x.x" -m "Tag message"
git push --follow-tags