Skip to main content

Janis assistant templates for Parkville

Project description

Janis Templates (assistant)

Janis-assistant is a component of Janis used to run workflows across various execution environments. This repository contains a number of preconfigured "templates" that you can use to run your workflow.

A (non-exhaustive) list of examples:

  • Peter MacCallum Cancer Centre (pmac)
  • University of Melbourne (spartan)
  • Walter and Eliza Hall Institute of Medical Research (wehi)
  • Pawsey Supercomputing Centre (pawsey)

These templates are provided with no warranty of guarantee.

Using a template

You can see the parameters you will need to pass the template through the cli, for example:

janis init pawsey --help
usage: janis init pawsey [-h] --executionDir EXECUTIONDIR --containerDir
                         CONTAINERDIR [--queues QUEUES]
                         [--singularityVersion SINGULARITYVERSION]
                         [--catchSlurmErrors] [--sendSlurmEmails]
                         [--singularityBuildInstructions SINGULARITYBUILDINSTRUCTIONS]
                         [--max_cores MAX_CORES] [--max_ram MAX_RAM]

optional arguments:
  -h, --help            show this help message and exit

required arguments:
  --executionDir EXECUTIONDIR
  --containerDir CONTAINERDIR
                        Location where to save and execute containers from

optional arguments:
  --queues QUEUES       A single or list of queues that woork should be
                        submitted to
  --singularityVersion SINGULARITYVERSION
                        Version of singularity to load
  --catchSlurmErrors    Catch Slurm errors (like OOM or walltime)
  --sendSlurmEmails     (requires JanisConfiguration.notifications.email to be
                        set) Send emails for mail types END
  --singularityBuildInstructions SINGULARITYBUILDINSTRUCTIONS
                        Instructions for building singularity, it's
                        recommended to not touch this setting.
  --max_cores MAX_CORES
                        Maximum number of cores a task can request
  --max_ram MAX_RAM     Maximum amount of ram (GB) that a task can request

Create your own template

Step 1 - Create the file + inherit

Create a file within the janis_templates folder along the lines of yourinstitute.py. Create a Python class that at least implements the EnvironmentTemplate declared in janis_assistant.templates.base. It's likely you'll be able to inherit from either:

  • Slurm Singularity: from janis_assistant.templates.slurm import SlurmSingularityTemplate
  • PBS Singularity: from janis_assistant.templates.pbs import PbsSingularityTemplate

For example: vim janis_templates/yourinstitute.py Contents:

from janis_assistant.templates.slurm import SlurmSingularityTemplate

class MyTemplate(SlurmSingularityTemplate):
    pass

Step 2 - Creating the file

Ensure you pass all the required properties to your super class's super().__init__. You can declare extra properties, and override the cromwell method, or the engine_config method too.

It's best to include type annotations, eg:

from listing import List
from janis_assistant.templates.slurm import SlurmSingularityTemplate

class MyTemplate(SlurmSingularityTemplate):
    def __init__(self, param1: str, param2: List[str]=None):
        super().__init__(executionDir=param1, **kwargs)
        self.param2 = param2

Step 2a - Optional step:

Add docstrings beneath the __init__ method with the following format:

from listing import List
from janis_assistant.templates.slurm import SlurmSingularityTemplate

class MyTemplate(SlurmSingularityTemplate):
    def __init__(self, param1: str, param2: List[str]=None):
        """
        :param param1: docstring for param1
        :param param2: docstring for param2
        """
        super().__init__(executionDir=param1, **kwargs)
        self.param2 = param2

Step 3 - Add an entrypoint

Add an entrypoint (best tutorial ever) in setup.py like the following:

    # other setup.py stuff
    entry_points={
        "janis.templates": [
            "yourtemplatename=janis_templates.yourinstitute:MyTemplate",
            # other templates here
        ]
    }

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for janis-pipelines.templates, version 0.8.0
Filename, size File type Python version Upload date Hashes
Filename, size janis_pipelines.templates-0.8.0-py3-none-any.whl (9.3 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size janis-pipelines.templates-0.8.0.tar.gz (5.7 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page