Skip to main content

Tools for working with the pathogen pipeline at The Wellcome Trust Sanger Institute

Project description

This is probably not very interesting unless you work in the pathogen informatics team at the Wellcome Trust Sanger Institute and almost certainly not it you don’t use the vr-pipe pipeline.

Given a config directory, this script finds all of the *_pipeline.conf files in the directory and its immediate children. It parses these files and outputs their contents in json. This is either saved to a file or stdout.

This is used in collaboration with jsontoemail and a cron job to send updates / reminders to the pathogen informatics team.

Usage

$ list-pathogen-pipeline-jobs -h
usage: list-pathogen-pipeline-jobs [-h] [--output_file OUTPUT_FILE] config_dir

Outputs details of all of the pathogen jobs in the pipeline

positional arguments:
  config_dir            Config directory to be searched for job trackers (also
                        searches child dircectories)

optional arguments:
  -h, --help            show this help message and exit
  --output_file OUTPUT_FILE, -o OUTPUT_FILE
                        File to output results to (defaults to stdout)

Example output

{
  "created_at": "2015-03-24T15:26:17.246253",
  "jobs": [
    {
      "approval_required": true,
      "config_file": "/parent_dir/assembly_jobs/job_1.conf",
      "job_type": "__Assembly__",
      "pipeline_tracker": "/parent_dir/assembly_job_tracker.conf"
    },
    {
      "approval_required": true,
      "config_file": "/parent_dir/assembly_jobs/job_2.conf",
      "job_type": "__Assembly__",
      "pipeline_tracker": "/parent_dir/assembly_job_tracker.conf"
    },
    {
      "approval_required": false,
      "config_file": "/parent_dir/assembly_jobs/job_3.conf",
      "job_type": "__Assembly__",
      "pipeline_tracker": "/parent_dir/assembly_job_tracker.conf"
    },
    {
      "approval_required": true,
      "config_file": "/parent_dir/annotation_jobs/job_1.conf",
      "job_type": "__Annotation__",
      "pipeline_tracker": "/parent_dir/annotation_job_tracker.conf"
    },
    {
      "approval_required": false,
      "config_file": "/parent_dir/mapping_jobs/job_1.conf",
      "job_type": "__Mapping__",
      "pipeline_tracker": "/parent_dir/mapping_job_tracker.conf"
    },
    {
      "approval_required": false,
      "config_file": "/parent_dir/mapping_jobs/job_2.conf",
      "job_type": "__Mapping__",
      "pipeline_tracker": "/parent_dir/mapping_job_tracker.conf"
    }
  ]
}

Testing

./run_tests.sh

Dependencies

Tests require:

  • mock

path_pipe_conf_tools has been tested on Ubuntu 12.04 with python 2.7.3

Project details


Release history Release notifications

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for path_pipe_conf_tools, version 0.0.2
Filename, size File type Python version Upload date Hashes
Filename, size path_pipe_conf_tools-0.0.2.tar.gz (3.7 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page