bioblueprint

bioblueprint is a python library designed to enable workflow language-interchangeable dependency graph compilation and development automation. It operates by compiling workflows, their dependencies, Git diff between branches, and using the modified files to trace testing paths within the dependency graph.

NOTE: bioblueprint operates on local branches

Install

python3 -m pip install bioblueprint

Usage

Run bioblueprint after a development iteration or to automate testing, then edit the "DESCRIPTION" field in the IO TSVs. You can also run after PRs have been generated.

Please see the help menu for a comprehensive list of input options.

bioblueprint -i <REPO_BASE_DIR> -d <DEVELOPMENT_BRANCH>

DEVELOPMENT: -d is the dev branch; -s is main (default)

VALIDATION: -d is main; -s is the previous release tag

PULL REQUESTS: A blank pull request is generated by default, but append -pr # to pull and use an existing PR.

Outputs

An output directory bioblueprint_YYYYmmdd/ will be generated containing the following files:

<REPO>.pr.md

A populated pull request template with I/O modifications, WF modifications, and testing paths. If -pr is specified, the PR will be downloaded and relevant fields populated with I/O and testing information - existing testing data will be retained and unmodified if formatted as a checklist with exact workflow name matches that are the first entry following the markdown checkbox (links are permitted). This function is tailored for accounted repositories:

• Public Health Bioinformatics

<REPO>_inputs.tsv & <REPO>_outputs.tsv

Updated inputs/outputs tables for Public Health Bioinformatics

testing/<WORKFLOW>.testing.json

A JSON formatted with testing parameters (designed for bioforklift integration) for hosted workflows:

{
  "<WF_NAME>": {
    "<TERRA_ENTITY_NAME>: {
      "samples": [], # empty list indicates all samples
      "path": "<PATH_RELATIVE_TO_REPO>",
      "modified": 
          "<TASK/WF_1>",
      ],
      "workflow_name": "<HOSTED_NAME>",
      "branch": "<DERIVED_BRANCH>",
      "repository": "<OWNER/REPOSITORY>",
      "table": "<WF_NAME>_<TESTING_SUFFIX>",
      "comment": "PR: <PR#>",
      "input_json": "<REPOSITORY_INPUTS_JSON>",
      "output_json": "<AVAILABLE_OUTPUTS>"
    },
  }
}

The input_json is derived from an inputs JSON file hosted in the repository that corresponds to the testing table that is hosted in the Terra workspace.

<REPO>.io.json

A JSON formatted to convey inputs and outputs, including defaults and types, for workflows:

{
  <WF_NAME_1>: {
    "path": <PATH_RELATIVE_TO_REPO>,
    "inputs": {
        <INPUT_1>:
        {
            "type": <WF_LANGUAGE_TYPE>,
            "default": <DEFAULT_VAL>
        },
        ..
    },
    "outputs": {
        <OUTPUT_1>: <WF_LANGUAGE_TYPE>,
        ..
    }
  },
  ..
}