Command line tool to upload and download CWL to and from SB powered platforms.
Project description
sbpack
Upload (sbpack
) and download (sbpull
) CWL apps to/from any Seven Bridges powered platform.
Resolves linked processes, schemadefs and $include
s and $import
s.
Installation
(It is good practice to install Python programs in a virtual environment. pipx is a very effective tool for installing command line Python tools in isolated environments)
sbpack
needs Python 3.6 or later
pip3 install pipx # in case you don't have pipx
pipx ensurepath # ensures CLI application directory is on your $PATH
Install latest release on pypi
pipx install sbpack
# or pipx upgrade
Install latest (unreleased) code
pipx install git+https://github.com/rabix/sbpack.git
# use pipx upgrade ... if upgrading an existing install
Usage
Usage
sbpack <profile> <id> <cwl>
where:
<profile> refers to a SB platform profile as set in the SB API credentials file.
<id> takes the form {user}/{project}/{app_id} which installs (or updates)
"app_id" located in "project" of "user".
<cwl> is the path to the main CWL file to be uploaded. This can be a remote file.
Uploading workflows defined remotely
sbpack
handles local paths and remote URLs in a principled manner. This means that
sbpack
will handle packing and uploading a local workflow that links to a remote workflow
which itself has linked workflows. It will therefore also handle packing a fully
remote workflow.
For example, to pack and upload the workflow located at https://github.com/Duke-GCB/GGR-cwl/blob/master/v1.0/ATAC-seq_pipeline/pipeline-se.cwl
go to the raw
button and use that URL, like:
sbpack sbg kghosesbg/sbpla-31744/ATAC-seq-pipeline-se https://raw.githubusercontent.com/Duke-GCB/GGR-cwl/master/v1.0/ATAC-seq_pipeline/pipeline-se.cwl
Local packing
Usage
cwlpack <cwl> > packed.cwl
The cwlpack
utility allows you to pack a workflow and print it out on stdout
instead of
uploading it to a SB platform.
Side-note
As an interesting side note, packing a workflow can get around at least two cwltool
bugs
[1], [2].
Pulling (and unpacking)
sbpull
will retrieve CWL from any SB powered platform and save it to local disk.
sbpull sbg admin/sbg-public-data/salmon-workflow-1-2-0/ salmon.cwl
With the --unpack
option set, it will also explode the workflow recursively, extracting out each
sub-process into its own file.
sbpull sbg admin/sbg-public-data/salmon-workflow-1-2-0/ salmon.cwl --unpack
This is useful if you want to use SB platform CWL with your own workflows. You can pull the relevant CWL into your code repository and use it with the rest of your code. If you use the
--unpack
option you can access the individual components of the SB CWL workflow separately.
Pulling a particular revision
While
sbpull sbg admin/sbg-public-data/bismark-0-21-0/ bismark.cwl
will pull the latest version of Bismark on the platform,
sbpull sbg admin/sbg-public-data/bismark-0-21-0/2 bismark.cwl
will pull revision 2 of this tool
Note on reversibility
sbpack
and sbpull --unpack
are not textually reversible. The packed and unpacked CWL
representations are functionally identical, however if you sbpack
a workflow, and
then sbpull --unpack
it, they will look different.
Credentials file and profiles
If you use the SBG API you already have an API configuration file. If
not, you should create one. It is located in
~/.sevenbridges/credentials
. (Documentation)
Briefly, each section in the SBG configuration file (e.g. [cgc]
) is a
profile name and has two entries. The end-point and an authentication
token, which you get from your developer tab on the platform.
[sbg-us]
api_endpoint = https://api.sbgenomics.com/v2
auth_token = <dev token here>
[sbg-eu]
api_endpoint = https://eu-api.sbgenomics.com/v2
auth_token = <dev token here>
[sbg-china]
api_endpoint = https://api.sevenbridges.cn/v2
auth_token = <dev token here>
[cgc]
api_endpoint = https://cgc-api.sbgenomics.com/v2
auth_token = <dev token here>
[cavatica]
api_endpoint = https://cavatica-api.sbgenomics.com/v2
auth_token = <dev token here>
[nhlbi]
api_endpoint = https://api.sb.biodatacatalyst.nhlbi.nih.gov/v2
auth_token = <dev token here>
You can have several profiles on the same platform if, for example, you are an enterprise user and you belong to several divisions. Please refer to the API documentation for more detail.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for sbpack-2020.9.11-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e794cd32ee0d9bdb4f2650657241d35491eee370013c5558e742801deba70423 |
|
MD5 | 99ceb0c6ac4529f8a5acbf58f2708388 |
|
BLAKE2b-256 | 4666b0deb35f733193694edd802e0d6724f42d03a193769efe94df3291d52b3f |