mccloud-cli 0.0.5

A command line to assist McCloud users.

Intro

mccloud_cli is a command line interface to the McCloud service. McCloud provides a simple means to start a Shasta assembly in the cloud. mccloud_cli currently provides an easy means to generate presigned URLs for use with the McCloud web UI, and the capability to submit McCloud jobs from the command line.

Quick start

1. [optional] we highly suggest you run all python programs in a virtual environment.
2. pip install mccloud-cli
3. mccloud_cli --help

Presigned URLs example

mccloud_cli will generate AWS presigned URLs. This is useful when you want to store your data in your own AWS S3 bucket (ie, privately), but want to temporarily grant McCloud access to read input data, and write results back to your bucket.

Usage:

$ mccloud_cli generate-s3-presigned-url --help
Usage: mccloud_cli generate-s3-presigned-url [OPTIONS]

Options:
  --profile TEXT             Use a specific profile from your AWS credential file.
  --no-confirmation BOOLEAN  Perform requested actions without user confirmation.
  -i, --input-url TEXT       S3 URL which will be presigned to allow READ (input) access.
  -o, --output-url TEXT      S3 URL which will be presigned to allow WRITE (output) access.
  --expires-in INTEGER       Number of seconds until the pre-signed URLs expire.  [default: 28800]
  --help                     Show this message and exit.

For example:

$ mccloud_cli generate-s3-presigned-url --profile my-aws-profile -i s3://my-bucket/reads.fasta.gz -o s3://my-bucket/results.tar.gz

This action will generate AWS S3 presigned URLs for the input and output URLs you have specified, with an expiration time 28800 seconds [8 hours] from now.

Read (input) URLs:
------------------
s3://my-bucket/reads.fasta.gz

Write (output) URLs:
--------------------
s3://my-bucket/results.tar.gz

Do you wish to proceed? [y/N]: Y

Your presigned URLs will expire in 8 hours.

Please launch the McCloud job with these URLs.

Presigned INPUT URLs:
---------------------
https://s3.amazonaws.com/my-bucket/reads.fasta.gz?AWSAccessKeyId=...

Presigned OUTPUT URLs:
----------------------
https://s3.amazonaws.com/my-bucket/results.tar.gz?AWSAccessKeyId=...

Job submission example

mccloud_cli will also submit jobs directly from the command line. Input and output URLs must be https URLs or s3 URLs in a bucket you have access to.

Usage:

$ mccloud_cli submit-job --help
Usage: mccloud_cli submit-job [OPTIONS]

Options:
  --mcat TEXT                     McCloud Access Token, ie, MCAT-****.  [required]
  --email EMAIL ADDRESS           Email address for job notifications.  [required]
  -i, --input-url TEXT            S3 URL which will be presigned to allow READ (input) access.  [required]
  -o, --output-url TEXT           A single S3 URL which will be presigned to allow WRITE (output) access.  [required]
  --shasta-version TEXT           [required]
  --shasta-config TEXT            [required]
  --shasta-cli-opts TEXT
  --I-have-read-and-agree-to-McCloud-terms-of-use
                                  Confirm that you have read, and agree to, the McCloud terms of use, available at
                                  https://mccloud.czi.technology/terms-of-use
  --tail-job-log                  After job is submitted, print out job log messages.
  --profile TEXT                  Use a specific profile from your AWS credential file.
  --help                          Show this message and exit.

For example:

$ mccloud_cli submit-job --mcat 'MCAT-XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX' --email [e-mail] --profile my-aws-profile -i s3://my-bucket/reads.fasta.gz -o s3://my-bucket/output.tar.gz --shasta-version 0.7.0 --shasta-config Nanopore-Sep2020
I have read and agree to mccloud terms of use [y/N]: Y
Starting McCloud job, id = ec39baa4-ea68-4c1e-8040-40277b2a48a0
You can view logs at https://mccloud.czi.technology/jobs/ec39baa4-ea68-4c1e-8040-40277b2a48a0/monitor