firecloud 0.14.1

Firecloud API bindings and FISS CLI

====================
1. Description
====================

FireCloud API bindings and FireCloud Service Selector (FISS)
Command-Line interface.

FireCloud:
https://software.broadinstitute.org/firecloud/

Github:
https://github.com/broadinstitute/fiss

====================
2. Contents
====================
1. Description
2. Contents
3. Requirements
4. Release Notes
5. FISS
6. Low-Level API
7. High-Level Classes
8. Yapsy Plugin System

This package contains low-level api bindings to FireCloud services,
high-level class abstractions for workspaces and entities, and the FISS
command-line tool for scripting FireCloud Tasks.

====================
3. Requirements
====================

Install this package via pip:
% pip install firecloud

Or download the source and install with setup.py:
% python setup.py install

This package depends on yapsy, oauth2client, and requests.

FireCloud uses the Google Cloud SDK (https://cloud.google.com/sdk/) to manage
authorization. To use the firecloud CLI or API, you must install the SDK and
login locally with

gcloud auth login

**Note: FireCloud encryts all web traffic with TLSv1.2. Some versions of
python will not function correctly, and must be recompiled with openssl
v1.02 or later. If recompiling is not an option, there is also a
dockerized version of the tool available on docker hub
(https://hub.docker.com/r/broadgdac/fiss/).

% docker pull broadgdac/fiss

====================
4. Release Notes
====================
v0.14.1 - Bugfixes and validation for supervisor mode. Make flow_set_acl smarter,
and make the snapshot_id argument be optional, defaulting to the latest.

v0.14.0 - proj_list command for listing available billing projects. Rename
recover_supervisor to supervise_recover, for conceptual symmetry.

v0.13.2 - Fix getting workspace attributes when not using the attributes flag.

v0.13.1 - More helpful error checking for loadfile format during entity_import.

v0.13.0 - Some fixes to match API changes in FireCloud Orchestration and
several new commands:

supervise -- a robust mechanism of submitting related
workflows similar to Firehose workflows. Workflows described in DOT
can be submitted and monitored in one command, and fissfc will
monitor the entire workflow until completion.

recover_supervisor -- supervise saves workflow execution state after
each API call, so that the monitor can be restarted with this
command in case of a FISS crash.

attr_set -- set attributes in a workspace.

attr_delete -- delete attributes in a workspace.

attr_copy -- copy attributes from one workspace to another.

entity_copy -- copy entities from one workspace to another. Note that
referential constraints currently limit the ability to overwrite
entities in the target workspace.

v0.12.0 - Large refactor of argument handling for FISS. Almost all arguments
are now passed using flags instead of positional arguments, enabling
the argument namespace to be shared by command iterators. This allows
sset_loop (a new looping command) to call multiple FISS commands
without having to reparse arguments.

New FISS command for submitting workflows, 'flow_submit'.

v0.11.0 - New endpoint for paginated entity queries, also used by attr_get
to more reliably return attributes on large workspaces.

Also added new FISS command `attr_fill_null`, which will assign null
sentinel files to entities which do not have a valid attribute. This,
in a sense, "fills in" the sparse matrix of data for a given list of
attributes.

v0.10.1 - Fix Py3 compatibility issues.

v0.10.0 - Remove `fiss` entrypoint, to not overwrite fiss 1.0 installs. New fiss
"mop" command, to clean files that are not referenced in a workspace
attribute from the underlying google bucket.

v0.9.6 - Fix expected response code for entity_import

v0.9.5 - Fix urllib compatibility issue in upload_entities

v0.9.4 - Add ability to change inputs and other configuration details in a
workspace configuration

v0.9.3 - Improved documentation. Fixed url for method configurations in a workspace.

v0.9.2 - Remove file-based check for getting application default credentials,
improving support for Jupyter-based sessions.

v0.9.1 - Fix expected return code for flow_new.

v0.9.0 - Replace Httplib2 with Requests, which resolves the SSL handshake issue
for python3. Added six to required packages, making FireCloud fully
compatible with python 2 & 3. Add links to FireCloud's swagger
documentation.

v0.8.1 - Improved test coverage. Enhanced interactive classes. Fixed error logic
to match new server response.

v0.8.0 - firecloud.api now boasts 100% coverage of existing (useful) API endpoints.

Several api endpoints for manipulating NIH credentials and profile info
will deliberately remain unimplemented unless a good reason arises. Any
discrepancies are noted in the api module's docstring; see
help(firecloud.api) for a lengthy overview.

New Method and Configuration objects.
>>> m = Method("broad-firecloud-gdac", "gistic2", 1)


v0.7.4 - FISS functions can now display their source code with the -F option.

v0.7.3 - Improved documentation, with docstrings for every public function.
This enables python's builtin help system to be used from the REPL.
To see all the available firecloud api calls:
% python
>>> import firecloud
>>> help(firecloud.api)

v0.7.2 - Fix some commands that broke with the API changes

v0.7.1 - Add workspace functions for importing entities and creating entity sets.

v0.7.0 - Renamed package "firecloud", replacing "fissfc"
- firecloud_cli.py renamed fiss.py
- firecloud_api.py renamed api.py
Added workspace class for more flexible python interactivity. Example:
>>> w = Workspace.new("broad-firecloud-testing", "my_workspace")
>>> w.bucket()
"fc-92e4ic0a-60fb-454a-8ue0-d0efea803421"
>>> w.lock()
Added better error messaging for unexpected FireCloud Errors

v0.6.2 - Add '--protected' option for workspace creation.

v0.6.1 - Firecloud production API is now https://api.firecloud.org/api

v0.6.0 - Refactored the argument structure for inferring a default project.
Instead of passing the namespace as an optional input to a
subcommand, the project can be overriden explicitly with the -p
top-level flag. This cleans up the syntax for many commands, and
makes the help clearer.

Add --all option to space_list
attr_get:
* attributes now given a a list of positional arguments,
instead of comma-separated
* entity type limited to firecloud supported entities
Fixed help messages for many commands


v0.5.0 - Add python 3 support, so that fissfc should be installable to
either version. A TLS issue may still prevent fissfc from
working on some versions of python.

v0.4.1 - Add attr_get subcommand, as a replacement for fiss annot_get.
Clarify error message.

v0.4.0 - Implement default value for google project into most subcommands

v0.3.2 - Add check in firecloud_api for the existence of gcloud credentials

v0.3.1 - Add confirmation prompt for delete commands, and a
Flag to override them.

v0.3.0 - Added support for Yapsy plugins.

v0.0.1 - Initial Release

=====================================
5. FireCloud Service Selector (FISS)
=====================================

FISS enables high-level commands to be executed directly from the command line,
and is installed locally to /usr/local/bin/fiss.

FISS includes several forms of self-documentation. Type

% fiss --help

to get an overview. To list available commands, run

% fiss -l

Each command includes it's own help:

% fiss space_list -h

and as a bonus, to see how a command is implemented (including the underlying api calls),
use the "-F" option:

% fiss -F space_list

===========================
6. FireCloud Low-Level API
===========================
The following is a list of available endpoints as of v0.9.3. Swagger
documentation for the FireCloud orchestration layer is available at
https://api.firecloud.org/. See help(firecloud.api) for more details.

abort_submission()
clone_workspace()
copy_config_from_repo()
copy_config_to_repo()
copy_entities()
create_submission()
create_workspace()
create_workspace_config()
delete_entity()
delete_pair()
delete_pair_set()
delete_participant()
delete_participant_set()
delete_repository_method()
delete_sample()
delete_sample_set()
delete_workspace()
delete_workspace_config()
get_config_template()
get_entities()
get_entities_tsv()
get_entities_with_type()
get_entity()
get_inputs_outputs()
get_repository_config()
get_repository_config_acl()
get_repository_method()
get_repository_method_acl()
get_status()
get_submission()
get_submission_queue()
get_workflow_outputs()
get_workspace()
get_workspace_acl()
get_workspace_config()
list_billing_projects()
list_entity_types()
list_repository_configs()
list_repository_methods()
list_submissions()
list_workspace_configs()
list_workspaces()
lock_workspace()
ping()
rename_workspace_config()
unlock_workspace()
update_repository_config_acl()
update_repository_method()
update_repository_method_acl()
update_workspace_acl()
update_workspace_attributes()
update_workspace_config()
upload_entities()
upload_entities_tsv()
validate_config()

=======================
7. High-Level Classes
=======================

Workspace (firecloud.workspace.Workspace)
Entity (firecloud.entity.Entity)
Submisssion (firecloud.submission.Submission)
Config (firecloud.config.Config)

==========================
8. Yapsy plugins for FISS
==========================

The FISS CLI includes a yapsy-based plugin system for defining additional
command line options. Documentation coming soon.