globus-pilot 0.5.0.dev0

A CLI tool for cataloging pilot 1 data

pilot1-tools

A Command Line tool for managing data in Globus Search as well as transferring corresponding data to and from a Globus Endpoint.

Installation

These tools are available on Conda for Python 3.6, you can install them with the following:

conda create -n pilot1-env -c conda-forge -c nickolaussaint pilot1-tools

You can see the Developer Guide Installation for more options.

Setting Your Local

The first time you run pilot, you may encounter an error an error like the one below:

RuntimeError: Click will abort further execution because Python 3 was
  configured to use ASCII as encoding for the environment.

This can happen if on systems with custom configured UTF-8 settings. You should see a list of encodings your system supports, simply choose one and set it. For example:

export LC_ALL=en_US.utf-8
export LANG=en_US.utf-8

Replace en_US.utf-8 with an encoding your system supports.

Quick Start

For a full walkthrough, see the User Guide. Administrators can also view the Admin Guide.

A quick walkthrough is below.

First, login using Globus:

pilot login

Then choose your project. See pilot project info for info on any listed project:

pilot project
pilot project set <myproject>

You can use list to get a high level overview of the data:

pilot list

If you want more detail about a specific search record, you can use describe to view details:

pilot describe dose_response/rescaled_combined_single_drug_growth

You can also download the data associated with the search record:

pilot download dose_response/rescaled_combined_single_drug_growth

When you want to add more data to the collection, you can use the upload command. This will upload the data in addition to creating a record in Globus Search to track it.

touch my_data.tsv
pilot upload my_data.tsv test_dir --dry-run --verbose -j my_metadata.json

The two flags ‘–dry-run –verbose’ are optional but handy for testing. ‘-j my_metadata.json’ is for providing any extra metadata the pilot tool can’t automatically determine. Here is an example of the metadata:

{
    "title": "Drug Identifiers",
    "description": "Drug identifiers, including InChIKey, SMILES, and PubChem.",
    "data_type": "Drug Response",
    "dataframe_type": "List",
    "source": [
        "InChIKey",
        "SMILES",
        "PubChem"
    ]
}

Running Tests

Ensure packages in test-requirements.txt are installed, then run:

pytest

And for coverage:

pytest --cov pilot

Building for Conda

Currently, the tableschema package has not been built for python 3.7, so this only lists instructions for python 3.6. Two channels must be used, nickolaussaint and conda-forge. The nickolaussaint channel contains fair-research-login, and conda-forge contains various other packages we need including the globus-sdk.

conda build -c nickolaussaint -c conda-forge --python 3.6 .