Command line interface (CLI) and Python 3 client library for interacting with the CosmosID API
Project description
Cosmos ID
Command line interface (CLI) and Python 3 client library for interacting with the CosmosID API. Only works with Python [3.6,3.7,3.8,3.9,3.10].
Requirements
OS Packages
- python3
- python3-pip
Python package
- poetry
Installation
This package provides:
- core Python 3 client library;
- a simple CLI for interacting with the CosmosID that uses the core library;
Basic installation
The CLI with the core Python library can be installed using pip3
.
- simply run from console
sudo pip3 install cosmosid_cli
Note: pip3 and setuptools should be upgraded to latest version. Please update those packages on you workstation regarding to your OS update process before setup cosmosid cli.
E.g. for Ubuntu 14.04 perform following steps: $ sudo apt-get update $ sudo apt-get upgrade $ sudo -H pip3 install -U pip setuptoolsIf you had have previously installed CosmosID CLI just upgrade CLI to latest version.
$ sudo -H pip3 install --upgrade cosmosid_cli
To install package locally from folder with source files do the following:
- install
poetry
check the doc cd cosmosid-cli/package/
poetry install
Using the CosmosID CLI
The CosmosID CLI supports authentication via CosmosID API Key.
Your API key can be found on the CosmosID profile page.
To automatically authenticate via CosmosID API Key you should create
credential file ~/.cosmosid
and store your API Key into it in
the following format:
{"api_key": "<your api key string>"}
You can directly use your CosmosID API Key, rather than storing it in a credential file. To use API Key authentication, pass your key as an argument to the cosmosid
command:
cosmosid --api_key=YOUR_API_KEY <command>
CLI supports files of following extensions: 'fasta', 'fna', 'fasta.gz', 'fastq', 'fq', 'fastq.gz', 'bam', 'sra'
Commands
There are several types of commands supported by the CosmosID CLI
- Commands for retrieving data to terminal (output) from CosmosID cloud - files, runs, analysis.
- Commands for uploading metagenomics or amplicon samples to CosmosID cloud for analysis - uploads.
- Commands for retrieving the reports archive from CosmosID cloud - reports
Note: Each command has options. To get usage information for each CosmosID CLI command, the user can simply run
cosmosid <command> --help
Retrieve Files
The commands for retrieving data have options for output format. The user can get data into the terminal (or another output) in a different format - csv, json, table, value, yaml (table is default), and specify the column(s) to show. In additional there are CSV format options, user can quote or unquote or partly quote output values - all, minimal, none, non-numeric (by default only non-numeric values are quoted)
Example of output for the --help options for the command:
$ cosmosid files --help
usage: cosmosid files [-h] [-f {csv,json,table,value,yaml}] [-c COLUMN]
[--noindent] [--max-width <integer>] [--fit-width]
[--print-empty] [--quote {all,minimal,none,nonnumeric}]
[--parent PARENT]
[--order {type,name,id,status,size,created}] [--up]
Show files in a given directory.
optional arguments:
-h, --help show this help message and exit
--parent PARENT, -p PARENT
ID of the parent directory. Default: Root
--order {type,name,id,status,reads,created}, -o {type,name,id,status,size,created}
field for ordering
--up order direction
output formatters:
output formatter options
-f {csv,json,table,value,yaml}, --format {csv,json,table,value,yaml}
the output format, defaults to table
-c COLUMN, --column COLUMN
specify the column(s) to include, can be repeated
json formatter:
--noindent whether to disable indenting the JSON
table formatter:
--max-width <integer>
Maximum display width, 1 to disable. You can also use
the CLIFF_MAX_TERM_WIDTH environment variable, but the
parameter takes precedence.
--fit-width Fit the table to the display width. Implied if --max-
width greater than 0. Set the environment variable
CLIFF_FIT_WIDTH=1 to always enable
--print-empty Print empty table if there is no data to show.
CSV Formatter:
--quote {all,minimal,none,nonnumeric} when to include quotes, defaults to nonnumeric
To retrieve files (samples) stored in CosmosID simply run the cosmosid
command with a files
subcommand. For example:
#to get contents of your CosmosID root folder
cosmosid files
#to get contents of appropriate folder use its id as argument
cosmosid files --parent=<folder_id>
#to get ordered list simply use the ordering argument with field name with/without order direction
cosmosid files --parent=<folder_id> --order size --up
Retrieve Sample Runs
An each file (sample) stored in CosmosID has one or more Sample Run(s) associated with it.
To retrieve sample run(s) associated with a file simply run the cosmosid
command with runs
subcommand. For example:
#to get runs associated with a speciffic file (sample)
cosmosid runs --id=<file_id>
Upload files
The CosmosID CLI supports uploading sample files into CosmosID for analysis. CosmosID supports the following file formats and extension names: .fasta, .fna, .fasta.gz, .fastq, .fq, .fastq.gz, bam, bam.gz, sra, sra.gz. (SRA files can be uploaded without extension)
CosmosID supports the following types of analysis:
- Metagenomics
- Amplicon - 16S or ITS (only 16S and ITS supported for now)
Note: you can get usage help for each command and arguments of CosmosID CLI by simply runnig
cosmosid --help
orcosmosid <command> --help
To upload sample file to CosmosID run cosmosid
command with upload
subcommand. By default samples will be uploaded into root folder. To upload sample into specific existing folder you must use id of the folder as parameter.
The CosmosID CLI supports uploading multiple Single-Read and Paired-End samples. For Paired-End samples, the CLI automatically parse and merge samples in pairs if the samples follow the naming conventions like: xxx_R1.fastq and xxx_R2.fastq OR xxx_R1_001.fastq and xxx_R2_001.fastq. Note: Paired-End samples require "fastq" format
To upload all samples from folder run cosmosid upload
command with path to folder specified by --dir/-d parameter
Note: This command respects Paired-End samples grouping with the same rules as for regular upload
Running example:
cosmosid upload --type metagenomics -f /pathtofile/test1_R1.fastq
-f /pathtofile/test1_R2.fastq -f /pathtofile/test2.fasta
#to upload one sample file for Metagenomics analysis
cosmosid upload --file <path to file> --type metagenomics
#to upload sample file into specific folder for Amplicon 16s analysis
cosmosid upload --file <path to file-1> --parent <folder id> --type amplicon-16s
#to upload all files from folder
cosmosid upload -d /home/user/samples/ --type metagenomics
Note: uploading of a big file takes time, please be patient
Once file has been uploaded to CosmosID the analyzing process will automatically begin. You can check the status of metagenomics analysis on the page CosmosID Samples. Amplicon analysis results available only from CosmosID CLI for now.
Retrieving Analysis Results
Analysis results can be retrieved from CosmosID by useing run id or file id. The latest run analysis results will be retrieved when file id used.
To retrieve analysis results for a specified run in CosmosID simply run cosmosid
command with analysis
subcommand. File For example:
#to get list of analysis for the latest run of file
cosmosid analysis --id=<file ID>
#to get list of analysis for a given run id
cosmosid analysis --run_id=<run ID>
#to get ordered list of analysis for a given file id simply use ordering argument with field name with/without order direction
cosmosid analysis --id=<file ID> --order created --up
Note: There is no analysis results for Amplicon 16S and Amplicon ITS sample. Use report generation instead of getting list of analysis for Amplicon 16S and Amplicon ITS.
Generate Analysis Report Archive
The CosmosID CLI supports retrieving the archive of analysis reports from CosmosID for a given File ID
with a given Run ID
and saving the archive to a given file.
To retrieve an analysis report archive with TSV files run the cosmosid
command with reports
subcommand.
# to create analysis report archive for the latest run of sample and save it in
# a current directory with a name equivalent to file name in CosmosID
cosmosid reports --id=<file ID>
# to create analysis report archive for the given run of sample and save it in
# a current directory with a name equivalent to file name in CosmosID
cosmosid reports --id=<file ID> --run_id=<run ID>
# to create analysis report archive for the given run of sample and save it
# in a given directory
cosmosid reports --id=<file ID> --run_id=<run ID> --dir ~/cosmosid/reports
# to create analysis report archive for the given run of sample and save it
# into a given local file
cosmosid reports --id=<file ID> --output /tmp/analysis_report.zip
Retrieving Artifacts Results
Artifacts results can be retrieved from CosmosID by using run id.
To retrieve artifacts results for a specified run in CosmosID simply run cosmosid
command with artifacts
subcommand. File For example:
#to get list of artifacts for a given run id
cosmosid artifacts --run_id=<run ID>
##to create artifacts archive for the given run id of sample and store it to given path
cosmosid artifacts --run_id=<run ID> --type=fastqc-zip --dir /home/user
##to create artifacts archive for the given run id of sample and store it with given name in current dir
cosmosid artifacts --run_id=<run ID> --type=fastqc-zip --output artifacts_report.zip
##to create artifacts archive for the given run id of sample and store it with given name and given dir
cosmosid artifacts --run_id=<run ID> --type=fastqc-zip --dir /home/user --output artifacts_report.zip
#to get url to download the archive
cosmosid artifacts --run_id=<run ID> --type=fastqc-zip --url
Download Original Samples
Original samples can be downloaded from CosmosID by using samples_ids.
To download samples for a specified samples_id in CosmosID simply run cosmosids
command with download
subcommand.
Note: We recommend installing pycurl for the best experience with a sample download, see: http://pycurl.io/docs/latest/index.html#installation
For example:
#to download the original samples and save them in the current dir
cosmosid download --samples_ids=<sample_id>
#to download the originals samples and save them in the current dir
cosmosid download --samples_ids=<sample_id>,<sample_id> #separated by comma ","
#to download the originals samples and store them in the given path
cosmosid download --samples_ids=<sample_id>,<sample_id> --dir=<path_to_directory>
#to download the originals samples without displaying download progress
cosmosid download --samples_ids=<samples_id>,<sample_id> --no-display
#to download the original samples with specified quantity of concurrent files downloads
cosmosid download --samples_ids=<sample_id>,<sample_id> --concurrent-downloads=<quantity>
Note: You can specify chunk size by CHUNK_SIZE environment variable
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for cosmosid_cli-2.1.6-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 59fe59db85b7482359aee59f420602fe40990d88b59e913fc0455805f2771163 |
|
MD5 | 2854cfb8beac692c2ea91a0d725129f9 |
|
BLAKE2b-256 | 65a8a066ed561342b05d3b6f2b24bfb124e63492732f63917c6820305e045da9 |