ndexstringloader 0.2.2

Loads STRING data into NDEx

NDEx STRING Content Loader

Python application for loading STRING data into NDEx.

This tool downloads and unpacks the STRING files below

9606.protein.links.full.v11.0.txt.gz

human.name_2_string.tsv.gz

human.entrez_2_string.2018.tsv.gz

human.uniprot_2_string.2018.tsv.gz

generates a new tsv file, transforms it to CX, and uploads it to NDEx server. Duplicate edges (edges that have the same Source and Target nodes and the same value of combined_score) are included to the generated tsv and CX files only once. Name of the newly generated network includes the value of cutoffscore argument, for example, STRING - Human Protein Links - High Confidence (Score >= 0.7). In case user didn’t specify --update UUID argument, then the network with this name gets over-written in case if already exists on NDEx server; otherwise, a new network is created. Specifying --update UUID command line argument will over-write network with this UUID if it is found. If not, then user is asked if (s)he wants to create a new network. When network is updated, only edges and nodes are changed; network attributes are not modified.

1) Below is an example of a record from 9606.protein.links.full.v11.0.txt.gz

9606.ENSP00000261819 9606.ENSP00000353549 0 0 0 0 0 102 90 987 260 900 0 754 622 999

To generate a STRING network, the loader reads rows from that file one by one and compares the value of the last column combined_score with the value cutoffscore argument. The row is not added to the network generated in case combined_score is less than the commad-line argument cutoffscore.

2) If combined_score is no than less cutoffscore, the loader process two first columns

column 1 - protein1 (9606.ENSP00000261819)
column 2 - protein2 (9606.ENSP00000353549)

When processing first column protein1, the script

replaces Ensembl Id with a display name, for example 9606.ENSP00000261819 becomes ANAPC5. Mapping of display names to Enseml Ids is found in human.name_2_string.tsv.gz

uses human.uniprot_2_string.2018.tsv.gz to create represents value. For example, represents for 9606.ENSP00000261819 is uniprot:Q9UJX4

uses human.entrez_2_string.2018.tsv.gz to create list of aliases for the current protein. Thus, list of aliases for 9606.ENSP00000261819 is ncbigene:51433|ensembl:ENSP00000261819

3) The second column protein2 is processed the same way as column 1.

4) In the generated tsv file 9606.protein.links.tsv, protein1 and protein2 values from the original file are replaced with

protein_display_name_1 represents_1 alias_1 protein_display_name_2 represents_2 alias_2

So, the original

9606.ENSP00000261819 9606.ENSP00000353549 0 0 0 0 0 102 90 987 260 900 0 754 622 999

becomes

ANAPC5 uniprot:Q9UJX4 ncbigene:51433|ensembl:ENSP00000261819 CDC16 uniprot:Q13042  ncbigene:8881|ensembl:ENSP00000353549 0 0 0 0 0 102 90 987 260 900 0 754 622 999

5) The generated tsv file 9606.protein.links.tsv is then transformed to CX 9606.protein.links.cx. The default style defined in style.cx distributed with this loader is applied to the generated network in case neither --style nor --template is specified. User can specify style template file with either --style argument or style template network UUID --template UUID_of_style_template_network. Specifying both --template and --style is not allowed.

6) 9606.protein.links.cx is then uploaded to NDEx server either replacing an existing network (in case --update UUID is specified or network with this name already exists), or creating a new network.

Dependencies

• ndex2

• ndexutil

Compatibility

• Python 3.3+

Installation

git clone https://github.com/ndexcontent/ndexstringloader
cd ndexstringloader
make dist
pip install dist/ndexloadstring*whl

Run make command with no arguments to see other build/deploy options including creation of Docker image

make

Output:

clean                remove all build, test, coverage and Python artifacts
clean-build          remove build artifacts
clean-pyc            remove Python file artifacts
clean-test           remove test and coverage artifacts
lint                 check style with flake8
test                 run tests quickly with the default Python
test-all             run tests on every Python version with tox
coverage             check code coverage quickly with the default Python
docs                 generate Sphinx HTML documentation, including API docs
servedocs            compile the docs watching for changes
testrelease          package and upload a TEST release
release              package and upload a release
dist                 builds source and wheel package
install              install the package to the active Python's site-packages
dockerbuild          build docker image and store in local repository
dockerpush           push image to dockerhub

Configuration

The ndexloadstring.py requires a configuration file to be created. The default path for this configuration is ~/.ndexutils.conf but can be overridden with --conf flag.

Configuration file

Networks listed in [network_ids] section need to be visible to the user

[ndexstringloader]
user = joe123
password = somepassword123
server = dev.ndexbio.org

Needed files

Load plan is required for running this script. string_plan.json found at ndexstringloader/ndexstringloader can be used for this purpose.

Usage

For information invoke ndexloadstring.py -h

Example usage

Here is how this command can be run for dev and prod targets:

ndexloadstring.py --profile dev tmpdir/

ndexloadstring.py --profile prod tmpdir/

Via Docker

Example usage

TODO: Add information about example usage

docker run -v `pwd`:`pwd` -w `pwd` vrynkov/ndexstringloader:0.1.0 ndexloadstring.py --conf conf # TODO Add other needed arguments here

Credits

This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.

History

0.2.2 (2019-09-12)

• Added new featured specified by UD-577 Quick improvement for new String loader (added optional –update argument that allows to specify the UUID of a target network to update; added optional –template argument that allows to specify the UUID of a target network to use as style template, the update operation now only changes nodes and edges, but leaves network properties untouched).

0.2.1 (2019-08-23)

• Improved README file.

• Added new JUnit tests (JUnit test coverage is 87%).

0.2.0 (2019-07-26)

• Removed duplicate edges. Every pair of connected nodes in STRING networks had the same edge duplicated (one edge going from A to B, and another going from B to A). Since edges in STRING are not directed, we can safely remove half of them.

• Added new arguments to command line:

  optional –cutoffscore (default is 0.7) - used to filter on combined_score column. To include edges with combined_score of 800 or higher, –cutoffscore 0.8 should be specified

  required –datadir specifies a working directory where STRING files will be downloaded to and processed style.cx file that contains style is supplied with the STRING loader and used by default. It can be overwritten with –style argument.

0.1.0 (2019-03-13)

• First release on PyPI.