Skip to main content

No project description provided

Project description

EFAAR_benchmarking

This library contains functions to build and benchmark a whole-genome perturbative map.

See our bioRxiv paper for details: https://www.biorxiv.org/content/10.1101/2022.12.09.519400v1

Here are the descriptions for the constants used in the code to configure and control various aspects of the map building and benchmarking process:

BENCHMARK_DATA_DIR: The directory path to the benchmark annotations data. It is obtained using the resources module from the importlib package.

BENCHMARK_SOURCES: A list of benchmark sources, including "Reactome", "HuMAP", "CORUM", "SIGNOR", and "StringDB".

RECALL_PERC_THRS: A list of tuples of two floats between 0 and 1 representing the threshold pair (lower threshold, upper threshold) for calculating recall.

RANDOM_SEED: The random seed value used for random number generation for sampling the null distribution.

N_NULL_SAMPLES: The number of null samples used in benchmarking.

MIN_REQ_ENT_CNT: The minimum required number of entities for benchmarking.

Besides above parameters, for each map we build, we utilize distinct constants to indicate the metadata columns for the perturbation, control, and batch information.

Installation:

This package is installable via pip.

pip install efaar_benchmarking

Usage guidance:

First, run notebooks/map_building_benchmarking.ipynb for GWPS, cpg0016, and cpg0021 individually. This process will build each of these maps and report the perturbation signal and biological relationship benchmarks. Afterwards, run notebooks/map_evaluation_comparison.ipynb to explore the constructed maps using the methods presented in our paper. In order for the latter notebook to work, make sure to set the save_results parameter to True in the former notebook.

We've uploaded the 128-dimensional PCA-TVN maps we constructed for GWPS, cpg0016, and cpg0021 to the notebooks/data directory. So, for convenience, one can run notebooks/map_evaluation_comparison.ipynb directly on these uploaded map files if they wish to explore the maps further without running notebooks/map_building_benchmarking.ipynb. See notebooks/data/LICENSE for terms of use for each dataset.

RxRx3 embeddings are available as separate parquet files per plate in the embeddings.tar file, downloadable from https://rxrx3.rxrx.ai/downloads. Note that in this data, all but 733 genes are anonymized.

You have to install GitHub LFS (Large File Storage) to properly utilize the files under notebooks/data and efaar_benchmarking/expression_data in your local clone of the repository.

Contribution guidance:

See CONTRIBUTING.md for code contribution guidance.

If you want to add any new annotation sources, make sure to follow the same format as in the benchmark_annotations folder. Your annotations file needs to be a file with a .txt extension. It is expected to be a comma-separated two column data frame where column names are entity1 and entity2.

References for the relationship annotation sources:

See efaar_benchmarking/benchmark_annotations/LICENSE for terms of use for each source.

Reactome:

Gillespie, M., Jassal, B., Stephan, R., Milacic, M., Rothfels, K., Senff-Ribeiro, A., Griss, J., Sevilla, C., Matthews, L., Gong, C., et al. (2022). The reactome pathway knowledgebase 2022. Nucleic Acids Res. 50, D687–D692. 10.1093/nar/gkab1028.

CORUM:

Giurgiu, M., Reinhard, J., Brauner, B., Dunger-Kaltenbach, I., Fobo, G., Frishman, G., Montrone, C., and Ruepp, A. (2019). CORUM: the comprehensive resource of mammalian protein complexes-2019. Nucleic Acids Res. 47, D559–D563. 10.1093/nar/gky973.

HuMAP:

Drew, K., Wallingford, J.B., and Marcotte, E.M. (2021). hu.MAP 2.0: integration of over 15,000 proteomic experiments builds a global compendium of human multiprotein assemblies. Mol. Syst. Biol. 17, e10016. 10.15252/msb.202010016.

SIGNOR:

Licata, L., Lo Surdo, P., Iannuccelli, M., Palma, A., Micarelli, E., Perfetto, L., Peluso, D., Calderone, A., Castagnoli, L., and Cesareni, G. (2019). SIGNOR 2.0, the SIGnaling Network Open Resource 2.0: 2019 update. Nucleic Acids Research. 10.1093/nar/gkz949.

StringDB:

von Mering C, Jensen LJ, Snel B, Hooper SD, Krupp M, Foglierini M, Jouffre N, Huynen MA, Bork P. STRING: known and predicted protein-protein associations, integrated and transferred across organisms. Nucleic Acids Res. 2005 Jan 1;33(Database issue):D433-7. doi: 10.1093/nar/gki005.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

efaar_benchmarking-1.0.tar.gz (2.0 MB view details)

Uploaded Source

Built Distribution

efaar_benchmarking-1.0-py2.py3-none-any.whl (2.0 MB view details)

Uploaded Python 2 Python 3

File details

Details for the file efaar_benchmarking-1.0.tar.gz.

File metadata

  • Download URL: efaar_benchmarking-1.0.tar.gz
  • Upload date:
  • Size: 2.0 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.9.19

File hashes

Hashes for efaar_benchmarking-1.0.tar.gz
Algorithm Hash digest
SHA256 8dbf094bf4ddb1ca5efd4b7848fba61d61cb0c81a856c88981540a86b5068298
MD5 2116bf477e1a303290fba3d0b75d26e8
BLAKE2b-256 1bdf7da5b7f7cf6648b1ffcd1eaad2dfa594c7f5045d64f12079884f58ddf482

See more details on using hashes here.

File details

Details for the file efaar_benchmarking-1.0-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for efaar_benchmarking-1.0-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 25641286f25db1b39f4f654e5f51a73503465a864993ba6e6f550b72ee42e565
MD5 31601ac3e598293a36c2dbb531c9fbbb
BLAKE2b-256 1961affecde2953fdbcbb4b64a534b8c8c73f0eae5cbe5abb976bd3dec1b9b68

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page