protein-metamorphisms-is 1.4.0

Comprehensive Python Module for Protein Data Management: Designed for streamlined integration and processing of protein information from both UniProt and PDB. Equipped with features for concurrent data fetching, robust error handling, and database synchronization.

Exploration of “Metamorphism” and “Multi-functionality” in Proteins

💡 This study focuses on exploring phenomena of metamorphism and multifunctionality in proteins, fundamental aspects for understanding protein evolution and functionality across various biological contexts. We begin with a massive search for protein sequences that exhibit high percentages of identity, indicative of functional conservation across different species. Subsequently, we identify structures that, in addition to meeting this high identity criterion, exhibit significant differences in their spatial configuration, suggesting possible structural metamorphisms.

The main objective is to develop a comprehensive dataset that includes sets of varied structural conformations, providing a solid basis for comparative and evolutionary structural analysis. We implement an initial clustering strategy using the CD-HIT algorithm, which groups sequences based on their similarity. The resulting groups are then re-clustered using a structural embedding generation model based on the ID3 alphabet. This clustering focuses on finding within a homologous set of proteins those with greater structural differences, allowing a detailed analysis of their three-dimensional differences.

We have created an environment for analyzing structural distances that enables the analysis of inter-subcluster distances using leading structural alignment and distance algorithms such as Fatcat, US-Align, and CE-Align. High differences in these values are indicative of metamorphism. Concurrently, we employ similarity analysis techniques in Gene Ontology (GO) ontologies to discover proteins that, in addition to their structural conservation, exhibit multifunctionality. This includes an analysis of semantic distances per protein to identify disparate terms indicating multifunctionality.

🚀 FANTASIA Support: This tool also supports FANTASIA, an extended system for functional annotation and task orchestration. FANTASIA comes with its own documentation and streamlined templates, ensuring clarity and usability for large-scale annotation workflows.
You can find the detailed documentation here.

prerequisites

• Python 3.11.6
• RabbitMQ
• PostgreSQL with pgVector extension installed.

---

Setup Instructions

1. Install Docker

Ensure Docker is installed on your system. If it’s not, you can download it from here.

2. Set Up PostgreSQL with pgvector

Run the following command to start a PostgreSQL container with the pgvector extension:

docker run -d --name pgvectorsql \
    -e POSTGRES_USER=usuario \
    -e POSTGRES_PASSWORD=clave \
    -e POSTGRES_DB=BioData \
    -p 5432:5432 \
    pgvector/pgvector:pg16

Once the container is running, connect to the database and enable the vector extension:

CREATE EXTENSION IF NOT EXISTS vector;

3. Set Up RabbitMQ

Start a RabbitMQ container using the command below:

docker run -d --name rabbitmq \
    -p 15672:15672 \
    -p 5672:5672 \
    rabbitmq:management

4. Install Poetry

Poetry is used for managing dependencies. Install it by following the official instructions here.

5. Clone the Repository

Clone the repository and navigate into its directory:

git clone https://github.com/CBBIO/protein-metamorphisms-is.git
cd protein-metamorphisms-is

6. Install Dependencies

Use Poetry to install the project dependencies:

poetry install

Configuration Parameters and Constants

Configuration Parameters (config.yaml)

The main config.yaml file defines various settings used across the application, including system parameters, database configuration, and task-specific settings, pipelines such as FANTASIA contains a reduced template instance of this file on its own directory.

• System Configuration:

  • max_workers: Specifies the maximum number of worker threads that the system can use. This controls the concurrency level for processing tasks.
  • binaries_path: Defines the relative path where binary files required by the application are stored.

• Database Configuration:

  • DB_USERNAME: The username used to connect to the PostgreSQL database.
  • DB_PASSWORD: The password associated with the PostgreSQL user.
  • DB_HOST: The hostname of the PostgreSQL server (e.g., localhost).
  • DB_PORT: The port on which the PostgreSQL server is running (default: 5432).
  • DB_NAME: The name of the PostgreSQL database where data is stored.

• RabbitMQ Configuration:

  • rabbitmq_host: The hostname of the RabbitMQ server.
  • rabbitmq_user: The username used to authenticate with RabbitMQ.
  • rabbitmq_password: The password associated with the RabbitMQ user.

• Information System:

  • load_accesion_csv: Path to the CSV file containing accession codes to be loaded into the system.
  • load_accesion_column: The column name in the CSV file that contains the accession IDs.
  • tag: A tag to be associated with the accession data for identification purposes.
  • max_attempts: Maximum number of attempts the system should make to process a given task before failing.
  • search_criteria: Criteria used to search for relevant data, typically in the format of a query string.
  • limit: The maximum number of results to return from a search query.

• PDB Extraction:

  • resolution_threshold: Maximum resolution (in Ångströms) for selecting PDB structures.
  • server: URL of the PDB server to download structure files.
  • data_directory: Path where PDB data files are stored.
  • file_format: Format of the PDB files (e.g., mmCif).
  • allow_multiple_chain_models: Boolean flag indicating whether to allow multiple chain models, typically set to False for NMR data.

• Operations:

  • constants: Path to the constants.yaml file containing various constant definitions used across the system.

• Sequence Clustering:

  • fasta_path: Path where the FASTA file containing sequences is saved.
  • cdhit_out_path: Path where CD-HIT output files are stored.
  • sequence_identity_threshold: Threshold for sequence identity used by CD-HIT to determine clusters.
  • alignment_coverage: Minimum alignment coverage required for sequences to be considered similar.
  • memory_usage: Maximum amount of memory (in MB) that CD-HIT is allowed to use.
  • most_representative_search: Boolean flag to enable or disable searching for the most representative sequence in each cluster.

• Structural Alignment:

  • structural_alignment: Contains settings for structural alignment tasks.
    • types: List of integers representing the types of alignments to perform.
    • retry_timeout: Time (in seconds) to wait before retrying a failed task.
    • retry_count: Maximum number of retry attempts for a failed task.
    • batch_size: Number of structures to process in a single batch.
    • task_timeout: Time (in seconds) to allow for the completion of a task before it times out.

• Embedding:

  • embedding: Contains settings for generating embeddings from sequences.
    • types: List of integers representing the types of embeddings to generate (e.g., ESM, Prost).
    • batch_size: Number of sequences to process in a single batch.

• GO Metrics:

  • obo: Path to the GO ontology file in OBO format.
  • go_annotation_file: Path to the file containing GO annotations.
  • allowed_evidences: List of evidence codes allowed when processing GO annotations.
  • k: Number of nearest neighbors to consider when predicting GO terms.

---

Constants (constants.yaml)

The constants.yaml file defines constant values used throughout the system, including types of structural alignments, complexity levels, embedding types, and prediction methods.

• Structural Alignment Types:

  • name: The name of the alignment method (e.g., "CE-align", "US-align").
  • description: A detailed explanation of the alignment method and its applications.
  • task_name: The internal name used by the system to reference the alignment task.

• Structural Complexity Levels:

  • name: The name representing the level of structural complexity (e.g., "Protein Chains", "Secondary Structures").
  • description: A description of what each complexity level represents in terms of protein structure.

• Sequence Embedding Types:

  • name: The name of the embedding model (e.g., "ESM", "Prost-T5").
  • description: A brief overview of the embedding model and its intended use.
  • task_name: The internal name used to reference the embedding task.
  • model_name: The specific model identifier used for generating embeddings.

• Structure Embedding Types:

  • name: The name of the structural embedding type (e.g., "3di").
  • description: A description of how the embedding type captures 3D structural information.
  • task_name: The internal name used for the embedding task.
  • model_name: The specific model identifier used for generating structural embeddings.

• Prediction Methods:

  • name: The name of the prediction method (e.g., "Cosine Similarity").
  • description: A description of how the prediction method works, particularly how it measures similarity between embeddings.

---

Get started:

To execute the full process chain, simply run:

python main.py

This command will trigger the complete workflow, starting from the initial data preprocessing stages and continuing through to the final analysis and output generation.

Customizing the Workflow:

You can customize the sequence of tasks executed by modifying main.py or adjusting the relevant parameters in the config.yaml file. This allows you to tailor the process flow to meet specific research needs or to experiment with different methodologies.