q2-eplacer 0.1.7

Plugin for ASV classifier with deep-learning and biogeography

q2-eplacer

A QIIME 2 plugin developed by Christopher Powers ([e-mail]) that alows for the ePlacer taxonomic classifier to interface with QIIME2.

ePlacer is a taxonomic classification tool that uses deep-learning approaches to incorporate both sequence information and biogeographic information into taxonomic assignment of DNA sequences.

Why use ePlacer

The machine learning architecture of ePlacer enables powerful prediction beyond sequence-only classification tools (e.g. sequence alignment with blast or naive-bayes classifiers) by directly incorporating additional data into the probabalistic estimate of taxonomy, specifically developed for metabarcoding data. This novel applciation of deep-learning is immensely useful, as there can be many cases in metabarcoding data where two reference species have 100% sequence overlap, but distinct geographic ranges. This tool discriminates these cases and provides additional data for downstream taxonomic curation. Due to this, ePlacer provides enhanced interoperability between metabarcoding datasets.

Currently, ePlacer offers pre-trained models for two popular metabarcoding regions: the MiFish and the ecoPrimer, or Riaz, marker gene regions. For these two regions, ePlacer offers the following benefits:

• Interoperability. ePlacer is trained on global datasets, allowing for direct comparison between metabarcoding datasets, regardless of geographic region.
• Portability. ePlacer has pre-trained models available for both MiFish and Riaz marker gene regions containerized and available for out-of-the-box use
• Increased Accuracy. The ePlacer model architecture provides increased accuracy, precision, and recall as compared to blast, Naive-Bayes, or least common ancestor approachers
• Trainability In addition to the two provided barcodes, this code repository provides tools for training new models.

For other barcode regions, there will be significant advantages with the training of new models. If you are interested in training a new model for ePlacer, please do not hesitate to reach out!

Installation instructions

The following instructions are intended to be a starting point and should be replaced when q2-eplacer is ready to share with others. They will enable you to install the most recent development version of q2-eplacer. Remember that release versions should be used for all "real" work (i.e., where you're not testing or prototyping) - if there aren't instructions for installing a release version of this plugin, it is probably not yet intended for use in practice.

Install Prerequisites

Miniconda provides the conda environment and package manager, and is currently the only supported way to install QIIME 2. Follow the instructions for downloading and installing Miniconda.

After installing Miniconda and opening a new terminal, make sure you're running the latest version of conda:

conda update conda

You also need to install the base qiime2 as a conda environment. Follow the install instructions here.

Install q2-eplacer

Next, you will install the ePlacer qiime plugin from pip

pip install q2-eplacer

This will also install all other required dependencies.

Using q2-eplacer

Data preparation

In order to use ePlacer, you must first prep the data prior to installing, including prepping input data and collecting a pre-trained model for inference.

Pre-Trained models

Currently, two pre-trained models are available: : the MiFish and the ecoPrimer, or Riaz, marker gene regions. These are available in a QIIME2-compatible format:

wget https://zenodo.org/records/20835759/files/mifish\ (2).tar.gz
tar -xzf mifish\ (2).tar.gz
wget https://zenodo.org/records/20835759/files/riaz_12SV5.tar.gz
tar -xzf riaz_12SV5.tar.gz

Import the downloaded model using the following:

qiime tools import \
    --type EplacerModel \
    --input-path ./model/ \
    --output-path model.qza \
    --input-format StagedModelDirectoryFormat

Prepping input data.

In addition to the models, users must import their input data properly. Input data formatting requirements may be seen in documentation for the original ePlacer package.

Sequence data

Input sequence data is required in fasta format, which can be imported into QIIME2 formats with the following:

qiime tools import \
    --type "FeatureData[Sequence]" \
    --input-path seqs.fa \
    --output-path seqs.qza

The sequence data should also be aligned, which can be done with the q2-eplacer function:

qiime eplacer align-sequences --i-fasta ./seqs.qza \
    --i-model ./model.qza \
    --o-aligned-sequences ./aligned_seqs.qza \
    --p-threads 8

Count data

Count data must first be converted to a .biom format, then to a .qza format

biom convert -i ./counts.tsv \
    -o ./counts.biom \
    --table-type="OTU table" \
    --to-hdf5
qiime tools import --input-path ./counts.biom \
    --type 'FeatureTable[Frequency]' \
    --input-format BIOMV210Format \
    --output-path ./counts.qza

geoData

The geographic data can be read in as a metadata file, and requires no further transformations.

blast data

Although not used by the machine learning model, blast results are incredible useful for screening the results for mismatches when 100% matches are possible. Thus, a function for running the blast results was also included:

qiime eplacer run-blast \
    --i-fasta ./seqs.qza \
    --i-model ./model.qza \
    --o-blast ./hits.qza \
    --p-threads 8

Note the unaligned sequences were used for blast.

Running the model

Congratulations! You are ready to run ePlacer!

qiime eplacer run-model \
    --i-fasta ./aligned_seqs.qza \
    --i-model ./model.qza \
    --i-blast ./hits.qza \
    --i-counts ./counts.qza \
    --m-geodata-file ./geoData.tsv \
    --o-eplacer-table ./ePlacerAssignment.qza \
    --o-curated-taxonomy ./qiimeAssignmentCurated.qza \
    --o-raw-taxonomy ./qiimeAssignmentRaw.qza
qiime tools export --input-path ../ePlacerAssignment.qza \
    --output-path ../ePlacerAssignment
qiime tools export --input-path ../qiimeAssignmentCurated.qza \
    --output-path ../qiimeAssignmentCurated
qiime tools export --input-path ../qiimeAssignmentRaw.qza \
    --output-path ../qiimeAssignmentRaw

You may notice there are three output files present. This is three different file formats. The first, --o-eplacer-table details the native ePlacer output format described in the ePlacer repository. The second, --o-curated-taxonomy, outputs the curated assignments in QIIME2 compatible format. The third, --o-raw-taxonomy, outputs the raw taxonomic assignments in QIIME2 compatible format.

A special note

As with all other taxonomic assignment tools, all taxonomic assignments should still be manuall curated after assignment. ePlacer exhibits higher accuracy than other tools, but is not perfect.

Training the model

The QIIME2 implementation of ePlacer also supports training new models. File format requirements are detailed in the ePlacer repository.

qiime eplacer train-model \
    --i-fasta ./unalignedSeqs.qza \
    --i-alignedfasta ./alignedSeqs.qza \
    --m-taxonomy-file ./taxonomy.tsv \
    --m-geodata-file ./geoData.tsv \
    --p-num-augments 100 \
    --o-model .toyModel.qza \
    --o-training-stats ./stats.qza \
    --verbose

About

The q2-eplacer Python package was created from a template. To learn more about q2-eplacer, refer to the project website. To learn how to use QIIME 2, refer to the QIIME 2 User Documentation. To learn QIIME 2 plugin development, refer to Developing with QIIME 2.

q2-eplacer is a QIIME 2 community plugin, meaning that it is not necessarily developed and maintained by the developers of QIIME 2. Please be aware that because community plugins are developed by the QIIME 2 developer community, and not necessarily the QIIME 2 developers themselves, some may not be actively maintained or compatible with current release versions of the QIIME 2 distributions. More information on development and support for community plugins can be found here. If you need help with a community plugin, first refer to the project website. If that page doesn't provide information on how to get help, or you need additional help, head to the Community Plugins category on the QIIME 2 Forum where the QIIME 2 developers will do their best to help you.

==============================================================

This repository is a scientific product and is not official communication of the National Oceanic and Atmospheric Administration, or the United States Department of Commerce. All NOAA GitHub project code is provided on an ‘as is’ basis and the user assumes responsibility for its use. Any claims against the Department of Commerce or Department of Commerce bureaus stemming from the use of this GitHub project will be governed by all applicable Federal law. Any reference to specific commercial products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply their endorsement, recommendation or favoring by the Department of Commerce. The Department of Commerce seal and logo, or the seal and logo of a DOC bureau, shall not be used in any manner to imply endorsement of any commercial product or activity by DOC or the United States Government.