tipp3 0.1

TIPP3 - A Phylogeney-Based Abundance Profiling Tool

Developer:

Chengze Shen

Table of contents

• News

• Method Overview

  • Note and Acknowledgment

• Installation

  • External requirements

  • TIPP3 reference package

    • Download precompiled refpkg

    • Create customized refpkg

  • Install with PyPI (pip)

  • Install from source files

    • Requirements

    • Installation Steps

  • main.config

  • User-specified config file

• Usage

  • Subcommand abundance

  • Subcommand download_refpkg

  • Examples

    • Scenario 1

    • Scenario 2

    • Scenario 3

• TODO list

News

• 3.6.2025 - Users can directly download the latest TIPP3 reference package by running the subcommand run_tipp3.py download_refpkg.

• 3.2.2025 - TIPP3 is now accepted at PLOS Computational Biology and will soon be publicly available!

• 1.20.2025 - TIPP3 now is feature complete for abundance profiling, for both the more accurate TIPP3 mode or the fast TIPP3-fast mode. By default, TIPP3-fast is used.

Method Overview

TIPP3 is a metagenomic profiling method that solves the following problems:

Taxonomic identification

• Input: A query read q

• Output: The taxonomic lineage of q (if identified)

Abundance profiling

• Input: A set Q of query reads

• Output: An abundance profile estimated on Q

TIPP3 continues the TIPP-family methods (prior methods: TIPP and TIPP2), which use a marker gene database to identify the taxonomic lineage of input reads (if the read comes from a marker gene). See the pipeline below for the TIPP3 workflow.

Publication(s)
(TIPP3) Shen, Chengze, Eleanor Wedell, Mihai Pop, and Tandy Warnow, “TIPP3 and TIPP3-fast: improved abundance profiling in metagenomics.” Accepted at PLOS Computational biology.
(TIPP2) Nguyen, Nam, Siavash Mirarab, Bo Liu, Mihai Pop, and Tandy Warnow, “TIPP: Taxonomic identification and phylogenetic profiling.” Bioinformatics, 2014. https://doi.org/10.1093/bioinformatics/btu721
(TIPP) Shah, Nidhi, Erin K. Molloy, Mihai Pop, and Tandy Warnow, “TIPP2: metagenomic taxonomic profiling using phylogenetic markers.” Bioinformatics, 2020. https://doi.org/10.1093/bioinformatics/btab023

Note and Acknowledgment

TIPP3 includes and uses:

1. pplacer (v1.1.alpha19).

2. Batch-SCAMPP (v1.0.0).

Installation

TIPP3 was tested on Python 3.7 to 3.12.

There are two ways to install and use TIPP3: with PyPI (pip install) or directly with this GitHub repository. If you have any difficulties installing or running TIPP3, please contact Chengze Shen ([e-mail]).

External requirements

BLAST is a hard requirement to run TIPP3. The software will automatically look for blastn in the $PATH environment variable. If you have not installed BLAST, you can find the latest version from https://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/LATEST/.

TIPP3 reference package

Download precompiled refpkg

At the time, you can download the TIPP3 reference package from https://databank.illinois.edu/datasets/IDB-4931852, hosted on the Illinois Data Bank. You can also download the latest version using run_tipp3.py download_refpkg. Once downloaded, unzip the file and please see Examples and Usage for referring to the reference package.

Create customized refpkg

If you would like to create a customized TIPP3 reference package, please refer to this Wiki page for the pipeline to do so.

Install with PyPI (pip)

The easiest way to install TIPP3 is to use the PyPI distribution.

# 1. Install with pip (--user if no root access)
pip install tipp3 [--user]

# 2. Three binary executables will be installed. The first time running
#    any of the binaries will create the TIPP3 config file at
#    ~/.tipp3/main.config
tipp3 [-h]           # (recommended) preset "TIPP3-fast" for abundance profiling
tipp3-accurate [-h]  # preset "TIPP3" for abundance profiling
run_tipp3.py [-h]    # see other options

Install from source files

Requirements

python>=3.7
configparser>=5.0.0
DendroPy>=4.5.2
numpy>=1.21.6
psutil>=5.0.0
setuptools>=60.0.0
treeswift>=1.1.28
witch-msa>=1.0.7
lz4>=4.3.2

Installation Steps

# 1. Install via GitHub repo
git clone https://github.com/c5shen/TIPP3.git

# 2. Install all requirements
pip3 install -r requirements.txt

# 3. Execute run_tipp3.py executable for the first time with "-h" to see
#    allowed commandline parameters and example usages
#    Running TIPP3 for the first time will also create the main config
#    file at "~/.tipp3/main.config", which stores the default behavior
#    for running TIPP3 (including all binary executable paths)
python3 run_tipp3.py [-h]

main.config

main.config file will be created the first time running TIPP3 at the user root directory (~/.tipp3/main.config). This file stores the default behavior for running TIPP3 and the paths to all binary executables that TIPP3 need to use.

User-specified config file

In addition, a user can specify a customized config file with -c or --config-file parameter option when running TIPP3 for abundance profiling (e.g., run_tipp3.py abundance -c user.config). The user.config file will override settings from main.config (if overlaps). Command-line arguments still have the highest priority and will override both config files, if any parameters overlap.

Usage

Subcommand abundance

The general command to run TIPP3 for abundance profiling is listed below. By default, preset “TIPP3-fast” is run, which is significantly faster than the more accurate TIPP3 mode. See Examples below for how to customize the TIPP3 pipeline.

# (Optional) change the logging level to DEBUG for more verbose logging
export TIPP_LOGGING_LEVEL=debug

# TIPP3 supports the following formats for "-i [query reads]"
# XXX.fasta[.gz, .gzip]
# XXX.fa[.gz, .gzip]
# XXX.fastq[.gz, .gzip]
# XXX.fq[.gz, .gzip]

python3 run_tipp3.py abundance -r [reference package path] -i [query reads] -d [output directory]

Subcommand download_refpkg

Users can also directly download the latest version of the TIPP3 reference package using the subcommand run_tipp3.py download_refpkg.

# download tipp3 refpkg to current directory and decompress
python3 run_tipp3.py download_refpkg -o ./ --decompress

Examples

Some examples of TIPP3 usage can be found at the bottom of the help text running:

python3 run_tipp3.py -h

All of the following examples can be found in the examples/run.sh bash script, with example data stored under examples/data. The default example data used is a small set of Illumina short reads denoted as illumina.small.queries.fasta.

Scenario 1

(TIPP3-fast) Use BLAST for query alignment, and Batch-SCAMPP (bscampp) for query placement.

python3 run_tipp3.py abundance -i examples/illumina.small.queries.fasta \
   --reference-package [reference package dir] --outdir tipp3_scenario1 \
   --alignment-method blast --placement-method bscampp \
   -t 16

Scenario 2

Use BLAST for query alignment, and pplacer with the taxtastic package for query placement (pplacer-taxtastic).

python3 run_tipp3.py abundance -i examples/illumina.small.queries.fasta \
   --reference-package [reference package dir] --outdir tipp3_scenario1 \
   --alignment-method blast --placement-method pplacer-taxtastic \
   -t 16

Scenario 3

(TIPP3) Use WITCH for query alignment, and pplacer-taxtastic for query placement. Keep all temporary files during the run.

python3 run_tipp3.py abundance -i examples/illumina.small.queries.fasta \
   --reference-package [reference package dir] --outdir tipp3_scenario1 \
   --alignment-method witch --placement-method pplacer-taxtastic \
   -t 16 --keeptemp

TODO list

• 1.26.2025 - Add a parameter option to allow users to set the support value for abundance profiling. Currently, the support values are empirically set for different placement methods (90% for pplacer-taxtastic and 95% for Batch-SCAMPP).