metaMuseomics

Python tools and modules for use in metagenomic assembly and analysis of degraded, metagenomic samples common in museum specimen.

Contents

1. Background
2. Getting started
3. metaMuseomics Modules: Use & Details
4. Some extra utility tools
5. Example usage
6. References

Background

There are 3 main modules, plus a suite of supplementary modules that can be used in a comprehensive pipeline. Each module wraps commonly used metagenomics tools into individual chunks that can be used in isolation where needed or as part of an automated pipeline. The main purpose of each module is to have optimised parameters specifically for dealing with hDNA.

A fourth module (binning_module.py) is also included as part of this package, but it has not been tested thoroughly on real data - so proceed with caution knowing there's a likelihood it'll break!

1. fastp_module.py

Performs initial adapter/quality trimming, poly-G trimming, correction and deduplication in a first step. This trimming step outputs separate stats summaries and an overlap summary. The output files of this first step are then used to merge reads, before a last round of summaries are generated.

The poly-G trimmming, adapter, quality, correction and deduplication are all defaults, but further fastp parameters can be included in the first step by the user.

2. decontam_module.py

The decontam module is designed to use references to remove any potential reads sourced from known contaminants. In this particular instance it is set up for PhiX and human read decontamination.

The important point is that PhiX removal happens before human decontamination, and the two stages are separately parallelised across files, so that there are separate outputs for each stage.

3. assembly_module.py

This is substantially larger than the other two modules: it combines assembly (with 3 assembler options: MEGAHIT, MetaSPADEs, IDBA-UD), validation/restarts, optional metaMIC correction, BUSCO assessment and SeqFu statistics. In essence assembly_module.py is really three pipelines in one: assembly → optional correction → evaluation/reporting.

In the first step, assembly happens with the chosen assembler. IDBA-UD first converts paired FASTQs into an interleaved FASTA using fq2fa; merged FASTQ can also be converted to FASTA. MEGAHIT supports three modes in the wrapper: merged reads only, paired reads only, or merged + paired reads.

Assemblies are then checked with check_assemblies(). This is particularly important because it acts as the gatekeeper before downstream correction/QC: it collects only assemblies with the expected contig file. IDBA-UD has a special fallback to scaffold.fa, whereas MEGAHIT and MetaSPAdes are restarted if their expected output is missing.

A correction strategy can then be implemented (with the flag --correction), using metaMIC, that utilises a fallback for contigs generated <1000 bp:

Assembly
   │
   ▼
BWA map reads → contigs
   │
   ▼
SAMtools filtering/sorting
   │
   ▼
SAMtools mpileup + AWK
   │
   ▼
metaMIC extract_feature
   │
   ▼
metaMIC predict
   │
   ├── success → corrected_contigs.fa
   │
   └── failure
         │
         ▼
   split contigs at 1000 bp
       ├── >=1000 bp → metaMIC
       └── <1000 bp → retain unchanged
              │
              ▼
       concatenate corrected + short

Finally, assemblies are assessed using BUSCO. BUSCO uses the selected lineage, defaults to fungi_odb10, runs in genome mode with --metaeuk, and is parallelised across assemblies. The resulting JSON files are then converted into "busco_summary.csv".

SeqFu is run once across all successful contig FASTAs with GC and CSV output enabled.

Note: The assembly module uses process-level parallelism, whereas the preprocessing/decontamination modules use thread pools. This is mainly due to the relatively heavyweight assembler/BUSCO processes versus the lighter orchestration around external commands.

4. binning_module.py (in development)

This module is a parallel, per-sample metagenomic binning pipeline.

For each sample, it:

1. Finds the expected assembly file based on the selected assembler.
2. Finds paired reads and maps them to the assembly with BWA.
3. Generates BAM and pileup coverage files using Samtools.
4. Runs any or all selected binners:
   • MetaDecoder
   • MetaBAT2
   • CONCOCT
   • MaxBin2
   • SemiBin2
5. Converts bin FASTA files into contig-to-bin tables.
6. Runs DAS Tool to optimise binning results if multiple binners are used.

It has not been tested thoroughly on real data - so proceed with caution knowing there's a likelihood it'll break!

Getting started

Installation

There are two installation options:

Option A) To install the python package only (assuming you have bioinformatics software already installed) run:

python3.12 -m pip install metaMuseomics

Option B) To install a complete conda environment, run:

conda env create -f environment.yml
conda activate metamuseomics

If you are planning on using the decontaminate module, then run the following in the directory above your working directory to download the reference sequences (HG38 Homo sapiens reference genome GRCh38.p14 (GCF_000001405.40) & PhiX NC_001422.1) used:

mkdir ref
cd ./ref

# PhiX
wget ftp://ftp.ncbi.nlm.nih.gov/genomes/genbank/viral/Sinsheimervirus_phiX174/latest_assembly_versions/GCA_000819615.1_ViralProj14015/GCA_000819615.1_ViralProj14015_genomic.fna.gz

# Homo sapiens 
wget https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/001/405/GCF_000001405.40_GRCh38.p14/GCF_000001405.40_GRCh38.p14_genomic.fna.gz
gunzip GCF_000001405.40_GRCh38.p14_genomic.fna.gz
bwa index GCF_000001405.40_GRCh38.p14_genomic.fna

Alternatively, you can specify your reference sequences using the arguments --phiX_ref and --human-ref.

In theory any reference can be used in place of either, however you should keep in mind that as the PhiX genome is so small, bbduk.sh from BBTools is used, while BWA MEM is used for the much larger (and more resource intense) --human-ref.

Important notes on certain packages and use cases:

IDBA-UD: IDBA-UD may throw an error in the presence of short insert sizes. A solution to this has been made available here.

metaMIC metaMIC is not a conda/bioconda package and requires it's own installation steps. Please consult their how-to guide for instructions on this. The use of metaMIC is an optional part of the assembly module activated with the flag --correction. This can be skipped if rounds of correction are not neccessary/wanted.

metaMuseomics Modules

The modules are designed to be used either as independent tools or as part of a metagenomic pipeline for historical (maybe museum-derived) data, as shown below.

Extra utility tools and wrappers

Example usage

Each module and script has it's own parameters. This is an example of basic usage:

Example usage: 
# Download raw data from sequencing company and flatten directory structure
mm-setup-library PRJXXXXX ./download-links.csv --url_id raw_data


# Run FastQC on all samples to check the libraries
cd ./PRJXXXXX
mm-fastqc --input_dir raw_data --output_dir fastqc_out

# QC, trim and merge the data using the fastp module
mm-fastp --input raw_data --output_dir fastp_processed --tracking_sheet Library_tracking_sheet.csv --prefix IDT --column_name Sample_ID --fastp_extra_args 

# Assess library complexity using reference-free nonpareil
mm-nonpareil -i fastp_processed -o nonpareil_out/ -R 100000

# Decontaminate data using the 
mm-decontam --input_dir fastp_processed --output_dir decontaminated_dir --suffix merged --merged

# Assemble the data
mm-assembly --input_dir decontaminated_dir --merged --assembler megahit --output_dir megahit_out --correction --merged_suffix merged --paired_suffix trimmed

# Classify output scaffolds
mm-kraken2 --input_dir ./megahit_out/ --filename final.contigs.fa --kraken_db ./fungi_db/ --output_dir ./kraken2_out_fungaldb/ --workers 8

References

• Simon Andrews, 2010. FastQC: A Quality Control Tool for High Throughput Sequence Data [Online]. Available online at: http://www.bioinformatics.babraham.ac.uk/projects/fastqc/
• Shifu Chen, 2023. Ultrafast one-pass FASTQ data preprocessing, quality control, and deduplication using fastp. iMeta 2: e107. https://doi.org/10.1002/imt2.107
• Shifu Chen, Yanqing Zhou, Yaru Chen, Jia Gu, 2018. fastp: an ultra-fast all-in-one FASTQ preprocessor, Bioinformatics, 34, 17, i884–i890. https://doi.org/10.1093/bioinformatics/bty560
• Bushnell B. – sourceforge.net/projects/bbmap/
• Heng Li, Richard Durbin, 2009. Fast and accurate short read alignment with Burrows–Wheeler transform, Bioinformatics, 25, 14, 1754–1760, https://doi.org/10.1093/bioinformatics/btp324
• Heng Li, 2013 Aligning sequence reads, clone sequences and assembly contigs with BWA-MEM. arXiv:1303.3997v2 [q-bio.GN].
• Li, D., Liu, C-M., Luo, R., Sadakane, K., and Lam, T-W., 2015. MEGAHIT: An ultra-fast single-node solution for large and complex metagenomics assembly via succinct de Bruijn graph. Bioinformatics, doi: 10.1093/bioinformatics/btv033 [PMID: 25609793].
• Li, D., Luo, R., Liu, C.M., Leung, C.M., Ting, H.F., Sadakane, K., Yamashita, H. and Lam, T.W., 2016. MEGAHIT v1.0: A Fast and Scalable Metagenome Assembler driven by Advanced Methodologies and Community Practices. Methods.
• Nurk S, Meleshko D, Korobeynikov A, Pevzner PA, 2017. metaSPAdes: a new versatile metagenomic assembler. Genome Res.;27(5):824-834. doi: 10.1101/gr.213959.116. Epub 2017 Mar 15. PMID: 28298430; PMCID: PMC5411777.
• Yu Peng, Henry C. M. Leung, S. M. Yiu, Francis Y. L. Chin, 2012. IDBA-UD: a de novo assembler for single-cell and metagenomic sequencing data with highly uneven depth, Bioinformatics, 28, 11, 1420–1428, https://doi.org/10.1093/bioinformatics/bts174
• Christian M. K. Sieber, Alexander J. Probst, Allison Sharrar, Brian C. Thomas, Matthias Hess, Susannah G. Tringe & Jillian F. Banfield (2018). Recovery of genomes from metagenomes via a dereplication, aggregation and scoring strategy. Nature Microbiology. https://doi.org/10.1038/s41564-018-0171-1.

MetaMuseomics was written by Maria Kamouyiaros @ NHMUK (2025).