MtbDb 0.1.1

MtbDb: Genomics Database Toolkit for Mycobacterium Tuberculosis.

MtbDb: Genomics Database Toolkit for Mycobacterium Tuberculosis

🇬🇧 English | 🇨🇳 简体中文

MtbDb is a Python toolkit specifically designed for Mycobacterium tuberculosis genomics analysis, providing efficient database querying and analysis capabilities for TB genomic data.

Features

• 🧬 TB genomic database queries
• 📊 Gene Variant Analysis (TBVA)
• 🔍 Gene Expression Analysis (TBGEA)
• 📈 Statistical analysis and visualization
• 🔗 Support for multiple data formats
• 🧪 FASTA/FASTQ/SNP to VCF conversion
• 📦 Built-in H37Rv reference genome (ready to use)

Installation

Quick Start: Using Conda (Highly Recommended)

If you need FASTA/FASTQ to VCF conversion features, we recommend using Conda, which automatically configures all Python packages and bioinformatics tools:

# Clone the repository
git clone https://github.com/16627517673/mtbdb.git
cd mtbdb

# Create and activate conda environment
conda env create -f environment.yml
conda activate mtbdb

# Verify installation
python -c "from mtbdb.tbva import check_all_dependencies; check_all_dependencies()"

Python Package Only (Without External Tools)

If you only need VCF parsing and annotation features, you can install via pip alone:

Install from PyPI

pip install MtbDb

Install from TestPyPI

pip install -i https://test.pypi.org/simple/ MtbDb

Install from Source

git clone https://github.com/16627517673/mtbdb.git
cd mtbdb
pip install -e .

Environment Configuration and Dependencies

Python Dependencies

• Python >= 3.8
• intervaltree >= 3.0.2
• numpy >= 1.20.0
• pandas >= 1.2.0
• scipy >= 1.6.0
• statsmodels >= 0.12.0

External Bioinformatics Tools (Required for FASTA/FASTQ to VCF)

⚠️ Important: The following tools are system binaries and cannot be installed via pip

Tools Required for FASTA to VCF

• MUMmer Suite (nucmer, delta-filter, show-snps)
  • For whole genome alignment and SNP extraction

Tools Required for FASTQ to VCF

• bwa: Short reads alignment
• samtools: SAM/BAM file processing
• varscan: Variant calling
• fastp: Quality control
• fastq-dump (optional): SRA file extraction

Installation Methods

Option 1: Using Conda (Recommended)

# Method A: Using provided environment configuration
conda env create -f environment.yml
conda activate mtbdb

# Method B: Manual installation
conda install -c bioconda mummer bwa samtools varscan fastp sra-tools

Option 2: Using Homebrew (macOS)

brew install brewsci/bio/mummer bwa samtools fastp
# varscan needs separate JAR file download

Option 3: Build from Source

• Refer to each tool's official documentation

Verify Tool Installation

from mtbdb.tbva import check_all_dependencies
check_all_dependencies()

Example output:

Checking MtbDb external tool dependencies...

【FASTA to VCF Tools】
  nucmer: ✓ Installed 4.0.0beta2
  delta-filter: ✓ Installed
  show-snps: ✓ Installed

【FASTQ to VCF Tools】
  bwa: ✓ Installed 0.7.17
  samtools: ✓ Installed 1.15
  varscan: ✓ Installed 2.4.4
  fastp: ✓ Installed 0.23.2
  fastq-dump: ✓ Installed (optional)

Summary: 8/8 tools available
✓ All required tools are properly installed!

Quick Start

Built-in Reference Genome

MtbDb includes the standard H37Rv reference genome (NC_000962.3, 4.4MB) with all necessary index files:

• H37Rv complete genome sequence (.fa)
• BWA index files (.bwt, .pac, .sa, .amb, .ann)
• samtools index (.fai)

Advantages:

• ✅ No manual download or configuration needed
• ✅ Ready to use with one line of code
• ✅ Custom reference genomes also supported

from mtbdb.tbva import get_default_reference_fasta

# View built-in reference genome path
ref_path = get_default_reference_fasta()
print(f"Built-in reference genome: {ref_path}")
# Output: /path/to/mtbdb/data/reference/H37Rv_complete_genome.fa

Basic VCF Parsing and Annotation

from mtbdb.tbva import VCFParser, VariantAnnotator

# Parse VCF file
parser = VCFParser('sample.vcf', filter_pass=True)
snps = parser.get_snps()
indels = parser.get_indels()

# Variant annotation
annotator = VariantAnnotator('sample.vcf')
annotated_variants = annotator.annotate()

FASTA to VCF (Comparative Genomics)

Compare two genomes and generate VCF file using MUMmer toolkit:

from mtbdb.tbva import fasta_to_vcf

# Method 1: Using built-in H37Rv reference (simplest)
vcf_file = fasta_to_vcf(
    query_fasta='sample_genome.fasta',
    output_vcf='output/sample.vcf'
)

# Method 2: Using custom reference genome
vcf_file = fasta_to_vcf(
    query_fasta='sample_genome.fasta',
    output_vcf='output/sample.vcf',
    reference_fasta='custom_reference.fasta',
    chrom_id='custom_chrom_id'
)

print(f"VCF file generated: {vcf_file}")

Prerequisites: MUMmer toolkit (see Environment Configuration section)

FASTQ to VCF (Re-sequencing Analysis)

Detect variants from FASTQ sequencing data and generate VCF file:

from mtbdb.tbva import fastq_to_vcf

# Method 1: Using built-in H37Rv reference (simplest)
vcf_file = fastq_to_vcf(
    output_dir='output',
    sample_name='ERR181314',
    fastq1='ERR181314_1.fastq.gz',
    fastq2='ERR181314_2.fastq.gz',
    threads=8
)

# Method 2: Using custom reference genome
vcf_file = fastq_to_vcf(
    output_dir='output',
    sample_name='sample001',
    fastq1='sample_R1.fastq',
    fastq2='sample_R2.fastq',
    reference_fasta='custom_reference.fa',
    threads=16
)

print(f"VCF file generated: {vcf_file}")

Prerequisites: bwa, samtools, fastp, varscan, etc. (see Environment Configuration section)

SNP to VCF (Existing show-snps Output)

If you already have SNP file generated by MUMmer show-snps, convert directly to VCF format:

from mtbdb.tbva import snp_to_vcf

# Basic usage: Simplest conversion (using H37Rv defaults)
vcf_file = snp_to_vcf(
    snp_file='sample.snps',
    output_vcf='sample.vcf'
)

# Full usage: Custom chromosome ID and reference info
vcf_file = snp_to_vcf(
    snp_file='sample.snps',
    output_vcf='sample.vcf',
    chrom_id='NC_000962.3',
    reference_name='H37Rv_genome.fasta',
    filter_indels=True  # Filter indels, keep only SNPs
)

print(f"VCF file generated: {vcf_file}")

Advantages:

• ✅ No external tool dependencies (pure Python)
• ✅ Fast conversion (seconds)
• ✅ Suitable for existing show-snps output
• ✅ Configurable chromosome ID and reference info

Use Cases:

• Already ran MUMmer alignment via other methods
• Need to convert SNP data to standard VCF format
• Integration with other VCF analysis tools

Complete Analysis Pipeline Example

Complete workflow from FASTQ raw data to variant annotation:

from mtbdb.tbva import fastq_to_vcf, VCFParser, VariantAnnotator

# Step 1: Generate VCF (using built-in H37Rv reference)
vcf_file = fastq_to_vcf(
    output_dir='analysis',
    sample_name='sample001',
    fastq1='sample001_R1.fastq',
    fastq2='sample001_R2.fastq',
    threads=16
)

# Step 2: Parse VCF
parser = VCFParser(vcf_file)
snps = parser.get_snps()
print(f"Detected {len(snps)} SNP variants")

# Step 3: Variant annotation
annotator = VariantAnnotator(vcf_file)
results = annotator.annotate()

Configure Reference Genome and Tool Paths

from mtbdb.tbva import set_reference_fasta, set_tool_path

# Set default reference genome
set_reference_fasta('/path/to/H37Rv_complete_genome.fa')

# Set external tool paths (if not in system PATH)
set_tool_path('bwa', '/usr/local/bin/bwa')
set_tool_path('samtools', '/usr/local/bin/samtools')
set_tool_path('nucmer', '/opt/mummer/bin/nucmer')

Or configure via environment variables:

export MTBDB_REFERENCE_FASTA=/path/to/H37Rv.fa
export MTBDB_BWA_PATH=/usr/local/bin/bwa
export MTBDB_SAMTOOLS_PATH=/usr/local/bin/samtools

License

This project is licensed under the Apache License 2.0. See LICENSE file for details.

Author

HengyuZhou ([e-mail])

Homepage

https://github.com/16627517673/mtbdb