Skip to main content

An implementation of a whole exome analysis pipeline using the library Luigi for workflow management.

Project description


An implementation of a whole exome analysis pipeline using Luigi <>_ for workflow management.

.. figure:: :alt: Steps Logo :align: center

This package provides with the implementation of tasks for executing partial or complete variant calling analysis with the advantages of having a workflow manager: dependency resolution, execution planner, modularity, monitoring and historic.

Documentation for the latest version is being hosted by readthedocs <>_

Installation ^^^^^^^^^^^^ Wespipeline is available through pip, conda and manual installation. Install it from the package repositories pip3 install wespipeline conda install -c jancho wespipeline, or download the project and build from source: git clone && cd wespipeline && python3 install.

Notice that executing the analysis will involve different additional dependencies depending on the steps that executed and the parameters set for these. All possible are cited below and can be downloaded with the Anaconda distribution:

  • Secuence retrieval : Sra Toolkit, Fastqc
  • Reference genome retrieval : No needed dependency
  • Secuence alignment : Bwa
  • Alignment processing : Bwa Samtools,
  • Variant calling : Freebayes, Varscan, Gatk, Deepvariant
  • Variant calling evaluation : Vcf tools

In addition to the dependencies, conda can be used for installing the wespipeline package. An example for installing the miniconda distribution, the package and the dependencies is:

.. code-block:: bash

wget -O ~/ bash ~/ -b -p $HOME/miniconda export PATH="$HOME/miniconda/bin:$PATH" source $HOME/miniconda/bin/activate &&
conda config --add channels bioconda &&
conda config --add channels conda-forge &&
conda config --add channels jancho &&
conda install -y samtools &&
conda install -y bwa &&
conda install -y picard &&
conda install -y platypus-variant &&
conda install -y varscan &&
conda install -y freebayes &&
conda install -y fastqc &&
conda install -y sra-tools &&
conda install -y wespipeline

rm ~/

Getting started ^^^^^^^^^^^^^^^

Installing or downloading the package will provide with a higher level task per step of the analysis, each of which can be executed in a similar fashion to other Luigi tasks.

Each of the six steps have a higher level task that can be scheduled in a similar fashion to other Luigi tasks:

.. code-block:: bash

python3 -m luigi --module wespipeline.<module> <Taskname> --<Taskname>-param value

Download the sequences using the NCBI accession number.

.. code-block:: bash

python3 -m luigi --module wespipeline.fastq FastqRetrieval \
	--FastqRetrieval-paired-end true \
	--FastqRetrieval-accession-number SRR9209557 \
	--FastqRetrieval-create-report true

Or an external url.

.. code-block:: bash

python3 -m luigi --module wespipeline.fastq FastqRetrieval \
	--FastqRetrieval-paired-end true \
	--FastqRetrieval-compressed false \
	--FastqRetrieval-accession-number SRR9209557 \
	--FastqRetrieval-create-report true

Download the reference genome and create a report using FastqC.

.. code-block:: bash

python3.6 -m luigi --module tasks.reference ReferenceRetrieval 
	--workers 3 \
	--ReferenceGenome-ref-url \
	--ReferenceGenome-from2bit True \
	--GlobalParams-base-dir ./tfm_experiment \
	--GlobalParams-log-dir .logs \
	--GlobalParams-exp-name hg19

Or run the whole analysis, specifying the parameters for each of the steps.

.. code-block:: bash

python3 -m luigi --module tasks.vcf VariantCalling 
	--workers 3 
	--VariantCalling-use-platypus true 
	--VariantCalling-use-freebayes true 
	--VariantCalling-use-samtools false 
	--VariantCalling-use-gatk false 
	--VariantCalling-use-deepcalling false 
	--AlignProcessing-cpus 6 
	--FastqAlign-cpus 6 
	--FastqAlign-create-report True 
	--GetFastq-gz-compressed True 
	--GetFastq-from-ebi False 
	--GetFastq-paired-end True 
	--ReferenceGenomeRetrieval-ref-url --ReferenceGenomeRetrieval-from2bit True 
	--GlobalParams-base-dir ./tfm_experiment 
	--GlobalParams-log-dir .logs 
	--GlobalParams-exp-name hg19 

Tasks implemented ^^^^^^^^^^^^^^^^^

+-----------------+----------------------------+ | Module | Task | +=================+============================+ | reference | ReferenceGenomeRetrieval | +-----------------+----------------------------+ | fastq | FastqRetrieval | +-----------------+----------------------------+ | align | FastqAlignment | +-----------------+----------------------------+ | processalign | FastqProcessing | +-----------------+----------------------------+ | variantcalling | VariantCalling | +-----------------+----------------------------+ | processalign | VariantProcessing | +-----------------+----------------------------+

Acknowledgements ^^^^^^^^^^^^^^^^

Special thanks to professor Luis Antonio Miguel Quintales for all the guidance and help provided during the development of this project.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for wespipeline, version 1.0.1
Filename, size File type Python version Upload date Hashes
Filename, size wespipeline-1.0.1-py3-none-any.whl (22.0 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size wespipeline-1.0.1.tar.gz (14.6 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page