Skip to main content

downsample NGS data sets

Project description

https://badge.fury.io/py/sequana-downsampling.svg JOSS (journal of open source software) DOI https://github.com/sequana/downsampling/actions/workflows/main.yml/badge.svg

This is is the downsampling pipeline from the Sequana project

Overview:

downsample NGS data sets

Input:

a set of FastQ or FASTA files

Output:

a set of downsampled files

Status:

production

Citation(sequana):

Cokelaer et al, (2017), ‘Sequana’: a Set of Snakemake NGS pipelines, Journal of Open Source Software, 2(16), 352, JOSS DOI doi:10.21105/joss.00352

Citation(pipeline):
https://zenodo.org/badge/DOI/10.5281/zenodo.4047837.svg

Installation

You must install Sequana first:

pip install sequana

Then, just install this package:

pip install sequana_downsampling

Usage

sequana_downsampling --help
sequana_downsampling --input-directory DATAPATHH
sequana_downsampling --downsampling-method random --downsampling-max-entries 100
sequana_downsampling --downsampling-method random_pct --downsampling-percent 10 --downsampling-input-format fasta --input-pattern "whatever*fasta"

Note that the current implementation handles fastq files (zipped or not) and fasta files (uncompressed only)

This creates a directory with the pipeline and configuration file. You will then need to execute the pipeline:

cd downsampling
sh downsampling.sh  # for a local run

This launch a snakemake pipeline. If you are familiar with snakemake, you can retrieve the pipeline itself and its configuration files and then execute the pipeline yourself with specific parameters:

snakemake -s downsampling.rules -c config.yaml --cores 4 --stats stats.txt

Or use sequanix interface.

Examples of a set of FastQ zipped files in the current directory:

sequana_downsampling –run –downsampling-method random_pct cd downsampling make clean

This will create a directory called downsampling, and randomly select 10% of the input reads for each file with extension .fastq.gz in the current directory. Since -run is used, the pipeline is executed automatically. The following commands will enter into the directory and called a Makefile. This will clean the directory for temporary files.

Requirements

This pipelines requires the following executable(s):

  • sequana

  • pigz

Details

This pipeline runs downsampling in parallel on the input fastq or fasta files (paired or not). If paired, the one-to-one mapping is conserved.

It can take as input a set of FastQ files, or FastA files. by default, the pipeline with randomly select 1000 entries from each input files. You can increase this number using –downsampling-max-entries option. If you prefer to select a percentage of the entries instead, you can change the downsamping method as follows:

--downsampling-method random_pct

and change the value if needed (default is 10%):

--downsampling-percent 20

Note that input FastQ can be gzipped. Output files are gzipped. FastA input files must be compressed for now

Rules and configuration details

Here is the latest documented configuration file to be used with the pipeline. Each rule used in the pipeline may have a section in the configuration file.

Changelog

Version

Description

0.8.5

  • cope with R1/R2 paired data properly. Improved make file

0.8.4

  • add missing MANIFEST to include missing requirements.txt

0.8.3

  • comply with new API from sequana_pipetools 0.2.4

0.8.2

  • add a –run option to execute the pipeline directly

0.8.1

  • fix input and N in the random selection

0.8.0

First release.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sequana_downsampling-0.9.0.tar.gz (7.2 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page