Mr_Demuxy 1.0.9.beta

demultiplexs combinatorially tagged reads

Mr. Demuxy: Combinotorial Demultixper INTRODUCTION: Author: Daniel E. Lefever
Author-email: [e-mail]
license: Too be determined Mr. Demuxy is a fairly simple, self-contained program that makes demultiplexing variable length, combinatorially barcoded NGS reads as painless as possible. As such, the goals for Mr_Demuxy were that the final program would require little to no software installation and to be as user-friendly as possible. This program was written with sequencing data generated by following Travis Glenn’s Tagimatrix barcoding strategy [Tagimatrix Ref] in mind, however, it would work on any type of data multiplexed with combinatorial barcodes. In short, this approach uses two pairs of barcodes denoted as the internal and external barcodes (also referred to as tags and indexes). The idea is to pool many different projects onto one MiSeq/HiSeq run cheaply by having external Illumina barcodes to differentiate the pooled projects and internal ‘homebrewed’ variable length barcodes to distinguish samples within each project. The internal barcoding strategy is based on the layout of a 96-well plate: the forward tag corresponds to row letter (A-H) and the reverse tag corresponds to column number (1-12). This program uses both tags to come up with the sample ID. There are two main scripts, called merged_demuxer and pe_demuxer which can handle either demultiplexing a merged fastq file or paired-end fastq files, respectively. DEPENDENCIES: python 2.7.X unix/linux OS INSTALLATION: Step 1: Download Mr. Demuxy and Extract it
After downloading Mr. Demuxy extract the folder contents. To extract file contents, try clicking on it. If that doesn't work, try this:
$ tar -xvzf Mr_Demuxy-1.0.0-dist.tar.gz Step 2: Install package From the command line change dir to wherever the unarchived folder lives. Then install to your local $ python dir like so: $ python setup.py install --user --force Step 3: Create bash aliases Next, run the alias_maker script so you can run this program from anywhere without needing to write out the programs full path. Here's how to run it:
$ python alias_maker.py Step 4: Restart terminal After restarting terminal, you should now be able to call the script from the console like this: $ merged_demuxer -i example_merged.fastq -f forward_bcs.txt -r reverse_bcs.txt -o example_demult.fasta $ pe_demuxer -r1 r1_seqs.fastq -r2 r2_seqs.fastq -r1_bc r1_bc_file.txt -r2_bc -r2_bc_file.txt
USAGE EXAMPLES: MERGED SINGLE FASTQ

merged_demuxer arguements:

required arguments:
-f forward bcs file
-r reverse bcs file
-i input merged fastq file

optional arguments:
-o name of output file or folder (default:
merged_demuxer_out)
--forward_primer_len base number of forward primer/adapter (default: 0)
--reverse_primer_len
base number of reverse primer/adapter (default: 0)
--no_rev_comp_reverse_bcs
prevents rev bcs from reverse complementing
-h, --help show this help message and exit
--keep_original_headers
pass this arg to keep original headers in
demultiplexed fasta/fastq file
--write_qual write a qual file containing quality scores
--individual_fastq demultiplex by writing a fastq file for each sample ID
--min_seq_length minimum length of seq to be retained (default: 20)
--file_extension format of the output file (default: fasta) ---Demultiplex merged fastq reads w/ output being in fasta format: $ merged_demuxer -i example_merged.fastq -f forward_bcs.txt -r reverse_bcs.txt -o example_demult.fasta

---Do the same as above and get a seperate qual file:

$ merged_demuxer -i example_merged.fastq -f forward_bcs.txt -r reverse_bcs.txt -o example_demult.fasta --write_qual

---Demultiplex and keep the original fastq headers:

$ merged_demuxer -i example_merged.fastq -f forward_bcs.txt -r reverse_bcs.txt -o example_demult.fasta --keep_original_headers
---Demultiplex merged reads and remove the 17 bp forward primer and 22 bp reverse primer: $ merged_demuxer -i example_merged.fastq -f forward_bcs.txt -r reverse_bcs.txt -o example_demult.fasta --forward_primer_len 17 --reverse_primer_len 22 ---Do the same as above except have the output be in fastq format: $ merged_demuxer -i example_merged.fastq -f forward_bcs.txt -r reverse_bcs.txt -o example_demult.fastq --forward_primer_len 17 --reverse_primer_len 22 --file_extension fastq ---Demultiplex reads and split into one fastq file per sample ID: $ merged_demuxer -i example_merged.fastq -f forward_bcs.txt -r reverse_bcs.txt -o example_demult_ind_fastq --individual_fastq

---Get individual fastq files, chop off the 17 bp forward primer and 22 bp reverse primer, and keep reads that are only
longer than 100 bp:

$ merged_demuxer -i example_merged.fastq -f forward_bcs.txt -r reverse_bcs.txt -o example_demult_ind_fastq --individual_fastq --forward_primer_len 17 --reverse_primer_len 22 --individual_fastq --min_seq_length 100

USAGE EXAMPLES: TWO PAIRED-END FASTQ FILES

pe_demuxer arguements:

required arguments:
-r1_bc BC file for R1 reads
-r2_bc BC file for R2 reads
-r1 R1 fastq file
-r2 R2 fastq file

optional arguments:
-h, --help show this help message and exit
-o name of output file or dir
--forward_primer_len base number of primer/adapter on R1 seq (default: 0)
--reverse_primer_len
base number of primer/adapter on R2 seq (default: 0)
--keep_original_headers
pass this arg to keep original headers in
demultiplexed fasta/fastq file
--min_seq_length minimum length of seq to be retained (default: 20)
--check_headers ensures the R1 and R2 read match before demultiplexing
WARNING: this is super slow ---Demultiplex paired-end fastq files and write individual fastqs for each sample in their respective directories: $ pe_demuxer -r1 r1_seqs.fastq -r2 r2_seqs.fastq -r1_bc r1_bc_file.txt -r2_bc -r2_bc_file.txt

---Do the same as above and remove a 10 bp primer on R1 reads and a 16 bp primer on R2 reads:

$ pe_demuxer -r1 r1_seqs.fastq -r2 r2_seqs.fastq -r1_bc r1_bc_file.txt -r2_bc -r2_bc_file.txt --forward_primer_len 10 --reverse_primer_len 16

---Demultiplex paired-end reads in which the read order may be different and/or the files contain un-paired reads:
Note: using this option is not recommended and should not be neccessary since Illumina sequencers should produce files with reads in the same order and discard un-paired reads. I didn't realize that until after added this feature so I decided to include it anyways.

$ pe_demuxer -r1 r1_seqs.fastq -r2 r2_seqs.fastq -r1_bc r1_bc_file.txt -r2_bc -r2_bc_file.txt --check_headers


BARCODE FILES:

The forward barcode file, as you might expect, is the file containing the forward barcodes and letter (or whatever the barcode corresponds to). You will probably need to make your own forward barcode file, and the formatting is important. The easieast way to make one, is to open a new sheet in excel and copy/paste the barcode and letter pairs with the first column being the letter and the second containing the barcode. It should look like this: A GGTAC B CAACAC C ATCGGTT D TCGGTCAA NOTE: Both barcode files do not have any header information. I’m not sure if it will screw things up, but it’s safest to not include it. Once you’re done inputing the forward barcodes, save the excel sheet as a ‘tab delineated text’ file. Be sure to name it something so you know what it is. The reverse barcode file is just like the forward barcode file, except for the reverse barcodes. It should look like this: 1 AGGAA 2 GAGTGG 3 CCACGTC 4 TTCTCAGC Once you’ve followed the same process as with the forward barcode file, save it as a ‘tab delineated text’ file. NOTE: This program reverse complements (RC) the reverse barcodes instead of RCing the whole sequence. This makes it go faster, but if your reverse barcodes do not need to be RC'd, pass the --no_rev_comp_reverse_bcs arguement. OUTPUT: The output file/files contain reads that have been demultiplexed, and optionally trimmed of primers. This program automatically removes the barcode from each read and will remove the primers if length is specified by the --forward_primer_len and --reverse_primer_len arguments. Since indels are rare with illumina sequencing, the primers are not actually matched and are instead removed by trimming a set number of bases after the barcode. For example, if the forward primer was 17 bp long, and the reverse 15, pass --forward_primer_len 17 --reverse_primer_len 15 to remove it from each read. If these options are not passed, it defaults to 0. This program can also create individual fastq files split by each sample. To do this, simply pass the --individual_fastq argument. The -o arguement will become the name of the output directory with one fastq per sample. The dir structure is as followd:

-output_dir/
<sample_id_1>.fastq
<sample_id_2>.fastq
.
.
.
<sample_id_n>.fastq

pe_demuxer will only output individual fastq files. The R1 and R2 files are seperated into two folders each containing individual fastq files split by sample. This is the structure:

-output_dir/
R1/
<sample_id_1>_R1.fastq
<sample_id_2>_R1.fastq
.
.
.
<sample_id_n>_R1.fastq

R2/
<sample_id_1>_R2.fastq
<sample_id_2>_R2.fastq
.
.
.
<sample_id_n>_R1.fastq


There are number of different options for the output file. These include: HEADER OPTIONS: -Demultiplexed ID only- --FASTA Format-- >A6_1 AATATT....CACGCAGA --FASTQ Format-- @A6_1 AATATT....CACGCAGA + GGGGGG...@GGFGGFC< -Original header with demultiplexed ID- --FASTA Format-- >M02849:...:1886 A6_1 AATATT....CACGCAGA --FASTQ Format-- @M02849:...:1886 A6_1 AATATT....CACGCAGA + GGGGGG...@GGFGGFC< PROGRAM INFO: This program was written and predominantly tested using a macbook pro 2011 with 8 GB memory and a 2.4 GHz Intel Core i7 processor running OS X El Captitan using python 2.7.10. With this setup, both merged (written to single or individual files) and paired-end demultiplexing of ~7,000,000 reads took around 3-4 minutes. References: Cock, P. J., Antao, T., Chang, J. T., Chapman, B. A., Cox, C. J., Dalke, A., . . . de Hoon, M. J. (2009). Bio$ python: freely available $ python tools for computational molecular biology and bioinformatics. Bioinformatics, 25(11), 1422-1423. doi:10.1093/bioinformatics/btp163