rfmix-reader 0.1.14

RFMix-reader is a Python package designed to efficiently read and process output files generated by RFMix, a popular tool for estimating local ancestry in admixed populations. The package employs a lazy loading approach, which minimizes memory consumption by reading only the loci that are accessed by the user, rather than loading the entire dataset into memory at once.

RFMix-reader

RFMix-reader is a Python package designed to efficiently read and process output files generated by RFMix, a popular tool for estimating local ancestry in admixed populations. The package employs a lazy loading approach, which minimizes memory consumption by reading only the loci that are accessed by the user, rather than loading the entire dataset into memory at once. Additionally, we leverage GPU acceleration to improve computational speed.

Install

rfmix-reader can be installed using pip:

pip install rfmix-reader

GPU Acceleration: rfmix-reader leverages GPU acceleration for improved performance. To use this functionality, you will need to install the following libraries for your specific CUDA version:

• RAPIDS: Refer to official installation guide here
• PyTorch: Installation instructions can be found here

Additoinal Notes:

• We have not tested installation with Docker or Conda environemnts. Compatibility may vary.
• If you do not have GPU, you can still use the basic functionality of rfmix-reader. This is still much faster than processing the files with stardard scripting.

Key Features

Lazy Loading

• Reads data on-the-fly as requested, reducing memory footprint.
• Ideal for working with large RFMix output files that may not fit entirely in memory.

Efficient Data Access

• Provides convenient access to specific loci or regions of interest.
• Allows for selective loading of data, enabling faster processing times.

Seamless Integration

• Designed to work seamlessly with existing Python data analysis workflows.
• Facilitates downstream analysis and manipulation of RFMix output data.

Whether you're working with large-scale genomic datasets or have limited computational resources, RFMix-reader offers an efficient and memory-conscious solution for reading and processing RFMix output files. Its lazy loading approach ensures optimal resource utilization, making it a valuable tool for researchers and bioinformaticians working with admixed population data.

Usage

This works similarly to pandas-plink:

Two population admixture example

This is a two part process.

Generate binary files

To reduce computational time and memory, we leverage binary files. As this is not generated by RFMix, we provide a function to do this before running.

from rfmix_reader import create_binaries

# Generate binary files
file_path = "examples/two_popuations/out/"
binary_dir = "./binary_files"
create_binaries(file_path, binary_dir=binary_dir)

You can also do this on the fly.

from rfmix_reader import read_rfmix

file_path = "examples/two_popuations/out/"
binary_dir = "./binary_files"
loci, rf_q, admix = read_rfmix(file_path, binary_dir=binary_dir,
                               generate_binary_files=True)

We do not have this turned on by default, as it is the rate limiting step. It can take upwards of 20 to 25 minutes to run depending on *fb.tsv file size.

Main function

Once binary files are generated, you can the main function to process the RFMix results. With GPU this takes less than 5 minutes.

from rfmix_reader import read_rfmix

file_path = "examples/two_popuations/out/"
loci, rf_q, admix = read_rfmix(file_path) 

Note: ./binary_files is the default for binary_dir, so this is an optional parameter.

BED format

One helpful function we provide is export_loci_admix_to_bed. This function takes the output of the read_rfmix and exports a BED format with haplotypes condensed to regional variation in parquet files per chromosome.

export_loci_admix_to_bed(loci, rf_q, admix)

Unlike generating binary files, this takes a large amount of memory to write files, so it must be called separately outside of the main function.

Three population admixture example

RFMix-reader is adaptable for as many population admixtures as needed. However, due to some conservative parameters, the use of BED formatting will take longer than a two population analysis.

from rfmix_reader import read_rfmix

file_path = "examples/three_popuations/out/"
binary_dir = "./binary_files"
loci, rf_q, admix = read_rfmix(file_path, binary_dir=binary_dir,
                               generate_binary_files=True)

export_loci_admix_to_bed(loci, rf_q, admix)

Authors

• Kynon JM Benjamin

Citation

Please cite: XXXX.