Skip to main content
Join the official 2019 Python Developers SurveyStart the survey!

Efficient subroutines for computing summary statistics for the SAM FLAG field

Project description

pyflagstats

PyPI version

Given a stream of k-bit words, we seek to sum the bit values at indexes 0, 1, 2, ..., k-1 across multiple words by computing k distinct sums. If the k-bit words are one-hot encoded then the sums corresponds to their frequencies.

This multiple-sum problem is a generalization of the population-count problem where we count the total number of set bits in independent machine words. We refer to this new problem as the positional population-count problem.

Using SIMD (Single Instruction, Multiple Data) instructions from recent Intel processors, we describe algorithms for computing the 16-bit position population count using about one eighth (0.125) of a CPU cycle per 16-bit word. Our best approach is about 140-fold faster than competitive code using only non-SIMD instructions in terms of CPU cycles.

This package contains native Python bindings for the applying the efficient positional population count operator to computing summary statistics for the SAM FLAG field

Intallation

Install with

pip3 install .

or locally with

python3 setup.py build_ext --inplace

Uninstall with

pip3 uninstall pyflagstats

Example

import numpy as np
import pyflagstats as fs

# Compute summary statistics for 100 million random FLAG fields.
# Completes in around 1 second.
fs.flagstats(np.random.randint(0,8192,100000000,dtype="uint16"))

returns (for example)

{'passed': array([ 624787,  312748, 2500089,  312384,  312314,  312678,  312045,
        311845, 2499502, 4999279, 2497500, 1248979,  389744,  156194,
        156029,       0], dtype=uint32), 'failed': array([ 625143,  312906, 2498840,  312818,  312129,  312802,  311869,
        312105, 2501477, 5000721, 2499178, 1249105,  390962,  155828,
        156018,       0], dtype=uint32)}

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for pyflagstats, version 0.1.4
Filename, size File type Python version Upload date Hashes
Filename, size pyflagstats-0.1.4.tar.gz (148.7 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page