Skip to main content

No project description provided

Project description

bit-counter

Package for counting the number of one bits in a numpy array of uint8 values. Implemented as a Python module using Rust, providing high performance counting.

Building

To build this package an installation of Rust and Python with the maturin package is required. The Maturin documentation on maturin local development is a useful reference for more details.

This project is configured by default to target CPUs with the POPCNT instruction. The builds available on PyPI have been built with this configuration. If you require a version for an older CPU without popcnt support, build with the RUSTFLAGS environment variable to exclude the popcnt target-feature. i.e. RUSTFLAGS='-C target-feature=-popcnt' maturin build -r.

By default the bit counting is done with a parallel map across all available CPUs through the use of rayon. Number of threads can be configured with the environment variable RAYON_NUM_THREADS.

Example usage

import numpy as np
from bit_counter import count_ones

arr = np.packbits(np.random.choice([True, False], 1000000))

count_of_true_values = count_ones(arr)

Performance

When built to target a CPU with popcnt support the count_ones method provided is substantially faster than a naive np.unpackbits(arr).sum(). The count_ones method also doesn't require unpacking the bit packed numpy array, so doesn't require any addtional memory to do the calculation.

For example with the following test code for 100 million True/False values that are packed:

import numpy as np
from bit_counter import count_ones

arr = np.packbits(np.random.choice([True, False], 100000000))

Using this package on an AMD Ryzen 9 3900X 12-Core Processor yields:

%timeit count_ones(arr)

905 µs ± 4.64 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

Compared to the simple unpack and sum case:

%timeit np.unpackbits(arr).sum()

60.4 ms ± 421 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

bit_counter-0.2.0.tar.gz (4.2 kB view hashes)

Uploaded Source

Built Distributions

bit_counter-0.2.0-pp37-pypy37_pp73-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.1 MB view hashes)

Uploaded PyPy manylinux: glibc 2.5+ x86-64

bit_counter-0.2.0-cp310-none-win_amd64.whl (151.0 kB view hashes)

Uploaded CPython 3.10 Windows x86-64

bit_counter-0.2.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.1 MB view hashes)

Uploaded CPython 3.10 manylinux: glibc 2.5+ x86-64

bit_counter-0.2.0-cp310-cp310-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl (525.7 kB view hashes)

Uploaded CPython 3.10 macOS 10.9+ universal2 (ARM64, x86-64) macOS 10.9+ x86-64 macOS 11.0+ ARM64

bit_counter-0.2.0-cp39-none-win_amd64.whl (151.1 kB view hashes)

Uploaded CPython 3.9 Windows x86-64

bit_counter-0.2.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.1 MB view hashes)

Uploaded CPython 3.9 manylinux: glibc 2.5+ x86-64

bit_counter-0.2.0-cp39-cp39-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl (525.1 kB view hashes)

Uploaded CPython 3.9 macOS 10.9+ universal2 (ARM64, x86-64) macOS 10.9+ x86-64 macOS 11.0+ ARM64

bit_counter-0.2.0-cp38-none-win_amd64.whl (150.9 kB view hashes)

Uploaded CPython 3.8 Windows x86-64

bit_counter-0.2.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.1 MB view hashes)

Uploaded CPython 3.8 manylinux: glibc 2.5+ x86-64

bit_counter-0.2.0-cp38-cp38-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl (526.0 kB view hashes)

Uploaded CPython 3.8 macOS 10.9+ universal2 (ARM64, x86-64) macOS 10.9+ x86-64 macOS 11.0+ ARM64

bit_counter-0.2.0-cp37-none-win_amd64.whl (150.9 kB view hashes)

Uploaded CPython 3.7 Windows x86-64

bit_counter-0.2.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.1 MB view hashes)

Uploaded CPython 3.7m manylinux: glibc 2.5+ x86-64

bit_counter-0.2.0-cp37-cp37m-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl (526.0 kB view hashes)

Uploaded CPython 3.7m macOS 10.9+ universal2 (ARM64, x86-64) macOS 10.9+ x86-64 macOS 11.0+ ARM64

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page