Parallel multiradial LBP features
Project description
fastLBP
Highly parallel LBP implementation
Installation
Note: it is not recommended to proceed on a head node; consider starting an ijob or a jupyter instance.
- Activate a Python 3.11 environment (e.g. using
conda create -y -n p11 python=3.11 && conda activate p11
) - Verify you are using the right env
python --version
таpip --version
- Install a stable version from PyPI
pip install fastlbp-imbg
- Or build the latest version from sources
git clone git@github.com:imbg-ua/fastLBP.git
cd fastLBP
# git checkout <branchname> # if you need a specific branch
pip install . # this will install the fastlbp_imbg package in the current env
- You can use
import fastlbp_imbg
now
Cython2 branch details
TODO:
- Data Contiguity (
e3293cd
) - Job sorting (from slow to quick) in fastlbp.py (
8a21a04
) - Change data types in lbp.pyx from float64 to something smaller (
f177823
) - Add mask feature to lbp.pyx (
d70bc93
) - Add mask feature to fastlbp.py (
d812831
) - Fill all commit numbers
Implemented modules
run_skimage
Computes multiradial LBP of a single multichannel image in a parallel fashion.
This is a quick sample implementation that could be a baseline for further benchmarking.
Features:
- Powered by
skimage.feature.local_binary_pattern
- Concurrency is managed by Python's
multiprocessing
module - Parallel computation via
multiprocessing.Pool
of sizencpus
- Efficient memory usage via
multiprocessing.shared_memory
to make sure processes do not create redundant copies of data - It computes everything in RAM, no filesystem usage
TODO:
- Use
max_ram
parameter to estimate optimal number of sub-processes and collect memory stats. Nowmax_ram
is ignored.
Planned modules
run_chunked_skimage
Similar to 1. run_skimage, but each subprocess should compute LBP for its image chunk, not the whole image.
run_dask and run_chunked_dask
Similar to 1. run_skimage, but use Dask and dask.array.map_overlap
for parallelisation instead of multiprocessing
and manual data wrangling
Other notable things to try
- Perform benchmarking of an in-house optimised cython version of
skimage.feature.local_binary_pattern
(see skimage_lbp.pyx at imbg-ua/nf-img-benchmarking) - Do some research on Numba - is it applicable here?
- Add annotations to run_skimage results using
anndata
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
fastlbp_imbg-0.1.2.tar.gz
(205.8 kB
view hashes)
Built Distribution
Close
Hashes for fastlbp_imbg-0.1.2-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a88f6939676118f20acd58e086edaeec642ea7f8c12db0381f4f37d061ce6877 |
|
MD5 | f3e7755d56a25735ad6ba41452a8d217 |
|
BLAKE2b-256 | 05208b98b570571d4ba6432a9e0b13b589c12110637b8166ed8e94c54f19561e |