A simple multiprocessing function pool
Project description
pytaskpool
A simple multiprocess function pooler which create results generators
pytaskpool use the python multiprocessing module
pytaskpool provide an easy way to execute python functions in a pool of process (not threads) for using all your CPU cores. Functions results can then be then obtained in order or not (via a generator method).
- Download and docs:
- Source code & Development:
- Issues:
Installation
pytaskpool is available in the python index package (pip), It can be installed running the following command:
$ pip install pytaskpool
or by git like this:
$ git clone https://github.com/govlog/pytaskpool $ cd pytaskpool $ sudo python setup.py install
Usage
This sample code will simultanously launch 8 functions with differents parameters using a pool of 8 processes. The results returned by all launched functions will be get by the method get_sorted_results(), which is a generator returning the function result in order.
The excepted execution time should be 1 second.
Sample code:
import pytaskpool as tp from time import sleep def my_func(x): sleep(1) return [x ** x] mypool = tp.TaskPool([], 8) for r in range(8): mypool.launch(my_func, r) unsorted = [r for r in mypool.get_unsorted_results()] sorted = [r for r in mypool.get_sorted_results()] print "unsorted :",unsorted print "sorted :",sorted
Terminal output should look like this:
$ time python test.py unsorted : [[1], [4], [1], [3125], [823543], [256], [27], [46656]] sorted : [[1], [1], [4], [27], [256], [3125], [46656], [823543]] real 0m1.031s user 0m0.858s sys 0m0.193s
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.