Skip to main content

A simple multiprocessing function pool

Project description

pytaskpool (v1.5)

A simple multiprocess function pooler which create results generators

pytaskpool use the python multiprocessing module

pytaskpool provide an easy way to execute python functions in a pool of process (not threads) for using all your CPU cores. Functions results can then be then obtained in order or not (via a generator method).

pytaskpool works on windows and linux.

Download and docs:

http://pypi.python.org/pypi/pytaskpool

Source code & Development:

https://github.com/govlog/pytaskpool

Issues:

christopher.amiaud@gmail.com

Installation

pytaskpool is available in the python index package (pip), It can be installed running the following command:

$ pip install pytaskpool

or by git like this:

$ git clone https://github.com/govlog/pytaskpool
$ cd pytaskpool
$ sudo python setup.py install

or even in a zip over http : https://github.com/govlog/pytaskpool/archive/master.zip

Usage

This sample code will simultanously launch 8 functions with differents parameters using a pool of 8 processes. The results returned by all launched functions will be get by the method get_sorted_results(), which is a generator returning the functions results in order.

The excepted execution time should be around 1 second.

Sample code:

import pytaskpool as tp
from time import sleep

def my_func(x):
    sleep(1)
    return x ** x

mypool = tp.TaskPool(8)

for r in range(8):
    mypool.launch(my_func, r)

unsorted = [r for r in mypool.get_unsorted_results()]
sorted = [r for r in mypool.get_sorted_results()]

print ("unsorted :",unsorted)
print ("sorted   :",sorted)

Terminal output should look like this:

$ time python examples/pytask_ex2.py
unsorted : [1, 1, 4, 27, 256, 46656, 3125, 823543]
sorted   : [1, 1, 4, 27, 256, 3125, 46656, 823543]

real    0m1,055s
user    0m0,800s
sys     0m0,272s

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pytaskpool-1.5.tar.gz (16.5 kB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page