Simple pool manager
Project description
poolmanager
Simple pool manager implementation for unordered results
poolmanager is compatible with python 2.6, 2.7, 3.6 and 3.7
Usage
ctrl+c
will automatically terminate the main process and
thus all the associated subprocesses. You'll need to wait for all the
functions to finish their current execution before being able to stop
all the processes. One can also easily create a callback function to monitor
the computation state.
from poolmanager import PoolManager
def add(x):
return x + 1.5
def callback(counter, result):
print 'counter: %s' % counter
print 'result: %s' % result
def main():
chunk = 2
iterator = xrange(0, 10)
pm = PoolManager(numProcs=2, factor=2, store=True)
pm.imap_unordered(add, iterator, chunk, callback=callback)
if __name__ == '__main__':
main()
>> counter: 1
>> result: 1.5
>> counter: 2
>> result: 2.5
>> counter: 3
>> result: 5.5
>> counter: 4
>> result: 6.5
>> counter: 5
>> result: 7.5
>> counter: 6
>> result: 8.5
>> counter: 7
>> result: 3.5
>> counter: 8
>> result: 4.5
>> counter: 9
>> result: 9.5
>> counter: 10
>> result: 10.5
print [i for i in iterator]
print pm.results
>> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>> [1.5, 2.5, 5.5, 6.5, 7.5, 8.5, 3.5, 4.5, 9.5, 10.5]
Tests
python setup.py test
Publish a new version of the module
Edit $HOME/.pypirc
and add (username and password in keepass):
[distutils]
index-servers =
pypi
pypitest
[pypi]
repository=https://upload.pypi.org/legacy/
username=iwi***
password=
[pypitest]
repository=https://test.pypi.org/legacy/
username=iwi***
password=
Bump version in setup.py
.
Upload the new module to the test repository:
pip install --upgrade setuptools
python setup.py sdist upload -r pypitest
python setup.py bdist_wheel upload -r pypitest
Test local install from test repository.
pip install -i https://test.pypi.org/simple/ poolmanager
If everything is ok, push the new version to the default repository.
python setup.py sdist upload -r pypi
python setup.py bdist_wheel upload -r pypi
Test the newly created module.
Create a RELEASE in github.
CONTRIBUTORS:
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file poolmanager-0.2.1.tar.gz
.
File metadata
- Download URL: poolmanager-0.2.1.tar.gz
- Upload date:
- Size: 3.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/50.1.0 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.7.2
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 84e12062ad2485c7c612221cbc5019cb989ee25155b7f1630452c32439443739 |
|
MD5 | d2da930342a2be0c288383d9ca41b431 |
|
BLAKE2b-256 | c47c8c34ef7e554f15ef47328daa3cd4b54fefb71766bb9e1246294f63d8873e |
File details
Details for the file poolmanager-0.2.1-py3-none-any.whl
.
File metadata
- Download URL: poolmanager-0.2.1-py3-none-any.whl
- Upload date:
- Size: 3.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/50.1.0 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.7.2
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 94c3304aa18cda481f5db2680081101f6dd57d90335e1eb5fb5aeac1d54bdc0c |
|
MD5 | deea83870779eb6fb5a2c7f5aa027045 |
|
BLAKE2b-256 | 2deed1bd0eede857fb105aacac15c4ce9633b790577501cd5d0511c669413360 |