better multiprocessing and multithreading in python
multiprocess is a fork of multiprocessing, and is developed as part of pathos: https://github.com/uqfoundation/pathos
multiprocessing is a package for the Python language which supports the spawning of processes using the API of the standard library’s threading module. multiprocessing has been distributed in the standard library since python 2.6.
- Objects can be transferred between processes using pipes or multi-producer/multi-consumer queues.
- Objects can be shared between processes using a server process or (for simple data) shared memory.
- Equivalents of all the synchronization primitives in threading are available.
- A Pool class makes it easy to submit tasks to a pool of worker processes.
multiprocess is part of pathos, a python framework for heterogeneous computing. multiprocess is in active development, so any user feedback, bug reports, comments, or suggestions are highly appreciated. A list of known issues is maintained at http://trac.mystic.cacr.caltech.edu/project/pathos/query.html, with a public ticket list at https://github.com/uqfoundation/multiprocess/issues.
NOTE: A C compiler is required to build the included extension module. For python 3.3 and above, a C compiler is suggested, but not required.
- enhanced serialization, using dill
This documentation is for version multiprocess-0.70.9 (a fork of multiprocessing-0.70a1).
The latest released version of multiprocess is available from:
Multiprocessing is distributed under a BSD license.
You can get the latest development version with all the shiny new features at:
If you have a new contribution, please submit a pull request.
multiprocess is packaged to install from source, so you must download the tarball, unzip, and run the installer:
[download] $ tar -xvzf multiprocess-0.70.9.tgz $ cd multiprocess-0.70.9 $ python setup.py build $ python setup.py install
You will be warned of any missing dependencies and/or settings after you run the “build” step above.
Alternately, multiprocess can be installed with pip or easy_install:
$ pip install multiprocess
NOTE: A C compiler is required to build the included extension module from source. For python 3.3 and above, a C compiler is suggested, but not required. Binary installs do not require a C compiler.
- ``python``, **version >= 2.5** or **version >= 3.1** - ``dill``, **version >= 0.3.1**
- ``setuptools``, **version >= 0.6**
Probably the best way to get started is to look at the documentation at http://multiprocess.rtfd.io. See multiprocess.examples for a set of example scripts. You can also run the test suite with python -m multiprocess.tests. Please feel free to submit a ticket on github, or ask a question on stackoverflow (@Mike McKerns). If you would like to share how you use multiprocess in your work, please post send an email (to mmckerns at uqfoundation dot org).
If you use multiprocess to do research that leads to publication, we ask that you acknowledge use of multiprocess by citing the following in your publication:
M.M. McKerns, L. Strand, T. Sullivan, A. Fang, M.A.G. Aivazis, "Building a framework for predictive science", Proceedings of the 10th Python in Science Conference, 2011; http://arxiv.org/pdf/1202.1056 Michael McKerns and Michael Aivazis, "pathos: a framework for heterogeneous computing", 2010- ; http://trac.mystic.cacr.caltech.edu/project/pathos
Please see http://trac.mystic.cacr.caltech.edu/project/pathos or http://arxiv.org/pdf/1202.1056 for further information.
Release history Release notifications
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size multiprocess-0.70.9-cp27-cp27m-win32.whl (91.6 kB)||File type Wheel||Python version cp27||Upload date||Hashes View hashes|
|Filename, size multiprocess-0.70.9-cp27-cp27m-win_amd64.whl (92.6 kB)||File type Wheel||Python version cp27||Upload date||Hashes View hashes|
|Filename, size multiprocess-0.70.9.tar.gz (1.6 MB)||File type Source||Python version None||Upload date||Hashes View hashes|
Hashes for multiprocess-0.70.9-cp27-cp27m-win32.whl
Hashes for multiprocess-0.70.9-cp27-cp27m-win_amd64.whl