pyumpf
Python Unified Multiprocessing Parallel Functions
This is a small library that aims to bring together the standard library map/reduce functions with both the multiprocessing module in the standard library and the PP module (www.parallelpython.com).
Each has their own niche, and there own advantages and disadvantages. The map/reduce functions in the standard library are good for implementing a functional paradigm, but are limited by the GIL to only run one task at a time. The multiprocessing module is good for single machines with multiple CPUs, but can be awkward to debug and use. The PP module is great for clusters, but is even more awkward to debug and use. Plus all of these have different APIs, and so cannot be drop-in replacements.
pyumpf gets around this by providing a unified interface via umpf.map and umpf.reduce that can easily be expanded to use multiprocessing or PP when needed, but also collapsed to the built-in map/reduce for simpler debugging.
To install: python setup.py install
For an example, see umpf_test.py Note that this example gives PP a poor performance. This is because it is running on a single machine, and this example is too small to make best use of the parallelism. Ideally, tasks should run for several seconds each and have easily pickleable arguments and returned values.
By default, umpf defaults to pythons built-in map/reduce. These are single-threaded.
To use multiprocessing, do the following:
import umpf import multiprocessing as mp umpf.Hub.pool = mp.Pool()
To use Parallel Python, do the following:
import umpy import pp umpy.Hub.pool = pp.Server()
Metadata
Release files for pyumpf 0.1.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| pyumpf-0.1.1.tar.gz | 3.5 kB | Without provenance | Details for pyumpf-0.1.1.tar.gz |
Release files / pyumpf-0.1.1.tar.gz
| Download URL | pyumpf-0.1.1.tar.gz |
|---|---|
| Size | 3.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
2f2e5a5f592ca6adb97c2cbe6e11cd1a8e99d0f9925ac0d065d72fa65bd83e1f
|
|
BLAKE2b-256 checksum How to use checksums |
38f2973765bd9b801320c200d90026fc7d25740c327cb310c97b2d645b46da20
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
Security for pyumpf 0.1.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.