# Para(llel) processing utilities This library implements a simple set of parallel processing utilities that take advantage of python’s multiprocessing module to distribute processing over multiple CPUs on a single machine. The most salient feature of this library is the map() function that can be used to distribute CPU-intensive processing of a collection of items over multiple cores.
Installation pip install para
## Basic usage
>>> import para >>> import gzip >>> >>> items = ["examples/big-file1.gz", "examples/big-file2.gz", ... "examples/big-file3.gz"] >>> def log_lines(path): ... with gzip.open(path, 'rt') as f: ... for lineno, line in enumerate(f): ... if len(line) > 50: ... yield path, lineno, line ... >>> for path, lineno, line in para.map(log_lines, items): ... print(path, lineno, repr(line)) ... examples/big-file1.gz 2 'this line is going to be much longer than 80 chars -- at least I hope it will\n' examples/big-file3.gz 0 'again with the long lines -- this is going to show up in the output, I hope\n'
## Authors * Aaron Halfaker – https://github.com/halfak
Metadata
Release files for para 0.0.8
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| para-0.0.8.tar.gz | 5.4 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| para-0.0.8-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 11.9 kB
Release files / para-0.0.8.tar.gz
| Download URL | para-0.0.8.tar.gz |
|---|---|
| Size | 5.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
46c3232ae9d8ea9d886cfd08cdd112892202bed8645f40b6255597ba4cfef217
|
|
BLAKE2b-256 checksum How to use checksums |
a6ac192f715b8260eb05d7dfd822e9626583e2aa658f33afd407e2febbb37373
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.3 requests-toolbelt/0.9.1 tqdm/4.41.1 CPython/3.5.6
|
Release files / para-0.0.8-py3-none-any.whl
| Download URL | para-0.0.8-py3-none-any.whl |
|---|---|
| Size | 6.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
c63b030658cafd84f8fabfc000142324d51c7440e50ef5012fd1a54972ca25f4
|
|
BLAKE2b-256 checksum How to use checksums |
716255f612856021efa8fe7425289914f3641cca7eb9bc9374bf888597f37ce7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.3 requests-toolbelt/0.9.1 tqdm/4.41.1 CPython/3.5.6
|