Skip to main content

# Para(llel) processing utilities This library implements a simple set of parallel processing utilities that take advantage of python’s multiprocessing module to distribute processing over multiple CPUs on a single machine. The most salient feature of this library is the map() function that can be used to distribute CPU-intensive processing of a collection of items over multiple cores.

  • Installation pip install para

## Basic usage

>>> import para
>>> import gzip
>>>
>>> items = ["examples/big-file1.gz", "examples/big-file2.gz",
...          "examples/big-file3.gz"]
>>> def log_lines(path):
...     with gzip.open(path, 'rt') as f:
...         for lineno, line in enumerate(f):
...             if len(line) > 50:
...                 yield path, lineno, line
...
>>> for path, lineno, line in para.map(log_lines, items):
...     print(path, lineno, repr(line))
...
examples/big-file1.gz 2 'this line is going to be much longer than 80 chars -- at least I hope it will\n'
examples/big-file3.gz 0 'again with the long lines -- this is going to show up in the output, I hope\n'

## Authors * Aaron Halfaker – https://github.com/halfak

Metadata

Release files for para 0.0.8

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for para 0.0.8
File Size Uploaded
para-0.0.8.tar.gz 5.4 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for para 0.0.8
File Interpreter ABI Platform
para-0.0.8-py3-none-any.whl Python 3 none any Details

Total release size: 11.9 kB

Release files / para-0.0.8.tar.gz

Download URL para-0.0.8.tar.gz
Size 5.4 kB
Tags Source
SHA-256 checksum
How to use checksums
46c3232ae9d8ea9d886cfd08cdd112892202bed8645f40b6255597ba4cfef217
BLAKE2b-256 checksum
How to use checksums
a6ac192f715b8260eb05d7dfd822e9626583e2aa658f33afd407e2febbb37373
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.3 requests-toolbelt/0.9.1 tqdm/4.41.1 CPython/3.5.6

Release files / para-0.0.8-py3-none-any.whl

Download URL para-0.0.8-py3-none-any.whl
Size 6.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
c63b030658cafd84f8fabfc000142324d51c7440e50ef5012fd1a54972ca25f4
BLAKE2b-256 checksum
How to use checksums
716255f612856021efa8fe7425289914f3641cca7eb9bc9374bf888597f37ce7
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.3 requests-toolbelt/0.9.1 tqdm/4.41.1 CPython/3.5.6

Release history Release notifications | RSS feed

This release

0.0.8 This release

2 release files

0.0.7

2 release files

0.0.6

2 release files

0.0.5

2 release files

0.0.4

2 release files

0.0.3

2 release files

0.0.2

2 release files

0.0.1

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page