link.parallel

link.parallel is a database agnostic query system.

See documentation for more informations.

Installation

pip install link.parallel

Features

parallel loops interface with IPython and multiprocessing drivers

Map/Reduce middleware

Examples

Create your map/reduce functions:

from b3j0f.task import register_task


@register_task('mymap')
def mymap(mapper, item):
    if item['i'] < 5:
        mapper.emit('i_lt_5', item)

    elif item['i'] > 5:
        mapper.emit('i_gt_5', item)

    else:
        mapper.emit('i_eq_5', item)


@register_task('myreduce')
def myreduce(key, values):
    return (key, len(values))

Get input data and pass it to the middleware:

from link.middleware.core import Middleware

mapreduce = Middleware.get_middleware_by_uri(
    'mapreduce+ipython:///test/classify?mapcb=mymap&reducecb=myreduce'
)

items = # load items
result = dict(mapreduce(items))

print(result)

Donating

Metadata

Release files for link.parallel 1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for link.parallel 1.1
File	Size	Uploaded
link.parallel-1.1.tar.gz	6.4 kB	Sep 15, 2016	Details

Built distribution (wheel)

Table of built distributions (wheels) for link.parallel 1.1
File	Interpreter	ABI	Platform	Reset
link.parallel-1.1-py2.py3-none-any.whl 12.6 kB Sep 15, 2016	Python 2, Python 3	none	any	Details

Total release size: 19.0 kB

Release files / link.parallel-1.1.tar.gz

Download URL	link.parallel-1.1.tar.gz
Size	6.4 kB
Tags	Source
SHA-256 checksum How to use checksums	`404d346a1ade4479d9ce2db851a00e13605f3907e5dd3c40257a79ef99de0fbc`
BLAKE2b-256 checksum How to use checksums	`3d1a5e54c8dc78cb35dc09bcbf1b5eef2f413319e12c4d7266d0d0fa893767bb`
Upload date	Sep 15, 2016
Uploaded using Trusted Publishing? What is trusted publishing?	No

Release files / link.parallel-1.1-py2.py3-none-any.whl

Download URL	link.parallel-1.1-py2.py3-none-any.whl
Size	12.6 kB
Tags	Python 2 Python 3
SHA-256 checksum How to use checksums	`2a3f2d5facd6e6779189586adbd670967fb05a1b2fc8ebc0fd60b5ad6174b1b3`
BLAKE2b-256 checksum How to use checksums	`be9ab3c7b8605d91a35456af6cd4d2fb26e487cec226fd5c03b06e62414ad7c7`
Upload date	Sep 15, 2016
Uploaded using Trusted Publishing? What is trusted publishing?	No