Skip to main content
This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

Parallel computing framework

Project Description

link.parallel is a database agnostic query system.

See documentation for more informations.

Installation

pip install link.parallel

Features

  • parallel loops interface with IPython and multiprocessing drivers
  • Map/Reduce middleware

Examples

Create your map/reduce functions:

from b3j0f.task import register_task


@register_task('mymap')
def mymap(mapper, item):
    if item['i'] < 5:
        mapper.emit('i_lt_5', item)

    elif item['i'] > 5:
        mapper.emit('i_gt_5', item)

    else:
        mapper.emit('i_eq_5', item)


@register_task('myreduce')
def myreduce(key, values):
    return (key, len(values))

Get input data and pass it to the middleware:

from link.middleware.core import Middleware

mapreduce = Middleware.get_middleware_by_uri(
    'mapreduce+ipython:///test/classify?mapcb=mymap&reducecb=myreduce'
)

items = # load items
result = dict(mapreduce(items))

print(result)

Donating

Release History

Release History

This version
History Node

1.1

History Node

1.0

History Node

0.4

History Node

0.3

History Node

0.2

History Node

0.1

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
link.parallel-1.1-py2.py3-none-any.whl (12.6 kB) Copy SHA256 Checksum SHA256 3.4 Wheel Sep 15, 2016
link.parallel-1.1.tar.gz (6.4 kB) Copy SHA256 Checksum SHA256 Source Sep 15, 2016

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting