mapreduce-py 0.5.0

"A simple multi-processing based MapReduce framework"

A Multi-processing based single-host MapReduce Framework

This is a simple mapreduce framework.

Sample program

from mapreduce import *


def mapper_2(item):
    return (item +5)*23 - 1


def reducer_2(accumulated, item):
    return accumulated + item

def is_prime(n):
    for i in range(2,ceil(sqrt(n))):
        if n%i == 0:
            return False
    return True

if __name__ == "__main__":
    workers = 1
    N = 150000
    print('* map & reduce ')
    mr = MapReducer().workers(workers).mapper(mapper_2).reducer(reducer_2,0)
    result=mr(range(N))
    print('  MR Result  :', result)
    n=sum([(n +5)*23 - 1 for n in range(N)])
    print('  Validation:',n)

History

0.5.0

• process speed optimized

0.1.0

• First release