Skip to main content

A Python library for prototyping MapReduce jobs

Project description

Documentation Status


mockr is a Python library for writing MapReduce jobs in an Educational setting. It is intended to be used as a conceptual teaching tool.

mockr provides an interface for defining and running MapReduce locally. Simply define your map and reduce functions, input your data and call the run function. Everything is run sequentially and locally.


pip install mockr


Full documentation available here

Streaming Jobs

StreamingJob class which expects the input to be a byte stream of characters. The chunks of data are separated by newline ("\n") characters. Each line is sent to a separate map worker.

Native Python Sequence Jobs

PythonJob class expects input to be a type object e.g. Python List. Python Jobs provide two exection methods:

  • the sequence is divided into chunks and each chunk is sent to a separate map worker
  • each item in the list is individually sent to a dedicated map worker

Pandas Jobs

PandasJob class expects input to be a Pandas DataFrame. The rows of the data frame are equally divided into chunks and each chunk is sent to a separate map worker

Example Usage

import re
from mockr import run_stream_job

WORD_RE = re.compile(r"[\w']+")

def map_fn(chunk):
    # yield each word in the line
    for word in WORD_RE.findall(chunk):
        yield (word.lower(), 1)

def reduce_fn(key, values):
    yield (key, sum(values))

input_str = "Hello!\nThis is a sample string.\nIt is very simple.\nGoodbye!"

results = run_stream_job(input_str, map_fn, reduce_fn)



[('hello', 1), ('this', 1), ('is', 2), ('a', 1), ('sample', 1), ('string', 1), ('it', 1), ('very', 1), ('simple', 1), ('goodbye', 1)]

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for mockr, version 0.36
Filename, size File type Python version Upload date Hashes
Filename, size mockr-0.36-py3-none-any.whl (4.1 kB) File type Wheel Python version py3 Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page