Skip to main content

A library for incremental, in-memory map-reduces

Project description

The Scenic Overlook library contains datastructures for incremental map-reduces.

These datastructures are implemented as trees, and store at each node, intermediate values of the reduce. This means that when you slice or combine structures, the new output of the maps/reduces can be efficiently computed. (by reusing old outputs from unchanged parts of the tree)

Typical usage looks like this:

#!/usr/bin/env python

from scenicoverlook import viewablelist

space_concat = lambda x, y: x + ' ' + y
l = viewablelist(['the', 'quick', 'brown', 'fox'])
print l.reduce(space_concat)

# This yields 'the quick stealthy brown fox', reusing cached intermediate
# substrings from the earlier call like 'the quick' and 'brown fox':

print (l[:2] + ['stealthy'] + l[2:]).reduce(space_concat)

See the pydocs for more examples:

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for ScenicOverlook, version 0.3.0
Filename, size File type Python version Upload date Hashes
Filename, size ScenicOverlook-0.3.0-py2.py3-none-any.whl (13.5 kB) File type Wheel Python version py2.py3 Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page