Skip to main content

a dataflow library for python

Project description is an experimental port of larrytheliquid’s ruby dataflow gem, mostly to see if a python version (without blocks) would be useable. Turns out it is, which is not what I’d initially expected. I’m not really doing anything with it (or working on it), but hopefully it can be of use or interest to others.

dataflow functions:

  • dataflow_vars: decorator for generating dataflow variables for a function
  • spawn(callable, *args): start a thread using the given callable, plus any additional arguments
  • var(name=None): create a new dataflow variable (with optional name)
  • unify(var, value): set the value of a dataflow variable.

dataflow can provide arguments automatically:

def sum_items(x, y, z):
        # notice how the order automatically gets resolved
        spawn(lambda: unify(y, x() + 2))
        spawn(lambda: unify(z, y() + 3))
        spawn(lambda: unify(x, 1))
        return z() # => 6

or you can create them whenever you like:

f = var()
spawn(lambda: unify(f, 'f'))
f() # => 6

Accessing any attribute or item (dictionary key) of a dataflow variable automatically waits for it to be assigned, and passes that access onto its value:

f = var()
spawn(lambda: unify(f, {'key': 'val'})
f['key'] # => 'val'

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
dataflow-0.1.1.tar.gz (2.8 kB) Copy SHA256 hash SHA256 Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page