Skip to main content

Streaming reactive and dataflow graphs in Python

Project description

Python Data Streams

Build Status Coverage PyPI PyPI Binder

Tributary is a library for constructing dataflow graphs in python. Unlike many other DAG libraries in python (airflow, luigi, prefect, dagster, dask, kedro, etc), tributary is not designed with data/etl pipelines or scheduling in mind. Instead, tributary is more similar to libraries like mdf, loman, pyungo, streamz, or pyfunctional, in that it is designed to be used as the implementation for a data model. One such example is the greeks library, which leverages tributary to build data models for options pricing.

Installation

Install with pip:

pip install tributary

or with conda:

conda install -c conda-forge tributary

or from source:

python setup.py install

Note: If installing from source or with pip, you'll also need Graphviz itself if you want to visualize the graph using the .graphviz() method.

Stream Types

Tributary offers several kinds of streams:

Streaming

These are synchronous, reactive data streams, built using asynchronous python generators. They are designed to mimic complex event processors in terms of event ordering.

Functional

These are functional streams, built by currying python functions (callbacks).

Lazy

These are lazily-evaluated python streams, where outputs are propogated only as inputs change. They are implemented as directed acyclic graphs.

Examples

  • Streaming: In this example, we construct a variety of forward propogating reactive graphs.
  • Lazy: In this example, we construct a variety of lazily-evaluated directed acyclic computation graphs.
  • Automatic Differentiation: In this example, we use tributary to perform automatic differentiation on both lazy and streaming graphs.

Graph Visualization

You can visualize the graph with Graphviz. All streaming and lazy nodes support a graphviz method.

Streaming and lazy nodes also support ipydagred3 for live update monitoring.

Streaming

Here green indicates executing, yellow indicates stalled for backpressure, and red indicates that StreamEnd has been propogated (e.g. stream has ended).

Lazy

Here green indicates executing, and red indicates that the node is dirty. Note the the determination if a node is dirty is also done lazily (we can check with isDirty whcih will update the node's graph state.

Catalog

See the CATALOG for a full list of functions, transforms, sources, and sinks.

Support / Contributors

Thanks to the following organizations for providing code or financial support.

Nemoulous

License

This software is licensed under the Apache 2.0 license. See the LICENSE file for details.

Alternatives

Here is an incomplete list of libraries which implement similar/overlapping functionality

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tributary-0.2.1.tar.gz (81.7 kB view details)

Uploaded Source

Built Distribution

tributary-0.2.1-py2.py3-none-any.whl (113.4 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file tributary-0.2.1.tar.gz.

File metadata

  • Download URL: tributary-0.2.1.tar.gz
  • Upload date:
  • Size: 81.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.3

File hashes

Hashes for tributary-0.2.1.tar.gz
Algorithm Hash digest
SHA256 7f99e8257e989fabf5bb60b06dea5dd10d13c14b90de93490855d5b68d6c6fe6
MD5 867c53dfd794976456d8071735cf3e77
BLAKE2b-256 fbcfcf7180aa27943a361783c8bde4eb888cb63b5ac9da626df33151a66fa9df

See more details on using hashes here.

File details

Details for the file tributary-0.2.1-py2.py3-none-any.whl.

File metadata

  • Download URL: tributary-0.2.1-py2.py3-none-any.whl
  • Upload date:
  • Size: 113.4 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.3

File hashes

Hashes for tributary-0.2.1-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 206f033d090649dba375340c394a0f8874a4f59bdda123a216797ca866abf17c
MD5 9ee6c1fed31dd3affd62566b95c90548
BLAKE2b-256 0656d66d6705d1fb42af81ca3abdd6e102e16f0ec7360691139345774e0dea19

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page