Streaming reactive and dataflow graphs in Python
Project description
Python Data Streams
Tributary is a library for constructing dataflow graphs in python. Unlike many other DAG libraries in python (airflow, luigi, prefect, dagster, dask, kedro, etc), tributary is not designed with data/etl pipelines or scheduling in mind. Instead, tributary is more similar to libraries like mdf, loman, pyungo, streamz, or pyfunctional, in that it is designed to be used as the implementation for a data model. One such example is the greeks library, which leverages tributary to build data models for options pricing.
Installation
Install with pip:
pip install tributary
or with conda:
conda install -c conda-forge tributary
or from source:
python setup.py install
Note: If installing from source or with pip, you'll also need Graphviz itself if you want to visualize the graph using the .graphviz()
method.
Stream Types
Tributary offers several kinds of streams:
Streaming
These are synchronous, reactive data streams, built using asynchronous python generators. They are designed to mimic complex event processors in terms of event ordering.
Functional
These are functional streams, built by currying python functions (callbacks).
Lazy
These are lazily-evaluated python streams, where outputs are propogated only as inputs change. They are implemented as directed acyclic graphs.
Examples
- Streaming: In this example, we construct a variety of forward propogating reactive graphs.
- Lazy: In this example, we construct a variety of lazily-evaluated directed acyclic computation graphs.
- Automatic Differentiation: In this example, we use
tributary
to perform automatic differentiation on both lazy and streaming graphs.
Graph Visualization
You can visualize the graph with Graphviz. All streaming and lazy nodes support a graphviz
method.
Streaming and lazy nodes also support ipydagred3 for live update monitoring.
Streaming
Here green indicates executing, yellow indicates stalled for backpressure, and red indicates that StreamEnd
has been propogated (e.g. stream has ended).
Lazy
Here green indicates executing, and red indicates that the node is dirty. Note the the determination if a node is dirty is also done lazily (we can check with isDirty
whcih will update the node's graph state.
Catalog
See the CATALOG for a full list of functions, transforms, sources, and sinks.
Support / Contributors
Thanks to the following organizations for providing code or financial support.
License
This software is licensed under the Apache 2.0 license. See the LICENSE file for details.
Alternatives
Here is an incomplete list of libraries which implement similar/overlapping functionality
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for tributary-0.2.1-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 206f033d090649dba375340c394a0f8874a4f59bdda123a216797ca866abf17c |
|
MD5 | 9ee6c1fed31dd3affd62566b95c90548 |
|
BLAKE2b-256 | 0656d66d6705d1fb42af81ca3abdd6e102e16f0ec7360691139345774e0dea19 |