Skip to main content

No project description provided

Project description

DagStream

DagStream is the Python package in order to manage relationship between functions, especially for data-preprocessing functions for machine learning applications. It offers simple API similar to taskflow.

For more details, please refer to docs below.

How to install

You can intall dagstream by using pip.

pip install dagstream

Basic Usage

Definition of Dag

DagStream class convert your functions into dag nodes.

import dagstream

def funcA():
    print("funcA")

def funcB():
    print("funcB")

def funcC():
    print("funcC")

def funcD():
    print("funcD")

def funcE():
    print("funcE")

def funcF():
    print("funcF")


stream = dagstream.DagStream()
A, B, C, D, E, F = stream.emplace(funcA, funcB, funcC, funcD, funcE, funcF)

A.precede(B, C)
E.succeed(B, C, D)
D.succeed(C)
F.succeed(E)

Execute whole dag functions

dagstream execute the functions based on relationship you defined.

from dagstream.executor import StreamExecutor

# construct functional dag
functional_dag = stream.construct()
executor = StreamExecutor(functional_dag)
executor.run()

In console, follwing items are shown.

funcA
funcC
funcB
funcD
funcE
funcF

Draw relationship of functions using Mermaid

You can draw function dag relationship by using Mermaid.

from dagstream.viewers import MermaidDrawer

functional_dag = stream.construct()
drawer.output(functional_dag, "path/to/output.mmd")

The output is shown like below.

stateDiagram
    direction LR
    state "funcE" as state_0
    state "funcB" as state_1
    state "funcF" as state_2
    state "funcD" as state_3
    state "funcC" as state_4
    state "funcA" as state_5
    state_0 --> state_2
    state_1 --> state_0
    state_2 --> [*]
    state_3 --> state_0
    state_4 --> state_0
    state_4 --> state_3
    [*] --> state_5
    state_5 --> state_4
    state_5 --> state_1

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

dagstream-0.1.3-py3-none-any.whl (11.2 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page