Run tasks as a DAG
Project description
Python DAG Runner
Python DAG Runner is a Python library for running Directed Acyclic Graphs (DAGs) efficiently. It provides a simple and intuitive interface for defining and executing complex workflows. It uses networkx library to generate directed acyclic graph from the tasks and it's dependencies.
Installation
You can install python-dag-runner library easily using pip:
pip install python-dag-runner
Usage
You can easily define a dag in python-dag-runner using Dag class. Furthermore, task can be defined using Task class. You can easily define dependencies of tasks using the "|=" operator.
from python_dag_runner import Dag, Task
# Create an executable function
def print_task_1():
"""A sample function"""
print("Task 1 called")
def print_task_1():
"""A sample function"""
print("Task 2 called")
# Create Task objects
task_1 = Task(name="task one", executable=print_task_1)
task_2 = Task(name="task two", executable=print_task_2)
# Set dependencies of task using bitwise or operator
task_2 |= {task_1}
# Create a DAG by providing name and sequence of tasks
dag = Dag("My first DAG", tasks=[task_1, task_2])
# Initiate execution of Dag using initialize method
dag.initiate()
Example of complex dependencies
Multiple internal dependencies can be defined using dependency operator.
task_2 |= {task_1}
task_3 |= {task_2}
task_4 |= {task_2}
task_5 |= {task_2}
task_6 |= {task_3, task_4, task_5}
task_7 |= {task_6}
task_8 |= {task_7}
task_9 |= {task_7}
The above steps result in the dependency graph as shown below
%%{init: {'theme': 'neutral'}}%%
stateDiagram
direction LR
task_one --> task_two
task_two --> task_three
task_two --> task_four
task_two --> task_five
task_three --> task_six
task_four --> task_six
task_five --> task_six
task_six --> task_seven
task_seven --> task_eight
task_seven --> task_nine
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for python_dag_runner-0.0.5-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 07ca58d732ec2f6c36f8fc8a16d80c58a9b92d4054b2ae390837c9ce960a486b |
|
MD5 | f685b8b039025cc966f85129ba1345c6 |
|
BLAKE2b-256 | 465092252d061a1c49b9d9c01d3f18631e7b860f4e3aa48102576c38ab8e121f |