Make computer lazy and build a dynamic task graph for your project!
Build a dynamic task graph for your project with Task-Graph!
When facing a complex task, it's often necessary to recalculate the entire task when user wants to get one of the intermediate results, or to adjust one of the parameters or methods, but once you have Task-Graph,
If you need to get a result, Task-Graph will only compute all the upstream tasks of the result, and each task will cache for later use.
If one of the task's parameters or method is adjusted, Task-Graph will automatically check all the downstream tasks and clear their cached results.
In short, Task-Graph will make the computer lazy and just fulfill your needs without doing repetitive tasks, no more or less.
No doc available now, here is an minimal example.
# import from task_graph import TaskGraph # init a TaskGraph tg = TaskGraph() # user task methods def add(a, b): print("add", a, b) return a + b def sub(a, b): print("sub", a, b) return a - b # describe the task graph(much like dask.delayed) ret1 = tg.add_task(add)(1, 2) ret2 = tg.add_task(add)(3, ret1) ret3 = tg.add_task(add)(4, ret2) ret4 = tg.add_task(sub)(ret2, ret3) final = tg.add_task('to_list')(ret1, ret2, ret3, ret4) # trigger the computation and print print(final.compute()) # when I want to update a task tg.update_task(ret3)(add)(5, ret2) # Because ret3 is change, rest3, rest4 and final will be recalculated print(final.compute())
Or, a lazy way:
from task_graph import TaskGraph tg = TaskGraph() def add(a, b): print("add", a, b) return a + b def sub(a, b): print("sub", a, b) return a - b # omit add_task ret1 = tg(add)(1, 2) ret2 = tg(add)(3, ret1) ret3 = tg(add)(4, ret2) ret4 = tg(sub)(ret2, ret3) final = tg('to_list')(ret1, ret2, ret3, ret4) # auto compute and print final.print() # omit update_task tg(ret3)(add)(5, ret2) # auto compute and print final.print()
Task-Graph is at the very beginning, so many todos in the codes. Here are a few general ones:
- add parallel scheduler
- add decorator that can auto transfer function to task
- add graphviz-based visualization
- add unit test
- add doc
- add examples
I am lazy, so I build this to let computer be also lazy.
One of the biggest differences in Python from other languages is that the methods (like all other objects) are mutable, Task-Graph is designed to accommodate that.
Task-Graph was designed to speed up python project, and want to make it the simplest solution to avoid any recalculation.
I have referred to the Dask.delayed API in many places. The difference between Task-Graph and Dask is to try to adapt to the dynamics of the python and python projects, and make computer not only delayed but also lazy.
Release history Release notifications | RSS feed
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Hashes for task_graph-0.0.1-py3-none-any.whl