Simple DataPipeline Library
Project description
aksdp
Overview
A simple framework for writing data pipelines in Python
INSTALL
$ pip install git+https://github.com/YoshikazuArimitsu/aksdp.git
QuickStart
class TaskA(Task):
def main(self, ds):
return ds
class TaskB(Task):
...
class TaskC(Task):
...
class TaskD(Task):
...
graph = Graph()
task_a = graph.append(TaskA())
task_b = graph.append(TaskB(), [task_a])
task_c = graph.append(TaskC(), [task_b])
task_d = graph.append(TaskD(), [task_b, task_c])
graph.run()
Each task runs after each dependent task completes.
Also, the data output upstream can be received as input data and processed.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
aksdp-0.0.1.post1.tar.gz
(9.9 kB
view hashes)
Built Distribution
Close
Hashes for aksdp-0.0.1.post1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 81eade10edc3b7f44e2ac18412fe4b0525448a39f19a8e2f80b083b4de193c55 |
|
MD5 | 1fd1963ba2925bacae3377553dd1e173 |
|
BLAKE2b-256 | f61e3f3aa16b1a94335761d9ac1af6238e6978f3ff064d97fb3d08b67b4c81a1 |