SageFlownet: A distributed dataflow orchestration system. PyPI name: isage-flownet, import as: sage.flownet
Project description
SAGE-Flownet
SAGE-Flownet 是一个轻量、原生 Python 的动态数据流执行引擎。它以“嵌套动态图”为核心模型,把用户写下的普通 Python 函数编译为可执行的 FlowTask DAG,并由运行时按需调度执行,让复杂的工作流像调用函数一样自然。
它不是另一个固定模板的流水线框架。Flownet 将远程调用、闭包、回调以及子图嵌套视为一等公民,在不牺牲 Python 语义的前提下,支持灵活组合和横向扩展。
安装与使用
pip install isage-flownet
⚠️ PyPI 包名 vs. import 名
SAGE 生态约定:PyPI 包名使用
isage-前缀(因为sage在 PyPI 已被占用), 但 Python import 名使用sage.*命名空间。
操作 命令 安装 pip install isage-flownetimport from sage.flownet import flow, init
import isage_flownet不存在,请始终使用sage.flownet。
from sage.flownet import flow, init, create_actor
@flow
def my_pipeline(init_stream):
return init_stream.map(lambda x: x * 2)
init() # 启动运行时
✧ 项目如何工作(概述)
- Flow 定义与编译:通过
@flow装饰器将函数包装为 Flow 定义(FlowDef),编译为包含 Transformation 的 FlowTask 图。 - 数据流建模:
DataStream.map/flatmap等算子创建上下游关系,形成可追踪的依赖图和执行边。 - 运行时执行:
FlownetRuntime负责处理请求栈(TransformationFrame),并由 operator handler 决定目标调用与转发策略。 - Actor 调用统一化:
ActorRef/ActorMethodRef将本地与远程调用统一抽象,运行时根据目标地址完成派发。 - FlowRun 输出通道:运行时通过
FlowRunRef的输出通道暴露“写入输入—等待输出”的调用方式,负责 fork/join 逻辑与结果收集。
✧ 预期目标与效果
- 保持原生 Python 开发体验:无需额外 DSL,工作流像写函数一样简单。
- 支持动态与嵌套的复杂流程:运行时编译、依赖跟踪与延迟执行适配不确定结构。
- 可扩展的分布式执行:为本地/远程统一执行提供清晰路径,便于水平扩展。
- 更可靠的并发与结果聚合:由运行时完成 fork/join、收集与异常处理,降低业务代码复杂度。
Intellectual Property
All Rights Reserved applies to repository contents authored by Jingyuan Tian,
including code, documentation, benchmarks, and related assets.
The following copied lab projects are excluded from this claim and retain their
original ownership/license context: applications/SAGE, applications/sage-studio,
applications/sageData.
- License terms:
LICENSE - Ownership details:
COPYRIGHT.md
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file isage_flownet-0.1.4.tar.gz.
File metadata
- Download URL: isage_flownet-0.1.4.tar.gz
- Upload date:
- Size: 196.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.11.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
59d0c2054adbedd2deacca82920c1890725c98cff66f3db499617291ca24be8f
|
|
| MD5 |
69169b05793c7615f1303b9a3b47ca5b
|
|
| BLAKE2b-256 |
d295c91ff6b2bf02788727baf67dc611c5c3104c591fd86f07091829cd138aa9
|
File details
Details for the file isage_flownet-0.1.4-py2.py3-none-any.whl.
File metadata
- Download URL: isage_flownet-0.1.4-py2.py3-none-any.whl
- Upload date:
- Size: 259.1 kB
- Tags: Python 2, Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.11.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
e8603275f37403df4c8c696de185287b2276ed3fa88cfafa7eedf788ef22c916
|
|
| MD5 |
46300091ca4870d61a9458c1eb528db6
|
|
| BLAKE2b-256 |
7a34086e9357e9b593d356da70e361203ee4805b1fc2a10de0235baf5ab166b5
|