Skip to main content

FlowMesh

arXiv License Python 3.12+ Lint Tests

A service fabric for running LLM agentic workflows on distributed GPU workers.

FlowMesh accepts workflow definitions (YAML, JSON, or n8n graph format), parses them into a DAG of tasks, schedules and dispatches each task to a suitable worker, and collects results and artifacts. It supports inference (vLLM, HF transformers, diffusers), LLM fine-tuning (SFT, LoRA, DPO, PPO), image classification training, retrieval-augmented generation, agent execution, SSH-style interactive sessions, arbitrary container jobs, and miscellaneous data/utility tasks (data profiling, data retrieval, embeddings, HTTP API calls, echo).

Architecture

Client (CLI / SDK / HTTP)
    │
    ▼  HTTP (default 8000)
┌─────────────────────────────────────────────────┐
│ Server  (FastAPI orchestrator)                  │
│   • workflow parsing, DAG resolution            │
│   • task scheduling and dispatch                │
│   • result and artifact collection              │
│   • REST API + SSE log streaming                │
└──────────────┬──────────────────────────────────┘
               │  Redis pub/sub  (control + telemetry)
               ▼
┌─────────────────────────────────────────────────┐
│ Supervisor  (per-node agent)                    │
│   • registers node, manages worker containers   │
│   • relays tasks/events via gRPC streams        │
└──────────────┬──────────────────────────────────┘
               │  gRPC (default 50051)
               ▼
┌─────────────────────────────────────────────────┐
│ Worker  (executor process)                      │
│   • vllm, transformers, diffusers, training,    │
│     RAG, agent, SSH, echo, data profiling       │
│   • streams logs and events back via gRPC       │
└─────────────────────────────────────────────────┘

Server and Worker are the two top-level processes. The Supervisor is a subsystem that lives under src/server/ and runs as a child process spawned from the server (multiprocessing.Process); single-node deployments spawn one supervisor child alongside the server, multi-node deployments run a root server plus one supervisor-only server process per worker node.

Quick start

FlowMesh targets Linux only. Requires Docker, Docker Compose, and Python 3.12+. If you want to use GPU workers, ensure the NVIDIA Container Toolkit is also installed.

# 1. Install
git clone https://github.com/mlsys-io/FlowMesh.git
cd FlowMesh
pip install uv
uv sync --all-packages --group ci

# 2. Bring up the local stack (Server + Redis + Supervisor)
uv run flowmesh stack up

# 3. Start one CPU worker
uv run flowmesh stack worker up cpu 1

# 4. Submit a workflow
uv run flowmesh workflow submit examples/templates/echo_local.yaml

# 5. Watch it run
uv run flowmesh workflow list
uv run flowmesh workflow watch <workflow_id>

For a GPU worker:

# Pin to specific GPUs (or 'all')
uv run flowmesh stack worker up gpu --targets 0

For inference templates:

uv run flowmesh workflow submit examples/templates/inference_vllm_chat.yaml
uv run flowmesh workflow submit examples/templates/inference_hf_chat.yaml

Tear down:

uv run flowmesh stack worker down all
uv run flowmesh stack down

Workflow format

A minimal single-task workflow:

apiVersion: flowmesh/v1
kind: InferenceTask
metadata:
  name: hello-inference
spec:
  taskType: inference
  resources:
    hardware: { gpu: { type: any, count: 1 } }
  model:
    source: { type: huggingface, identifier: TinyLlama/TinyLlama-1.1B-Chat-v1.0 }
    vllm: { gpu_memory_utilization: 0.5 }
  data:
    type: list
    items:
      - - role: user
          content: What is the capital of France?
  inference: { max_tokens: 64, temperature: 0.0 }
  output:
    destination: { type: http }

Multi-stage DAGs, conditional execution, graph-template prompts, task merging, and SSH sessions are all supported. See examples/templates/ for end-to-end examples and AGENTS.md for the full schema reference.

Extending FlowMesh

FlowMesh exposes plugin hooks for organisations that want to layer additional auth, submission policy, usage tracking, authorisation, supplier attribution, or resource lifecycle behaviour on top of the core server. Install the standalone hook contract with:

pip install "flowmesh[hook]"

A plugin is any Python module that exposes install() returning flowmesh_hook.HookBindings. Plugins are loaded by setting FLOWMESH_PLUGINS to a comma-separated list of importable module names. Plugins can ship as in-tree modules, sibling-mounted packages, or pip-installable wheels — the core never references plugin names.

See docs/PLUGINS.md for the full plugin contract.

Development

# Install dev tooling
uv sync --all-packages --group ci

# Format / lint / type-check
uv run pre-commit run --all-files

# Tests — skip the multiprocessing GPU-cleanup test because it requires a
# real CUDA device and isolated processes; CI also skips it.
uv run pytest tests/ --ignore=tests/worker/test_mp_executor_cleanup_gpu.py

Detailed contributor docs (project layout, env vars, dispatch internals, executor registry, commit-message conventions) live in AGENTS.md.

Contributing

We welcome bug fixes, new features, documentation improvements, and feedback. Please read CONTRIBUTING.md for the contributor setup, code style, testing, dependency-pin, and DCO sign-off conventions, and AGENTS.md for a deeper architecture and source-layout tour.

Citation

If you use FlowMesh in your research, please cite:

@misc{shen2025flowmesh,
      title={FlowMesh: A Service Fabric for Composable LLM Workflows}, 
      author={Junyi Shen and Noppanat Wadlom and Lingfeng Zhou and Dequan Wang and Xu Miao and Lei Fang and Yao Lu},
      year={2025},
      eprint={2510.26913},
      archivePrefix={arXiv},
      primaryClass={cs.DC},
      url={https://arxiv.org/abs/2510.26913}, 
}

License

Apache License 2.0. See LICENSE.

Metadata

Release files for flowmesh 0.1.14

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for flowmesh 0.1.14
File Size Uploaded Provenance Actions
flowmesh-0.1.14.tar.gz 10.7 kB Without provenance Details for flowmesh-0.1.14.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for flowmesh 0.1.14
File Interpreter ABI Platform Actions
flowmesh-0.1.14-py3-none-any.whl Python 3 none any Details for flowmesh-0.1.14-py3-none-any.whl

Total release size: 19.0 kB

Release files / flowmesh-0.1.14.tar.gz

Download URL flowmesh-0.1.14.tar.gz
Size 10.7 kB
Tags Source
SHA-256 checksum
How to use checksums
ab7a7c53c1104bc5e2ba0047d328185da08958d0b3a904f78cc7bbb41c8fede6
BLAKE2b-256 checksum
How to use checksums
070e6afbd79de48d58bd1d51b2247d1f50f5fea14d1b54e8b5372db55d1ebbbe
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.12.12

Release files / flowmesh-0.1.14-py3-none-any.whl

Download URL flowmesh-0.1.14-py3-none-any.whl
Size 8.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
d61fc89fb2c12926a06a5558814500cc2150dba020426bad8aa2b0c0647aff02
BLAKE2b-256 checksum
How to use checksums
f2b75424245d0a6247f2be86fbd5424689b4a7630c7e142dcbb8056af8d77083
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.12.12

Security for flowmesh 0.1.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page