xflow-py 0.2.0

A modular ML framework for training and evaluation tasks

Documentation · Report Bug · Request Feature

About the Project

XFlow is a lightweight modular machine-learning framework.

Originally created for physics research, it's now evolving toward generic scientific applications ML workflows: Data → Processing → Modeling

---

Core Data Processing Pipeline (Computational Map example)

flow is a step-based computation map for data processing.

Inputs (possibly different data types) move through discrete steps. At each step, a sample either passes through unchanged (identity) or is transformed by a node. Nodes can be multi-input and multi-output, so the map can split and merge data streams. Optional meta nodes (debug, checks, routing) can log, validate, stop, or redirect the pipeline without changing the core step structure.

%%{init: {"themeVariables": {"fontSize": "15px"}, "flowchart": {"htmlLabels": true}}}%%
flowchart TD
  classDef src fill:#0b1220,stroke:#334155,stroke-width:1px,color:#e2e8f0;
  classDef op fill:#0f172a,stroke:#38bdf8,stroke-width:2px,color:#e2e8f0;
  classDef io fill:#111827,stroke:#94a3b8,stroke-width:1px,color:#e5e7eb;
  classDef gate fill:#1f2937,stroke:#f59e0b,stroke-width:2px,color:#fde68a;
  classDef stop fill:#2a0f12,stroke:#fb7185,stroke-width:2px,color:#fecdd3;

  subgraph Inputs["Inputs"]
    DIR["dir: str<br/>/data/run_042"]:::src
    CFG["config: str<br/>YAML or JSON"]:::src
    A1["sensor A:<br/>array&lt;float&gt;"]:::src
    A2["sensor B:<br/>int"]:::src
  end

  READ["<b>ReadImages</b><br/>(dir -> images)"]:::op
  PARSE["<b>ParseConfig</b><br/>(str -> dict)"]:::op

  DIR --> READ
  CFG --> PARSE

  IMGS["images:<br/>tensor[H,W,C,N]"]:::io
  CONF["config:<br/>dict"]:::io

  READ --> IMGS
  PARSE --> CONF

  LOG["<b>LogConfig</b><br/>(print or save)"]:::op
  CONF --> LOG

  JOIN["<b>AlignAndEnrich</b><br/>(images -> 2 outputs)"]:::op
  IMGS --> JOIN

  subgraph JOIN_OUT[" "]
    direction LR
    ALN["aligned_images:<br/>tensor[...]"]:::io
    REP["report:<br/>md or json"]:::io
  end
  style JOIN_OUT fill:transparent,stroke:transparent

  JOIN --> ALN
  JOIN --> REP

  FUSE["<b>FuseSensors</b><br/>(2 signals -> 1 feature vector)"]:::op
  A1 --> FUSE
  A2 --> FUSE

  FEAT["features:<br/>vector&lt;float&gt;"]:::io
  FUSE --> FEAT

  GATE{"<b>QualityGate</b><br/>(meets requirements?)"}:::gate
  ALN --> GATE

  FIX["<b>Remediate</b><br/>(cleanup, re-run, notify)"]:::op
  STOP["STOP<br/>(fail fast)"]:::stop

  GATE -->|fail| FIX
  FIX --> STOP

  subgraph Outputs["Outputs"]
    OUT["artifacts:<br/>aligned_images + features + report"]:::io
  end

  ALN --> OUT
  FEAT --> OUT
  REP --> OUT

  GATE -->|pass| OUT

Getting Started

Installation

Install from PyPI:

pip install xflow-py

Clone the repository and install in editable mode:

git clone https://github.com/Andrew-XQY/XFlow.git
cd XFlow
pip install -e .

---

Built With

• Python 3.12
• TensorFlow 2.x
• Keras 3.x
• PyTorch 2.5.x

---

License

This project is licensed under the MIT License. See the LICENSE file for details.