SynaFlow 🌊🧠
📖 Full documentation: humansoftware.github.io/synaflow
Write plain Python functions. SynaFlow builds the pipeline for you.
SynaFlow is a lightweight, pure-Python pipeline engine that uses Type Hints to automatically wire and execute Directed Acyclic Graphs (DAGs) with lockstep streaming and optional bounded handoff.
Why the name? Synapse + Flow. Just like synapses automatically wire neurons together, SynaFlow automatically wires your functions together based on their types. "Flow" represents the lazy, streaming nature of how data moves through those connections.
Quickstart
from collections.abc import Generator, Iterator
from typing import NamedTuple
from synaflow import pipeline, step, run
class Params(NamedTuple):
count: int
def producer(count: int) -> Generator[int, None, None]:
yield from range(count)
def transformer(producer: Iterator[int]) -> Generator[int, None, None]:
for val in producer:
yield val * 10
def consumer(transformer: Iterator[int]) -> None:
for x in transformer:
print(f"Consumed: {x}")
p = pipeline(
name="example",
params=Params,
steps=[
step("producer", fn=producer),
step("transformer", fn=transformer),
step("consumer", fn=consumer),
],
)
run(p, Params(count=5))
Three functions, three step() calls, zero manual wiring. SynaFlow reads the
type hints and wires the DAG automatically.
What makes it different
Type-hint wiring
Parameter names match producer names — SynaFlow connects them automatically.
Singular/plural/suffix synonyms work too (item → items, user_list → users).
Lazy streaming with bounded handoff
SynaFlow streams lazily by default. Multiple consumers can stay lockstep, one
consumer can stay lazy while another materializes, and when you need a bounded
window between stages you can set max_in_flight on the producing step.
This is especially useful for I/O-bound pipelines where one step starts work and the next resolves it, such as HTTP requests, RPC calls, or object-store reads.
Static validation at build time
Type errors, missing dependencies, circular graphs, mode conflicts — all caught
when pipeline(...) is called. Materialization decisions are compiled into the
Dag too: mode resolution, per-dependency eager materialization, and the
resolved materializer callables are frozen before run() starts. If it
compiles, it's valid. No runtime surprises.
Runtime overrides on top of the compiled contract
When you need test-time swaps without patching module globals, pass
ExecutionOverrides to run() or async_run() and replace only compiled
runtime dependencies such as materializers or observers. Use
PIPELINE_SCOPE for pipeline-level observers. The DAG shape and semantics stay
fixed; only the runtime callable changes.
from synaflow import ExecutionOverrides, Observer, PIPELINE_SCOPE, Scope
overrides = ExecutionOverrides.empty(p)
sub = Scope("payments")
overrides.resources["db"] = FakeDatabase()
overrides.observers[PIPELINE_SCOPE] = [Observer(noop_metrics)]
overrides.observers[sub.scope("validate")] = [Observer(test_recorder)]
overrides.materializers[sub.scope("normalize")] = list
For included sub-pipelines, Scope(...) is the public helper for addressing
compiled step keys without hardcoding "payments__validate" by hand.
Declared resources={...} are production factories. They are called when a
step is injected. ExecutionOverrides.resources is optional and only replaces
that provider when you want a different runtime value.
See also: docs/user_docs/advanced/testability.md
Build your own runner
The DAG compiles to a deterministic JSON contract. Write custom runners or auto-generate native DAGs for Airflow, Prefect, or Dagster.
How it compares
| SynaFlow | Hamilton | Airflow / Prefect / Dagster | |
|---|---|---|---|
| Auto wiring | ✅ type hints + smart binding | ✅ type hints (exact names) | ❌ explicit A >> B |
| Lazy streaming | ✅ lockstep + bounded handoff | ❌ DataFrame-centric | ❌ task-based |
| Smart binding | ✅ singular/plural/suffix | ❌ | ❌ |
| Scope | In-process micro | Feature engineering | Cluster orchestration |
| DAG export | ✅ JSON | ✅ | ✅ |
| Sync/async parity | ✅ identical | ❌ | ✅ |
Detailed comparisons: Hamilton · Java Streams · LINQ
Installation
pip install synaflow
Documentation
Start here: humansoftware.github.io/synaflow
| Section | Description |
|---|---|
| Tutorial | 5-level step-by-step guide building a pipeline from scratch |
| Core Concepts | How the DAG is wired, lockstep flow, max in flight, build vs run, event-based processing |
| Advanced | Testability, resource factories, runtime overrides, custom materializers, observers, and export guidance |
| Examples | Every corpus pipeline with auto-generated diagrams and source code |
| Comparisons | Detailed comparisons with Hamilton, Java Streams, and LINQ |
| Design Philosophy | Architectural decisions, contracts, and design rationale |
License
MIT License
1 maintainer
Metadata
1 maintainer
Release files for synaflow 0.32.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| synaflow-0.32.4.tar.gz | 343.9 kB | With provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| synaflow-0.32.4-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 433.7 kB
Release files / synaflow-0.32.4.tar.gz
| Download URL | synaflow-0.32.4.tar.gz |
|---|---|
| Size | 343.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
3fbe47a30a65041e766daf11a9b868b2020e5d61d7c792645dde90eb474006c5
|
|
BLAKE2b-256 checksum How to use checksums |
b54251ed2384a2895f3cdfc0ba804b307571f346e3265ac9c6b68159f5560aac
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 20, 2026.
Transparency logRelease files / synaflow-0.32.4-py3-none-any.whl
| Download URL | synaflow-0.32.4-py3-none-any.whl |
|---|---|
| Size | 89.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
8932a8596062a65e30ace8925ec6f98a6681e8c3ca2185495f21c236008e2cf0
|
|
BLAKE2b-256 checksum How to use checksums |
3ca33023c02e101214deb85da31deb07660eb8cce7c647678a5a5b81dc15dae7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 20, 2026.
Transparency logSecurity for synaflow 0.32.4
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
All 2 files in this release have been uploaded with provenance metadata.
Attestation for synaflow-0.32.4.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 20, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
