Skip to main content

Framework for Graph-based execution and pipeline programming

Project description

GraphK

GraphK Toolkit Python 3.10+ License

Overview

GraphK is a Python framework for graph-based execution and pipeline programming. It models execution as explicit nodes, pipelines, runtime sessions, scoped belief state, policy checks, and a program hub that stores pipelines separately from execution.

The public surface is centered on:

  • Store and Matcher
  • Session, Belief, and Policy
  • Node
  • Pipeline, SequencePipe, BranchPipe, and MultiPipe
  • Program
  • Runner
  • Emitter

Report issues at GraphK Issues.

Getting Started

Install from PyPI when published:

pip install graphk

All examples use the public import form:

import graphk

Core Mental Model

The preferred architecture is:

  1. build one or more pipelines
  2. register them in a Program
  3. create an Emitter(program) or Runner(pipeline, session)
  4. dispatch a payload to a pipeline id
  5. read the final session from emitter.response

Program is the storage and binding hub. Emitter is a dispatcher. Runner is the transient execution engine.

Experiment 1: First Request And Response

What is being presented?

The smallest useful public GraphK flow: one program, one registered pipeline, one emitter, one session, one response.

What does it mean?

This example shows the preferred request/response surface after separating pipeline storage from execution.

import graphk


pipeline = graphk.SequencePipe(nodes=[graphk.demo.EchoNode()])
program = graphk.Program({"echo": pipeline})
emitter = graphk.Emitter(program)

emitter.request("echo", payload={"value": 5})

print(emitter.response)

Expected result:

{"echo": 5, "double": 10}

Experiment 2: Sequential Pipeline

What is being presented?

Several nodes running in a fixed order inside one pipeline registered under a program id.

What does it mean?

The pipeline is the execution architecture. The program stores it. The emitter dispatches a session to it.

import graphk


pipeline = graphk.SequencePipe(
    nodes=[
        graphk.demo.IncrementNode("A", increment=1, target_key="counter", order_key="order", response=True),
        graphk.demo.IncrementNode("B", increment=2, target_key="counter", order_key="order", response=True),
        graphk.demo.IncrementNode("C", increment=3, target_key="counter", order_key="order", response=True),
    ]
)

program = graphk.Program({"sequence": pipeline})
emitter = graphk.Emitter(program)
emitter.request("sequence", payload={"counter": 0})

print(emitter.response)

Expected result:

{"counter": 6, "order": ["A", "B", "C"]}

Experiment 3: Belief And Policy

What is being presented?

Scoped runtime values and rule-based execution checks flowing through Program, Runner, Pipeline, and Node.

What does it mean?

Program can hold shared belief state, Belief carries scoped execution data, and Policy checks entry and exit conditions against the active state.

import graphk


allow_entry = graphk.Policy(task="go", **{"@Node/stage": "work"})

pipeline = graphk.SequencePipe(
    nodes=[
        graphk.demo.IncrementNode(
            "Prepare",
            increment=2,
            target_key="counter",
            response=True,
            belief=graphk.Belief(stage="work"),
            entry_policy=allow_entry,
            exit_policy=graphk.Policy(counter=">=2"),
        )
    ]
)

program = graphk.Program(
    {"policy": pipeline},
    belief=graphk.Belief(mode="strict"),
)

emitter = graphk.Emitter(program)
emitter.request("policy", payload={"task": "go", "counter": 0})

print(emitter.response)

Experiment 4: Managed Execution With Runner

What is being presented?

The explicit execution engine for a selected pipeline.

What does it mean?

Runner is still the main execution API when session lifecycle and pipeline control need to remain visible. Program is only required when you want dispatch by pipeline id.

import graphk


pipeline = graphk.SequencePipe(
    nodes=[
        graphk.demo.IncrementNode("A", increment=1, target_key="counter"),
        graphk.demo.IncrementNode("B", increment=2, target_key="counter"),
    ]
)

session = graphk.Session(counter=0)
runner = graphk.Runner(pipeline, session)

runner.start()
runner.run()

print(session.to_dict())

Experiment 5: Branching And Fan-Out

What is being presented?

Two advanced routing forms: exclusive routing with BranchPipe and cumulative routing with MultiPipe, both registered under a Program.

What does it mean?

BranchPipe selects one matching branch. MultiPipe executes every matching branch. Emitter chooses which top-level pipeline to run by program id.

import graphk


branch = graphk.BranchPipe(
    nodes=[
        (graphk.Matcher(task="route-a"), graphk.SequencePipe(nodes=[graphk.demo.RouteNode("A")])),
        (graphk.Matcher(task="route-b"), graphk.SequencePipe(nodes=[graphk.demo.RouteNode("B")])),
    ]
)

multi = graphk.MultiPipe(
    nodes=[
        (graphk.Matcher(group="alpha"), graphk.SequencePipe(nodes=[graphk.demo.RouteNode("A")])),
        (graphk.Matcher(group="alpha"), graphk.SequencePipe(nodes=[graphk.demo.RouteNode("B")])),
    ]
)

program = graphk.Program(
    {
        "route": graphk.SequencePipe(nodes=[branch]),
        "fanout": graphk.SequencePipe(nodes=[multi]),
    }
)

emitter = graphk.Emitter(program)

route_session = emitter.request("route", payload={"task": "route-b"})
print(route_session.to_dict())

fanout_session = emitter.request("fanout", payload={"group": "alpha"})
print(fanout_session.to_dict())

Program And Emitter

Program stores pipelines in a registry:

program = graphk.Program()
program.set("main", pipeline)
pipeline = program.get("main")
ids = program.ids()

Emitter dispatches by pipeline id:

emitter = graphk.Emitter(program)
session = emitter.request("main", payload={"value": 10})
response = emitter.response

If the pipeline id is missing, Emitter.request() returns a session marked as error instead of crashing.

Learn More

The repository examples and tests work as learn-by-example material.

Examples:

Tests:

Wiki:

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

graphk-1.0.7.tar.gz (24.2 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

graphk-1.0.7-py3-none-any.whl (38.0 kB view details)

Uploaded Python 3

File details

Details for the file graphk-1.0.7.tar.gz.

File metadata

  • Download URL: graphk-1.0.7.tar.gz
  • Upload date:
  • Size: 24.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.14.3

File hashes

Hashes for graphk-1.0.7.tar.gz
Algorithm Hash digest
SHA256 fecc6e5affc5ddb273b7a63cd2d238a88bb5f0a1903b01e9753e42e387f5f554
MD5 e6b497976a8b8b7cc25d60db51c3e653
BLAKE2b-256 be6d7c068886b46aa2d4056d2d205d442a4122f9bbfdce0ca56eb87618638ac3

See more details on using hashes here.

File details

Details for the file graphk-1.0.7-py3-none-any.whl.

File metadata

  • Download URL: graphk-1.0.7-py3-none-any.whl
  • Upload date:
  • Size: 38.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.14.3

File hashes

Hashes for graphk-1.0.7-py3-none-any.whl
Algorithm Hash digest
SHA256 0c3bdc6d120ada4c06a57e3d1189c35f156f394b7cd344f73c5fe4ff527f3464
MD5 f5af7d2011571d3e35adab234ebf0624
BLAKE2b-256 34ddfc4f49cde51921d1aa204ba4891a7c15442f0575751ddea5035520c80769

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page