Polyad

Polyad (named after polyads in mathematics) is a Kubernetes operator for deploying, connecting and scaling applications as graphs. Compose batch jobs, persistent services and supporting resources into reusable Graphs and PolyGraphs. Define how work starts, how components communicate and which structural constraints must hold as the application changes—from a workflow in one cluster to a hierarchy spanning multiple clusters.^[2]

Get started · Documentation · Helm chart

• Compose workloads and resources. Run finite pipelines and persistent services using Jobs, Deployments, StatefulSets or DaemonSets, with dependencies, activation policies, storage and placement.
• Scale within graph constraints. KEDA and ReplicaGroups scale individual services, whole graphs or nested compositions. Polyad refreshes live graph state and enforces GraphRules, including size, shape and structural Cheeger bounds, before applying scaling changes. Optional Soul searching, Polyad's topology optimizer, uses application measurements to recommend or apply approved connection layouts and bounded Istio traffic splits. Demand selects a separate Cheeger target; routing percentages come from calibrated demand tiers or each replica's completed throughput and reported spare capacity. Fixed percentages also work without feedback. These routes can balance Workload, Daemon, Graph, PolyGraph or nested ReplicaGroup replicas through their service entrypoints.
• Make connectivity explicit. Choose replica connection patterns, including custom edges, and enforce network boundaries with optional NetworkPolicy and Istio integration.
• Let services participate. Through the standalone Python client, workloads can submit compositions, activate work, request TTL-bound connections and discover neighbors through topology events. Shared types are also available separately from the operator.
• Coordinate across clusters. A root operator can run in a dedicated management cluster, deploy graphs and execution replicas into registered workload clusters, and collect their observations centrally.
• Choose the control-plane layout. Select a single-replica or HA Helm profile. HA can run dense operators or separate gateway, executor and telemetry components managed through the operator's own Graph. Optionally persist graph state and tracked measurements in PostgreSQL.

Table of contents

• Polyad
  • Table of contents
  • Get started
  • What Polyad abstracts
    • Motivation
    • Graphs of graphs
    • Replica connections
    • Autoscaling the hierarchy
    • Constrained compositions
    • Network boundaries
    • Graphs across clusters
    • Graphs across node groups
    • Workloads calling the operator
    • Finite pipelines
    • Persistent services and recurrence
    • The operator as a Graph
  • What Polyad is not
  • License
  • References

Get started

Deploy the Kubernetes operator with the Helm chart, or try the local Python scheduler. The examples cover pipelines, services, spot work, storage and nested graphs.

Applications can install the Python client or just the shared types without installing the operator. From a checkout, use pip install ./pkg/polyad-types; the standalone distribution is named polyad-types and exposes polyad_types.

Explore graph concepts, graph rules, composition requests, the composition API, networking and event subscriptions, temporary connections, API keys and shared request lanes, or the development guide. See the documentation index for lifecycle, status, health and configuration references.

What Polyad abstracts

A graph groups related work and describes how its parts depend on each other. Its nodes can be tasks, services, resources or other graphs. A data pipeline might fetch records, process partitions in parallel, then publish the results; a service graph might keep consumers and their supporting resources running.^[3]

Motivation

While at Klaviyo, I briefly worked alongside engineers who had come from Medium. They described a setup where clusters managed other clusters. That idea helped motivate Polyad's root control plane and nested PolyGraphs. Medium's Kubernetes Infrastructure At Medium provides public background on its multi-cluster infrastructure, gradual rollouts and capacity planning.

Deploying a distributed application means deciding how its services connect, which work can run together, and how those relationships should change as demand grows. Polyad makes that topology an explicit, reusable part of the deployment, with rules that constrain its size, structure and permitted connections across nested graphs.

This lets teams scale individual services, complete pipelines or compositions of graphs while preserving the application's topology requirements. KEDA requests replica counts through ReplicaGroups; Polyad recomputes the live graph family's measurements and checks applicable constraints before creating or retiring copies. A scaling decision must fit the surrounding application's rules as well as the group's own limits.

For data pipelines, Cheeger bounds constrain structural bottlenecks. A minimum requires enough distinct connections across every split relative to the number of vertices on its smaller side, at each configured boundary. For example, a minimum of 1 requires at least three crossing connections for a split whose smaller side contains three vertices. This rejects overly sparse topologies; achieved throughput also depends on processing capacity, bandwidth and the work each request requires.

Soul searching connects application measurements to two possible actions: change the approved connection layout, or redistribute incoming traffic among connected replicas. Hard GraphRules bounds remain separate from the application-driven Cheeger target selected by a calibrated demand tier. Every automatic change must satisfy both, with fresh graph-family checks, stabilization, cooldowns and a change budget. Observe mode reports recommendations; Adapt permits bounded changes.

With optional Istio traffic balancing, Tiers mode selects configured percentages after a sustained throughput shortfall. Headroom mode uses each destination's completed work plus its reported additional sustainable capacity to rebalance under positive demand, even before aggregate throughput falls. Both modes can redistribute requests among Workload, Daemon, Graph, PolyGraph and nested ReplicaGroup copies through compatible service entrypoints. KEDA and ReplicaGroups separately control how many copies exist.

Both Cheeger bounds use the same unweighted structural measurement. Changing a traffic split from 90/10 to 50/50 can improve application throughput without changing the Cheeger value. A zero-percent destination still contributes its declared connection until that edge is removed. Traffic percentages and measured capacity are separate inputs to routing; they do not turn Cheeger into a measured data rate. See comparing Cheeger policies and traffic balancing between replicas for diagrams of how these controls work together.

Services can also participate in changing their own topology. By installing the Python client, applications can submit compositions, activate work and request temporary connections with a bounded lifetime through enabled, authorized APIs. Polyad checks requested changes against the applicable rules and removes temporary connection grants after expiry. Topology events let workloads discover their current neighbors as connections and replica membership change. This gives applications room to adapt while keeping deployment constraints under operator control.

Graphs of graphs

Compose smaller workflows into an application with PolyGraph. Each child reports progress to its parent, giving the root a combined view of the work.^[4]

In the diagrams below, green marks work and graph summaries, amber marks constraints or recurrence, and gray marks resources and containing boundaries.

---
config:
  theme: base
  htmlLabels: false
  themeVariables:
    primaryTextColor: "#163b29"
    secondaryTextColor: "#513900"
    tertiaryTextColor: "#344054"
    clusterBkg: "#f2f4f7"
    clusterBorder: "#667085"
    titleColor: "#344054"
    edgeLabelBackground: "#f2f4f7"
    lineColor: "#667085"
  flowchart:
    subGraphTitleMargin:
      top: 8
      bottom: 20
---
flowchart BT
    job["Workload"] --> batch["Graph · batch"]
    daemon["Daemon"] --> service["Graph · service"]
    spot["Graph · spot work"] --> group["PolyGraph · processing"]
    batch --> group
    group --> root["PolyGraph · application"]
    service --> root
    classDef execution fill:#e3f3e8,stroke:#247047,color:#163b29
    classDef constraint fill:#fff3d6,stroke:#926000,color:#513900
    class job,daemon,batch,service,group,root execution
    class spot constraint

Replica connections

Each ReplicaGroup can choose its own connection mode. Here, one subgraph connects whole graph replicas in a Ring; another combines daemon replicas in a Star with bidirectional connections and a FullMesh. Edges between enclosing graphs and groups are declared separately at their network boundaries.

---
config:
  theme: base
  htmlLabels: false
  themeVariables:
    primaryTextColor: "#163b29"
    secondaryTextColor: "#513900"
    tertiaryTextColor: "#344054"
    clusterBkg: "#f2f4f7"
    clusterBorder: "#667085"
    titleColor: "#344054"
    edgeLabelBackground: "#f2f4f7"
    lineColor: "#667085"
  flowchart:
    subGraphTitleMargin:
      top: 8
      bottom: 20
---
flowchart TB
    subgraph application["PolyGraph · application"]
        direction TB
        subgraph processing["Graph · processing"]
            subgraph pipelines["pipelines · Ring"]
                direction LR
                p0["replica-0<br/>Graph instance"]
                p1["replica-1<br/>Graph instance"]
                p2["replica-2<br/>Graph instance"]
                p0 --> p1 --> p2 --> p0
            end
        end

        subgraph serving["Graph · serving"]
            direction TB
            subgraph routers["routers · Star"]
                direction LR
                s0["replica-0<br/>Daemon · hub"]
                s1["replica-1<br/>Daemon"]
                s2["replica-2<br/>Daemon"]
                s3["replica-3<br/>Daemon"]
                s0 <--> s1
                s0 <--> s2
                s0 <--> s3
            end

            subgraph caches["caches · FullMesh"]
                direction LR
                m0["replica-0<br/>Daemon"]
                m1["replica-1<br/>Daemon"]
                m2["replica-2<br/>Daemon"]
                m0 <--> m1
                m1 <--> m2
                m2 <--> m0
            end

            routers -->|"TCP 6379"| caches
        end

        processing -->|"TCP 9000"| serving
    end

    classDef execution fill:#e3f3e8,stroke:#247047,color:#163b29
    classDef boundary fill:#f2f4f7,stroke:#667085,color:#344054
    classDef replicas fill:#ffffff,stroke:#667085,stroke-width:2px,color:#344054
    class p0,p1,p2,s0,s1,s2,s3,m0,m1,m2 execution
    class processing,serving boundary
    class pipelines,routers,caches replicas
    style application fill:#ffffff,stroke:#667085,stroke-width:2px,color:#344054
    linkStyle default stroke:#475467,stroke-width:2px

Autoscaling the hierarchy

KEDA can autoscale different levels of the hierarchy by targeting a ReplicaGroup's Kubernetes /scale subresource. The group's template determines what each additional replica creates:

• Daemon replicas: another service instance, backed by its selected Deployment or StatefulSet. Set the Daemon's own replicas: 1 when each group copy should represent one desired Pod.
• Graph replicas: another complete workflow or service graph, including its workloads, resources and internal connections.
• PolyGraph or nested ReplicaGroup replicas: another composition of graphs or replica groups, allowing scaling at multiple levels of the same application.

For example, scale a worker pool inside a processing graph as its queue grows, and scale copies of the whole processing graph as demand for complete pipelines grows. Scale one group instance independently, or scale a shared definition to update every inheriting instance; see instance and shared scaling.

Before creating or retiring copies, Polyad refreshes the owning graph family's topology and checks replica bounds and applicable GraphRules, including structural limits and Cheeger constraints. KEDA supplies the requested count; constraints can block its application. Target the ReplicaGroup to use these checks: directly autoscaling a generated Deployment or StatefulSet bypasses graph admission. See constraints before scaling.

When percentage routing is configured, a positive traffic assignment also blocks removing its destination. Drain its share to zero before scale-in; newly added copies need explicit routing assignments. See traffic balancing and scaling.

A ReplicaGroup of PolyGraphs can also scale a complete cross-cluster composition. Each destination can independently scale its own local groups. The multicluster scaling diagram shows where each cluster refreshes live values and enforces its own rules.

Constrained compositions

Build workflows from reusable definitions and trace each instance to its Kubernetes resources. GraphRule lets engineers constrain what users can schedule by size, shape, nesting and mathematical properties.^[5][6]

---
config:
  theme: base
  htmlLabels: false
  themeVariables:
    primaryTextColor: "#163b29"
    secondaryTextColor: "#513900"
    tertiaryTextColor: "#344054"
    clusterBkg: "#f2f4f7"
    clusterBorder: "#667085"
    titleColor: "#344054"
    edgeLabelBackground: "#f2f4f7"
    lineColor: "#667085"
  flowchart:
    subGraphTitleMargin:
      top: 8
      bottom: 20
---
flowchart LR
    classDef execution fill:#e3f3e8,stroke:#247047,color:#163b29
    classDef constraint fill:#fff3d6,stroke:#926000,color:#513900
    classDef resource fill:#eeeeee,stroke:#777777,color:#444444

    request["Composition request<br/>IDs and references"]:::resource
    rules["GraphRule<br/>size, shape, spectrum"]:::constraint
    root["PolyGraph root<br/>aggregate status"]:::execution
    request --> root
    rules -. "constrains each boundary" .-> root
    subgraph left["Graph instance: left"]
        a["Workload instance"]:::execution
    end
    subgraph right["Graph instance: right"]
        b["Workload instance"]:::execution
    end
    root --> a
    root --> b
    definition["Shared graph definition"]:::resource
    definition -. "instantiates" .-> a
    definition -. "instantiates" .-> b

Network boundaries

Group workloads into subgraphs with explicit network connections. Scoped rules control traffic across boundaries and namespaces; optional Istio integration adds HTTP and service-identity authorization.^[7][8]

Optional percentage routing controls how incoming requests are divided among connected replicas, while network policies control which traffic is permitted. Polyad generates Istio VirtualServices and DestinationRules for local sidecar HTTP, HTTP/2 and gRPC Services. A Graph or PolyGraph copy receives its share through its entrypoint workloads. Try the traffic-balancing example, or compare fixed percentages with the Tiers and Headroom feedback modes.

---
config:
  theme: base
  htmlLabels: false
  themeVariables:
    primaryTextColor: "#163b29"
    secondaryTextColor: "#513900"
    tertiaryTextColor: "#344054"
    clusterBkg: "#f2f4f7"
    clusterBorder: "#667085"
    titleColor: "#344054"
    edgeLabelBackground: "#f2f4f7"
    lineColor: "#667085"
  flowchart:
    subGraphTitleMargin:
      top: 8
      bottom: 20
---
flowchart TB
    rules["GraphRule · subtree scope"] -. inherits .-> group
    subgraph group["PolyGraph · application"]
        direction TB
        subgraph producers["Graph · producers"]
            task["Workload"] --> sender["Daemon"]
        end
        subgraph consumers["Graph · consumers"]
            receiver["Daemon"] --> report["Workload"]
        end
        producers -->|"TCP 8080"| consumers
    end
    external["Service identity · another namespace"] -->|"GET /status"| consumers
    classDef execution fill:#e3f3e8,stroke:#247047,color:#163b29
    classDef constraint fill:#fff3d6,stroke:#926000,color:#513900
    classDef boundary fill:#f2f4f7,stroke:#667085,color:#344054
    class task,sender,receiver,report execution
    class rules,external constraint
    class group,producers,consumers boundary

Graphs across clusters

Graphs execute within one cluster. PolyGraphs can optionally place child Graphs and nested PolyGraphs in registered remote clusters, composing regions and higher levels. Optional shared observers expose read-only graph snapshots while each cluster's operator enforces its own rules and executes its workloads. See cross-cluster placement, Istio transport and observers.

A root operator control plane can manage this entire hierarchy from a separate management cluster. It installs remote execution replicas, collects their observations centrally and coordinates KEDA through root-local scale targets. See the deployment architecture and complete configuration example.

---
config:
  theme: base
  htmlLabels: false
  themeVariables:
    primaryTextColor: "#163b29"
    secondaryTextColor: "#513900"
    tertiaryTextColor: "#344054"
    clusterBkg: "#f2f4f7"
    clusterBorder: "#667085"
    titleColor: "#344054"
    edgeLabelBackground: "#f2f4f7"
    lineColor: "#667085"
  flowchart:
    subGraphTitleMargin:
      top: 8
      bottom: 20
---
flowchart TB
    subgraph east["Cluster east"]
        root["PolyGraph<br/>Global application"]
        region["PolyGraph<br/>East region"]
        batch["Graph · batch<br/>Local Jobs"]
        root --> region --> batch
    end
    subgraph west["Cluster west"]
        group["PolyGraph<br/>Western regions"]
        service["Graph · service<br/>Local Deployments or StatefulSets"]
        group --> service
    end
    subgraph north["Cluster north"]
        analytics["Graph · analytics<br/>Local Jobs and resources"]
    end
    root -->|"cluster: west"| group
    group -->|"cluster: north"| analytics
    classDef execution fill:#e3f3e8,stroke:#247047,color:#163b29
    class root,region,batch,group,service,analytics execution

Graphs across node groups

Place whole graphs on groups of Kubernetes machines, such as general compute or accelerators. Here, three graphs share two worker groups while coordinated operator replicas and their shared Dragonfly cache run on a third.^[9][10]

---
config:
  theme: base
  htmlLabels: false
  themeVariables:
    primaryTextColor: "#163b29"
    secondaryTextColor: "#513900"
    tertiaryTextColor: "#344054"
    clusterBkg: "#f2f4f7"
    clusterBorder: "#667085"
    titleColor: "#344054"
    edgeLabelBackground: "#f2f4f7"
    lineColor: "#667085"
  flowchart:
    subGraphTitleMargin:
      top: 8
      bottom: 20
---
flowchart TB
    subgraph control["Node group · operators"]
        operator["Polyad operator replicas"]
        cache[("Shared Redis / Dragonfly cache")]
    end
    api["Kubernetes API"]
    operator <-->|"queues and coordination"| cache
    operator -->|"ordered resource writes"| api
    subgraph groupA["Node group · general compute"]
        subgraph graphA["Graph · ingest"]
            fetch["Fetch"] --> normalize["Normalize"]
        end
        subgraph graphB["Graph · publish"]
            package["Package"] --> publish["Publish"]
        end
    end
    subgraph groupB["Node group · accelerated compute"]
        subgraph graphC["Graph · processing"]
            compute["Compute"] --> aggregate["Aggregate"]
        end
    end
    api -->|"graph placement"| graphA
    api -->|"graph placement"| graphB
    api -->|"graph placement"| graphC
    classDef execution fill:#e3f3e8,stroke:#247047,color:#163b29
    classDef constraint fill:#fff3d6,stroke:#926000,color:#513900
    classDef resource fill:#eeeeee,stroke:#777777,color:#444444
    class fetch,normalize,package,publish,compute,aggregate execution
    class operator constraint
    class api,cache resource
    style control fill:#fff3d6,stroke:#926000,color:#513900
    style groupA fill:#eeeeee,stroke:#777777,color:#444444
    style groupB fill:#eeeeee,stroke:#777777,color:#444444

Workloads calling the operator

Running workloads can submit their next graph, read its status and subscribe to graph events through the operator's optional APIs. Services route requests to ready replicas; explicitly authorized callers can connect from other namespaces.^[15]

A running service can also pulse downstream workloads or daemon replica groups, with explicit concurrency and frequency policies.^[17]

With a capacity policy, Polyad forecasts upcoming stages while earlier work runs, giving a compatible node autoscaler advance notice. Dependencies and gates still decide when the next stage starts.^[16]

---
config:
  theme: base
  htmlLabels: false
  themeVariables:
    primaryTextColor: "#163b29"
    secondaryTextColor: "#513900"
    tertiaryTextColor: "#344054"
    clusterBkg: "#f2f4f7"
    clusterBorder: "#667085"
    titleColor: "#344054"
    edgeLabelBackground: "#f2f4f7"
    lineColor: "#667085"
  flowchart:
    subGraphTitleMargin:
      top: 8
      bottom: 20
---
flowchart TB
    subgraph local["Workloads · operator namespace"]
        caller["Managed workload"]
    end
    subgraph remote["Workloads · another namespace"]
        subscriber["Authorized workload"]
    end
    access["Explicit network access<br/>Optional Istio identity authorization"]
    composition["Composition Service · 8090<br/>Submit graphs and read status"]
    events["Event Service · 8091<br/>Subscribe to graph observations"]
    subgraph control["Node group · operators"]
        operator["Ready operator replicas<br/>APIs and graph scheduling"]
        cache[("Shared Dragonfly cache<br/>Queues and event history")]
    end
    demand["Kubernetes API<br/>ProvisioningRequest or placeholder Pods"]
    autoscaler["Node autoscaler"]
    machines["Target worker node group<br/>Capacity for upcoming stages"]
    caller -->|"Authenticated requests"| access
    subscriber -->|"Cross-namespace requests"| access
    access --> composition
    access -->|"GET /v1/events"| events
    composition --> operator
    events --> operator
    operator <-->|"Coordination and observations"| cache
    events -. "SSE observations" .-> local
    events -. "SSE observations" .-> remote
    operator -->|"Forecast before next stage"| demand
    demand -->|"Upcoming resource demand"| autoscaler
    autoscaler -->|"Provision nodes when supported"| machines
    classDef execution fill:#e3f3e8,stroke:#247047,color:#163b29
    classDef constraint fill:#ffe3a3,stroke:#926000,color:#513900
    classDef resource fill:#eeeeee,stroke:#777777,color:#444444
    class caller,subscriber execution
    class operator,access constraint
    class composition,events,cache,demand,autoscaler,machines resource
    style local fill:#ffffff,stroke:#667085,stroke-width:2px,color:#344054
    style remote fill:#e2e6ec,stroke:#667085,stroke-width:2px,color:#344054
    style control fill:#fff3d6,stroke:#926000,stroke-width:2px,color:#513900
    linkStyle default stroke:#475467,stroke-width:2px

Finite pipelines

Express a workflow from preparation to publication, with parallel tasks and gates that wait for a condition or delay. Ordinary Graph placement can select spot capacity; applications choose how to handle interruption and storage.^[11][12]

---
config:
  theme: base
  htmlLabels: false
  themeVariables:
    primaryTextColor: "#163b29"
    secondaryTextColor: "#513900"
    tertiaryTextColor: "#344054"
    clusterBkg: "#f2f4f7"
    clusterBorder: "#667085"
    titleColor: "#344054"
    edgeLabelBackground: "#f2f4f7"
    lineColor: "#667085"
  flowchart:
    subGraphTitleMargin:
      top: 8
      bottom: 20
---
flowchart TB
    subgraph pipeline["Graph · finite pipeline"]
        direction TB
        storage[("Resource · shared storage")]
        prepare["Workload · prepare"]
        gate{"Gate · admission condition"}
        subgraph workers["Graph · spot placement"]
            direction TB
            left["Workload · partition A"]
            right["Workload · partition B"]
        end
        subgraph publish["Graph · publish results"]
            merge["Workload · merge"]
            report["Workload · report"]
            merge -->|completed| report
        end
        storage -->|ready| prepare
        prepare -->|completed| gate
        gate -->|allowed| left
        gate -->|allowed| right
        workers -->|completed| publish
    end
    classDef execution fill:#e3f3e8,stroke:#247047,color:#163b29
    classDef constraint fill:#ffe3a3,stroke:#926000,color:#513900
    classDef resource fill:#eeeeee,stroke:#777777,color:#444444
    class prepare,left,right,merge,report execution
    class gate constraint
    class storage resource
    style pipeline fill:#e2e6ec,stroke:#667085,stroke-width:2px,color:#344054
    style workers fill:#fff3d6,stroke:#926000,stroke-width:2px,color:#513900
    style publish fill:#ffffff,stroke:#667085,stroke-width:2px,color:#344054
    linkStyle default stroke:#475467,stroke-width:2px

Persistent services and recurrence

Keep services running with Daemon, and repeat finite Graphs with activation requests. A producer or timer supplies each pulse; the application owns iteration limits, stop conditions and durable shared state.^[13]

---
config:
  theme: base
  htmlLabels: false
  themeVariables:
    primaryTextColor: "#163b29"
    secondaryTextColor: "#513900"
    tertiaryTextColor: "#344054"
    clusterBkg: "#f2f4f7"
    clusterBorder: "#667085"
    titleColor: "#344054"
    edgeLabelBackground: "#f2f4f7"
    lineColor: "#667085"
  flowchart:
    subGraphTitleMargin:
      top: 8
      bottom: 20
---
flowchart TB
    subgraph system["Graph · persistent service"]
        direction TB
        queue[("Resource · queue service")]
        subgraph service["Graph · persistent processing"]
            direction LR
            ingest["Daemon · ingest"]
            process["Daemon · process"]
            ingest -->|ready| process
            ingest <-.->|"events / acknowledgements"| process
        end
        subgraph activations["Graph · finite activation target"]
            direction LR
            sample["Workload · sample"]
            adjust["Workload · adjust"]
            sample -->|completed| adjust
        end
        next["Producer / timer pulse"]
        activations -->|"completion observed"| next
        next -.->|"activation request"| activations
        queue -->|ready| service
        service -->|ready| activations
    end
    classDef execution fill:#e3f3e8,stroke:#247047,color:#163b29
    classDef recurrence fill:#ffe3a3,stroke:#926000,color:#513900
    classDef resource fill:#eeeeee,stroke:#777777,color:#444444
    class ingest,process,sample,adjust execution
    class next recurrence
    class queue resource
    style system fill:#e2e6ec,stroke:#667085,stroke-width:2px,color:#344054
    style service fill:#ffffff,stroke:#667085,stroke-width:2px,color:#344054
    style activations fill:#fff3d6,stroke:#926000,stroke-width:2px,color:#513900
    linkStyle default stroke:#475467,stroke-width:2px

The operator can also request capacity ahead of upcoming stages, helping node autoscalers prepare machines while upstream work runs.

Queue pressure and graph hierarchies are available through the optional Prometheus and JSON metrics API.

Optional OpenTelemetry tracing exports API request, reconciliation and Kubernetes operation spans to an OTLP/HTTP collector, with configurable sampling and Secret-backed exporter credentials. Decision logs explain admissions, scaling, topology membership and conflicts, with trace correlation and independently enabled OTLP log export. KEDA can scale services or whole graph compositions through ReplicaGroup, using workload metrics served by the operator.

Argo CD and Flux health checks report graph and leaf health across nested applications.

The operator as a Graph

Polyad can run as a compact HA Deployment or manage its own service components in a Graph. With architecture.mode: Distributed, gateway, executor and telemetry ReplicaGroups scale independently through KEDA and fresh GraphRule checks. A root bootstrap Deployment retains planning and recovery responsibility. With root mode enabled, one reserved PolyGraph contains a Graph for each operator group. The root operator Graph contains the bootstrap, managed component pipeline, KEDA and all enabled local chart services; remote operator groups join as peers.

flowchart TB
    subgraph operators["Reserved root PolyGraph · all operator groups"]
        subgraph rootGroup["Graph · root operator group"]
            subgraph bootstrap["Graph · bootstrap observation"]
                root["Helm-owned root Deployment<br/>planning and recovery"]
            end
            subgraph self["Graph · local service components"]
                direction LR
                gateway["Gateway replicas<br/>APIs and event subscriptions"]
                executor["Executor replicas<br/>graph admission and workloads"]
                telemetry["Telemetry replicas<br/>observations and metrics"]
                gateway --> executor --> telemetry
            end
            bootstrap -->|"reconcile and restore components"| self
            self -->|"observations"| bootstrap
            keda["KEDA Graph<br/>Operator, metrics server and webhooks"]
            cache["Dragonfly Graph<br/>Controller, cache and Services"]
            pg["PostgreSQL Graph · optional<br/>State and authentication databases"]
            support["Other local service Graphs<br/>Endpoints, observer and Istio"]
            root <-->|"use and observe"| cache
            root <-->|"observe"| support
        end
        subgraph west["Graph · west workload cluster"]
            workers["DaemonSet · execution workers<br/>One Pod per eligible node"]
        end
        subgraph east["Graph · east workload cluster"]
            pool["Deployment · execution workers<br/>KEDA replica scaling"]
        end
        rootGroup -->|"root coordination"| west
        rootGroup -->|"root coordination"| east
        workers -->|"observations"| root
        pool -->|"observations"| root
    end
    rules["GraphRule<br/>Cheeger ≥ 1; recursive size bound"] -. constrains .-> self
    keda -->|"scrape demand"| telemetry
    keda -->|"request replica counts"| root
    root -. "optional state storage" .-> pg
    telemetry -. "persist graph state and parameters" .-> pg

The arrows inside the Graph describe logical stages; actual coordination uses Kubernetes and Dragonfly. Cheeger constrains topology, while queue backlog and HTTP demand drive capacity decisions. See the component and networking diagrams and deployable example.

Adding an OperatorPool in a registered downstream cluster automatically links its Graph into the reserved root PolyGraph. Deployment pools support KEDA replica scaling; DaemonSet capacity follows node eligibility. The root Graph's bootstrap branch observes its existing Deployment, preserving Helm ownership and recovery. The nested component Graph retains its own Cheeger and replica-budget checks. Application event streams exclude this operator tree.

The local service inventory observes the chart's enabled infrastructure while Helm and upstream operators retain lifecycle ownership. Set keda.install: true to install the optional pinned KEDA dependency, or configure references to existing KEDA with the typed KEDA values.

Administrators can also install downstream workers with Helm and attach their existing Deployments. Helm retains installation and upgrades; each attachment explicitly chooses root/KEDA or downstream replica scaling.

PostgreSQL is optional, disabled by default, and stores graph observations and tracked parameters when enabled. Its optional KEDA configuration scales CloudNativePG instances from operator connection counts, still scraped from the operator. Additional database instances provide standby/read capacity; writes continue through the primary.

What Polyad is not

Polyad coordinates application graphs alongside existing cluster components.

• A general-purpose policy engine such as OPA. GraphRule constrains the graphs Polyad admits and the resources it compiles. It does not evaluate Rego, replace application authorization, or enforce policy on every Kubernetes API request. Cluster-wide admission policy remains a separate concern.^[18][19]
• A replacement for the Kubernetes scheduler or node autoscaler. Polyad controls when graph work is admitted and propagates placement constraints. Kubernetes places Pods; the configured autoscaler provisions machines. Grouping work does not guarantee that every Pod starts together.^[9][20]
• A service mesh or network transport. Polyad generates network and Istio authorization and routing resources, including optional percentage splits. The cluster's networking implementation and mesh enforce them; drawing a graph connection does not transport application data.^[21]
• Automatic process checkpointing or exactly-once execution. Restarting containers with persistent storage requires application recovery logic. Workloads must handle retries and duplicate effects; graph ownership and ordered API writes do not make application operations transactional.^[22][10]

License

GNU General Public License v3.0 only.

References

Selected background reading for Polyad's architecture and less common design choices.

• Infrastructure at scale: Eduardo Barth, Medium Engineering, Kubernetes Infrastructure At Medium (February 14, 2023). A case study covering separate clusters, gradual infrastructure rollouts, resource sizing and spare capacity for traffic bursts.
• Structural bottlenecks: Shlomo Hoory, Nathan Linial and Avi Wigderson, Expander Graphs and Their Applications (2006, Section 2.1). The edge-expansion definition used by Polyad's hard Cheeger bounds and throughput-driven Cheeger targets. Application-reported demand selects a calibrated target; sustained completion shortfalls can trigger connection changes within the hard bounds. Optional traffic balancing also uses calibrated splits or per-replica throughput and headroom to redistribute requests. Those routing weights are separate from the unweighted Cheeger value used by both bounds.
• Rewriting and composition: Dimitri Ara et al., Polygraphs: From Rewriting to Higher Categories. Background for the rewriting, confluence and higher-dimensional diagrams in the mutation documentation.
• Advance capacity: The Cluster Autoscaler ProvisioningRequest FAQ. Reserving capacity before a workload stage becomes runnable; see Polyad's advance-capacity design.
• Pending delivery recovery: Dragonfly's XAUTOCLAIM. Recovering unacknowledged stream entries when a queue consumer loses ownership.
• GitOps health: Argo CD custom health checks and Flux health expressions. Reporting graph and descendant health to deployment controllers.