kubernify 1.0.7

Verify Kubernetes deployments match a version manifest with deep stability auditing. Checks convergence, revision consistency, and pod health.

kubernify

Verify Kubernetes deployments match a version manifest with deep stability auditing. Checks convergence, revision consistency, and pod health.

---

Features

• Manifest-driven verification - Provide a JSON manifest of expected versions; kubernify verifies the cluster matches
• Deep stability auditing - Goes beyond version checks: convergence, revision consistency, pod health, DaemonSet scheduling, Job completion
• Retry-until-converged loop - Waits for rollouts to complete rather than just snapshot-checking
• Repository-relative image parsing - Flexible component name extraction from any image registry format
• Comprehensive workload support - Deployments, StatefulSets, DaemonSets, Jobs, and CronJobs
• Zero-replica awareness - Verifies version from PodSpec even when HPA/KEDA has scaled to zero
• Structured JSON reports - Machine-readable output for CI/CD pipeline integration

---

Installation

pip install kubernify

Or with pipx for isolated CLI usage:

pipx install kubernify

Or with uv:

uv add kubernify

---

Quick Start

# Verify backend and frontend match expected versions in the "production" namespace
kubernify \
  --context my-cluster-context \
  --anchor my-app \
  --namespace production \
  --manifest '{"backend": "v1.2.3", "frontend": "v1.2.4"}'

kubernify will connect to the cluster, discover all matching workloads, verify their image versions against the manifest, run stability audits, and exit with code 0 (pass), 1 (fail), or 2 (timeout).

---

CLI Reference

kubernify [OPTIONS]

Argument	Description	Default
--context	Kubeconfig context name to use for cluster connection. Mutually exclusive with --gke-project. When omitted, the active kubeconfig context is used automatically.	From kubeconfig
--gke-project	GCP project ID — resolves the kube context from GKE-style context names (e.g., gke_my-project_us-central1_cluster-name). Mutually exclusive with --context.
--anchor	(required) The image path segment used as the anchor point for component name extraction. For example, given image registry.example.com/my-org/my-app/backend:v1.0, using --anchor my-app extracts the component name backend. See How Image Anchor Works.
--manifest	(required) JSON string containing the version manifest mapping component names to their expected versions, e.g. '{"backend": "v1.2.3", "frontend": "v2.0.0"}'.
--component-aliases	JSON string mapping manifest component names to their actual image names when they differ. Example: '{"foo": "bar-baz"}' means the manifest key foo corresponds to the container image named bar-baz. Multiple manifest keys can alias to the same image name — disambiguation is performed by matching the manifest key against the Kubernetes workload name (substring match). See Component Aliases.
--namespace	Kubernetes namespace to verify. Resolved automatically from kubeconfig context, in-cluster service account, or falls back to default.	From kubeconfig context
--required-workloads	Comma-separated substring patterns for workloads that must exist in the namespace, independent of the manifest. Useful for ensuring critical workloads (e.g., infrastructure sidecars, operators) are present even if they aren't version-verified. Each pattern is matched against discovered workload names using substring containment (e.g., frontend matches my-app-frontend). Verification fails if any pattern has no match.
--skip-containers	Comma-separated substring patterns to skip during verification. Each pattern is matched against both container names and workload names using substring containment (e.g., backend matches my-app-backend). Skipped workloads are excluded from both version verification and stability audits.
--min-uptime	Minimum pod uptime in seconds for stability checks. Pods running for less than this duration are flagged as unstable.	0
--restart-threshold	Maximum acceptable container restart count. Containers exceeding this threshold are flagged as unstable. Use 0 to forbid any restarts, or -1 to skip the restart check entirely.	3
--timeout	Global timeout in seconds for the verification loop. The tool retries discovery and verification until all checks pass or this timeout is reached. Returns exit code 2 on timeout.	300
--allow-zero-replicas	Allow all workloads with zero running replicas to pass verification (version is still checked via the pod spec template). Mutually exclusive with --allow-zero-replicas-for.	false
--allow-zero-replicas-for	Comma-separated list of workload name patterns allowed to have 0 running replicas (e.g., my-cronjob-worker,batch-processor). Uses substring matching: my-worker matches ns-123-my-worker. Mutually exclusive with --allow-zero-replicas.
--dry-run	Perform a single snapshot check against the current cluster state without waiting for convergence. Exits immediately with pass/fail result.	false
--include-statefulsets	Include StatefulSets in workload discovery. By default, only Deployments are inspected.	false
--include-daemonsets	Include DaemonSets in workload discovery. By default, only Deployments are inspected.	false
--include-jobs	Include Jobs and CronJobs in workload discovery. By default, only Deployments are inspected.	false
--ignore-tombstone-pods	When set, pods in phase Failed or Succeeded (OOMKilled, Evicted, Completed scripts) are excluded from per-pod health checks. These "gray" pods do not cause health check failures. The deployment availability check (available_replicas >= spec.replicas) always runs regardless of this flag.	false

---

Usage Examples

Basic Usage - Direct Kubeconfig Context

kubernify \
  --context my-cluster-context \
  --anchor my-app \
  --namespace production \
  --manifest '{"backend": "v1.2.3", "frontend": "v1.2.4"}'

GKE Shorthand - Resolve Context from GCP Project

kubernify \
  --gke-project my-gke-project-123456 \
  --anchor my-app \
  --namespace production \
  --manifest '{"backend": "v1.2.3", "frontend": "v1.2.4"}'

In-Cluster - Running Inside a Kubernetes Pod

# No --context needed; auto-detects in-cluster config and namespace
kubernify \
  --anchor my-app \
  --manifest '{"backend": "v1.2.3", "frontend": "v1.2.4"}'

Full-Featured - All Options

kubernify \
  --context my-cluster-context \
  --anchor my-app \
  --namespace production \
  --manifest '{"backend": "v1.2.3", "frontend": "v1.2.4", "worker": "v1.2.3"}' \
  --required-workloads "backend, frontend, worker" \
  --skip-containers "istio-proxy, envoy, fluent-bit" \
  --include-statefulsets \
  --include-daemonsets \
  --include-jobs \
  --min-uptime 120 \
  --restart-threshold 5 \
  --ignore-tombstone-pods \
  --timeout 600 \
  --allow-zero-replicas
  # OR selectively:
  # --allow-zero-replicas-for "worker, cron-handler"

Dry Run - Snapshot Check Without Waiting

kubernify \
  --context my-cluster-context \
  --anchor my-app \
  --manifest '{"backend": "v1.2.3"}' \
  --dry-run

CI/CD Integration - GitHub Actions

jobs:
  verify-deployment:
    runs-on: ubuntu-latest
    steps:
      - name: Set up kubeconfig
        run: |
          echo "${{ secrets.KUBECONFIG }}" > /tmp/kubeconfig
          export KUBECONFIG=/tmp/kubeconfig

      - name: Install kubernify
        run: pip install kubernify

      - name: Verify deployment
        run: |
          kubernify \
            --context ${{ secrets.KUBE_CONTEXT }} \
            --anchor my-app \
            --manifest '${{ steps.build.outputs.manifest }}' \
            --timeout 600 \
            --min-uptime 60

---

Programmatic Usage

kubernify can be used as a Python library for custom verification workflows:

from kubernify import __version__, VerificationStatus
from kubernify.kubernetes_controller import KubernetesController
from kubernify.workload_discovery import WorkloadDiscovery
from kubernify.cli import construct_component_map, verify_versions

controller = KubernetesController(context="my-cluster")
discovery = WorkloadDiscovery(k8s_controller=controller)

workloads, _ = discovery.discover_cluster_state(namespace="production")
component_map = construct_component_map(
    workloads=workloads,
    manifest={"backend": "v1.2.3"},
    repository_anchor="my-app",
)
results = verify_versions(manifest={"backend": "v1.2.3"}, component_map=component_map)

if results.errors:
    print(f"Verification failed: {results.errors}")

---

How Image Anchor Works

kubernify uses a repository-relative anchor to extract component names from container image paths. The --anchor argument specifies the path segment after which the component name is derived.

Image: registry.example.com/my-org-foo/my-app-bar/backend:v1.2.3-x
       └──── registry ─────┘ └─ org ─┘ └ anchor ┘└ comp.┘└─ tag ─┘

More examples:

Image	--anchor	Extracted Component
registry.example.com/my-org/my-app/backend:v1.2.3	my-app	backend
registry.example.com/my-org/my-app/api/server:v2.0.0	my-app	api/server
gcr.io/my-project/my-app/worker:v1.0.0	my-app	worker

The extracted component name is then matched against the keys in your --manifest JSON to verify the correct version is deployed.

---

Component Aliases

Use --component-aliases when a manifest component name differs from the container image name extracted by the anchor.

Basic Alias (One-to-One)

If your manifest uses the key foo but the container image is named bar-baz:

kubernify \
  --anchor my-app \
  --manifest '{"foo": "v1.0.0", "backend": "v2.0.0"}' \
  --component-aliases '{"foo": "bar-baz"}'

This tells kubernify: when you see image bar-baz, map it to the manifest key foo.

Shared Image Alias (Many-to-One)

Multiple manifest components can share the same container image name. kubernify disambiguates by matching each manifest key against the Kubernetes workload name (substring match).

For example, if both ingest and process use the same shared-svc image but run as separate workloads:

kubernify \
  --anchor my-app \
  --manifest '{"ingest": "v1.0.0", "process": "v1.0.0"}' \
  --component-aliases '{"ingest": "shared-svc", "process": "shared-svc"}' \
  --include-statefulsets

Given these workloads in the cluster:

• Deployment my-app-123-ingest → image shared-svc:v1.0.0 → mapped to manifest key ingest (because "ingest" is a substring of "my-app-123-ingest")
• StatefulSet my-app-123-process-node → image shared-svc:v1.0.0 → mapped to manifest key process (because "process" is a substring of "my-app-123-process-node")

Resolution priority when multiple candidates exist for the same image:

1. If only one candidate → use it directly
2. If multiple candidates → pick the one whose manifest key is a substring of the workload name
3. If no candidate matches the workload name → fall back to the raw image component name (if it's in the manifest)
4. If nothing matches → the workload is skipped (not mapped to any manifest key)

---

Exit Codes

Code	Meaning	Description
0	PASS	All workloads match the manifest and pass stability audits
1	FAIL	One or more workloads have version mismatches or stability issues
2	TIMEOUT	Verification did not converge within the --timeout window

---

Prerequisites

Python

• Python >= 3.10

For GKE Users

If using --gke-project for automatic GKE context resolution:

1. Install the Google Cloud SDK
2. Install the GKE auth plugin:

   gcloud components install gke-gcloud-auth-plugin
   

3. Authenticate:

   gcloud auth login
   gcloud container clusters get-credentials CLUSTER_NAME --project PROJECT_ID
   

RBAC Permissions

kubernify requires read-only access to workloads and pods. Apply the following RBAC configuration:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: kubernify-reader
  namespace: <namespace>
rules:
  - apiGroups: ["apps"]
    resources: ["deployments", "statefulsets", "daemonsets", "replicasets"]
    verbs: ["get", "list"]
  - apiGroups: ["batch"]
    resources: ["jobs", "cronjobs"]
    verbs: ["get", "list"]
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: kubernify-reader-binding
  namespace: <namespace>
subjects:
  - kind: ServiceAccount
    name: kubernify
    namespace: <namespace>
roleRef:
  kind: Role
  name: kubernify-reader
  apiGroup: rbac.authorization.k8s.io

---

Contributing

Contributions are welcome! Please see CONTRIBUTING.md for development setup, coding standards, and the PR process.

---

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.