Pyplines Builder

pyplines-builder is the Python/Typer authoring tool. pyplines remains the operational CLI: it installs and operates the resulting Distribution. Neither tool depends on the other. The obsolete Go/Buildah Action Builder is not used.

Install with pip install pyplines-builder. Development: uv sync --project apps/builder. Docker must provide Linux containers. Builds and execution use the Docker API; registry login/logout delegate to the Docker CLI for credential persistence. Source inspection runs in the target image using the SDK, not in the host interpreter.

Commands

Distribution and Action inspection share a machine contract and human presentation. Use --details for expanded output and --json for complete metadata; Distribution inspection accepts --trust-file for publisher verification. See inspection and metadata.

pyplines-builder version
pyplines-builder check ./hello-world
pyplines-builder keygen ./signing --source https://example.com --publisher example
pyplines-builder build ./hello-world --registry registry.example.com/example \
  --signing-key ./signing/publisher.pem --output ./hello-world-1.0.0.tar.gz
pyplines-builder inspect ./hello-world-1.0.0.tar.gz
pyplines-builder verify ./hello-world-1.0.0.tar.gz --trust-file ./signing/trust.json
pyplines-builder action build ./action --platform linux/arm64
pyplines-builder action inspect sha256:IMAGE_ID
pyplines-builder action run sha256:IMAGE_ID --input-file ./input.yaml

Every command supports --json. Data goes to stdout, failures to stderr; failures have nonzero exit codes. No prompts, animations, or credential values in diagnostics. Local image IDs are reported by action build. action run reports runtime logs and the terminal InvocationResult; it does not replace testing against the Server. --allow-network explicitly enables test egress.

Directory authoring

See Hello World. The root procedure.yaml uses kind: https://v1alpha1.pyplines.dev/schemas/Procedure.json, family, version, optional name/description, named inputs, actions, steps, and outputs. There is no build manifest and authors do not coordinate UUIDs. Source and published documents both use the authoritative Procedure.json contract; there is no separate authoring schema.

Action aliases reference ./actions/greet or oci://registry.example.com/actions/greet:1.0.0 (or @sha256:...). Bare names are not searched. Local paths resolve relative to the declaring Procedure and must remain within the source directory. Supporting Procedure aliases are declared under procedures, pointing to local files. A type: pypline step uses spec.procedure: '${{ procedures.alias }}'; installation resolves that included Procedure into a child Pypline. It is not a reference to an arbitrary existing deployment. Only reachable resources may enter the signed archive.

Steps use type, optional retry, and spec. Action steps use size (xs, sm, md, lg, xl; default sm), resolved through Workspace execution settings, and spec.action/spec.inputs. Delay, approval and switch specs contain their canonical durationSeconds, message, and expression/cases/default fields respectively. Steps execute sequentially in declaration order; needs is not supported. References must name earlier steps. Source output expressions use steps.NAME.output.FIELD. Published documents retain this syntax. Internal execution records are private implementation details. Published documents include order to preserve execution order across canonical JSON key sorting.

Input field definitions are JSON Schema fragments with a top-level optional boolean required (default true unless a default value exists). Nested object properties use standard JSON Schema, including nested required arrays. Output fields additionally require value, compiled into output bindings. Arbitrary Python is never executed to compile Procedures.

Stable UUIDv5 identities use the fixed builder namespace and the canonical tuple [publication authority, resource kind, family]; version is deliberately excluded. Remote Action identity uses its OCI repository as authority. Changing publisher authority or moving a remote dependency to another repository changes identity.

keygen writes publisher.yaml beside publisher.pem and the public trust file. Existing keys need this three-field configuration (source, publisher, key_id); use --publisher-config to select another file. Keep credentials out of source YAML. Registry access does not establish publisher trust.

--registry is the destination namespace: local Actions publish to NAMESPACE/actions/NAME:VERSION, and the Distribution publishes to NAMESPACE/distributions/FAMILY:VERSION. External Actions are not mirrored. --output optionally saves the identical signed archive. It is not an offline image bundle. The result includes an immutable Distribution reference.

Remote tag resolutions are recorded in pyplines.lock. Commit it for repeatable builds; --refresh resolves tags again and --frozen refuses missing resolutions without rewriting the lock. Lock entries are platform-specific. Local Action builds use Docker caching; image reproducibility still depends on pinned inputs. Identical existing images may be reused; conflicting version tags are rejected. Registry-side immutability is required to prevent concurrent publication races. Failed builds never delete published images. A local-write failure following publication reports the successfully published Distribution reference.

For a local HTTP registry, explicitly set PYPLINES_REGISTRY_ALLOW_HTTP to its host and port; Docker must also permit that registry. HTTPS is the default.

check performs offline structural checks; it does not import Action source or claim to validate runtime schemas. build validates the complete included resource graph. Server installation remains authoritative for semantic planning, authorization, trust and runtime compatibility.

Each Action's pyproject.toml supplies static project.name, version, requires-python, and the pyplines.action handler entry point. Include source files explicitly with [tool.pyplines.builder] include = ["hello_world.py"] (default: src). Include required README/license/package data there too. No symlinks, hidden directories, environments or private-key files are copied. Review this allowlist: an ordinary source file can still contain a hardcoded secret. Building executes author-controlled packaging code and must only be done for trusted source on a suitable Docker engine.

[tool.pyplines.action] owns network_required, retry_safety, and secret_aliases. The builder embeds them in the canonical Action image manifest; remote consumers read them from the image. Defaults are no network, unsafe retry, and no secrets. Resource size belongs to each Procedure Action step, not the Action.

Python 3.11–3.14 is supported; the newest compatible minor is selected, or use --python. --platform auto uses the Docker engine's architecture. Cross-platform builds require engine-provided emulation. A build targets one architecture; produce a separately signed Distribution for each platform. Multi-platform OCI index assembly is not provided. --base-image permits an explicitly pinned Python base. Otherwise the selected Python slim tag is resolved to a digest before building. Base, source, SDK and recipe attribution is embedded at /opt/pyplines/action/build.json and returned in the build report. Docker provides layer caching. Unlocked third-party dependencies can change between builds; pin dependencies for reproducibility. Distribution bytes are deterministic for the same resource documents and signing key; source-to-image reproducibility is not claimed for unpinned dependencies.

Released builders pin the matching published pyplines SDK. A development builder requires an explicit --sdk-wheel; it never silently includes the repository. Keep a private signing key outside Action sources. Keys are Ed25519 PKCS8 PEM, mode 0600 on POSIX. Existing keys and output archives are never overwritten. Existing registry version tags are reused only for identical images. Enable registry-side immutable tags as well to protect against concurrent publishers. The separate trust.json is public and can be supplied to Server trust configuration; the builder does not install trust or weaken verification.

Registry authentication

Existing docker login credentials work for both Action images and Distribution artifacts. The builder uses Docker's configuration and credential helpers, not a separate credential store:

pyplines-builder registry login registry.digitalocean.com
pyplines-builder registry status registry.digitalocean.com
pyplines-builder build ./hello-world \
  --registry registry.digitalocean.com/pyplines-dev \
  --signing-key ./signing/publisher.pem
pyplines-builder registry logout registry.digitalocean.com

Login prompts for username and a hidden password/token. Authentication is checked by Docker before it saves credentials; failed authentication preserves the existing login. Logout removes that exact registry's credentials for Docker as well as the builder. Other registries remain authenticated. One identity is saved per registry hostname/port. There is no active-registry selection or source fallback.

For automation, supply a username and read the password/token from stdin:

pyplines-builder registry login registry.digitalocean.com \
  --username YOUR_USERNAME --password-stdin < /secure/registry-token

AUTOMATION_MODE=enabled and --json produce structured output and never prompt. status checks registry access, not repository push permission. Build checks the destination's connectivity/authentication before expensive Action builds, but actual push remains the final permission check. Source Action references select their own credentials; --registry selects exactly one publishing destination.

DOCKER_CONFIG selects a Docker configuration directory. --registry-credentials or PYPLINES_REGISTRY_CONFIG_FILE explicitly selects a Docker-compatible JSON file for build/inspection (including helper configuration), without fallback to the default file. Unset the latter when managing login/logout; those commands manage the Docker configuration, not an independent override file. Never put credentials in source documents or image references.

The HTTP transport only sends credentials to the registry or a specifically approved HTTPS token service: Docker Hub's auth.docker.io and DigitalOcean's api.digitalocean.com are recognized for their respective registry authorities. Arbitrary cross-host challenges and credential-bearing redirects remain blocked. Docker daemon insecure-registry configuration is needed for a local HTTP registry; the builder does not alter it. Use PYPLINES_REGISTRY_ALLOW_HTTP=HOST:PORT for the builder's corresponding explicit HTTP opt-in.

Local runtime tests use the same entrypoint, Unix socket transport, non-root UID, read-only root, capability restrictions and scratch mounts as the appliance. Test limits are 1 CPU, 256 MiB, 128 PIDs, 16 MiB scratch, 64 KiB test input/secrets, 4 MiB collected logs and a configurable 1–3600 second deadline. Only the test's own containers/volume are removed. Build images/cache remain for reuse.

Development

python3 tooling/contracts/sync_builder_contracts.py --check
uv run --project apps/builder pytest -q apps/builder/tests
uv run --project apps/builder black --check apps/builder/pyplines_builder/app.py
uv build --project apps/builder

The generated readers and schema snapshot come from the same sources as the operational CLI. Edit those sources and regenerate, not the bundled copies.

Shared CLI behavior

The Builder follows the same standard as pyplines: readable output by default, or JSON with AUTOMATION_MODE=enabled / --json. Automation never prompts. Help requires no Docker connection. Use version or --version for the installed package version. Common behavior is provided by pyplines-cli-common.

Publish to an OCI registry

pyplines-builder publish ./maintenance.pyplines.tar.gz \
  --to oci://ghcr.io/acme/maintenance:1.2.0 \
  --trust-file ./publisher-trust.json

Publication verifies signatures before uploading and returns an immutable reference. No Docker daemon or Index is required for publishing a built archive. Publisher authority (source in signing configuration) is a stable HTTPS URI, not a required Index endpoint or registry hostname.

Use Docker-compatible credentials (DOCKER_CONFIG, credential helpers, or --registry-credentials path/to/config.json). PYPLINES_REGISTRY_CONFIG_FILE also selects a config file. Registry credentials are never publisher trust. Use a private file with explicit auths for CI rather than secrets in arguments. The builder refuses conflicting existing tags; enable registry-side immutable tags to protect against concurrent writers. HTTP is only enabled for exact hosts listed in PYPLINES_REGISTRY_ALLOW_HTTP, for isolated local tests.

These archives use the registry-independent v1alpha1 publication statements; rebuild older Index-signed authoring artifacts with the current builder.