Pyplines Builder

pyplines-builder is the Python/Typer authoring tool. pyplines remains the operational CLI: it installs and operates the resulting Distribution. Neither tool depends on the other. The obsolete Go/Buildah Action Builder is not used.

Install with pip install pyplines-builder. Development: uv sync --project apps/builder. Docker must provide Linux containers and OCI archive support through its containerd image store. Builds and execution use the Docker API; registry authentication and image transfer use the shared Pyplines HTTP transport. Source inspection runs in the target image using the SDK, not in the host interpreter.

Commands

Distribution and Action inspection share a machine contract and human presentation. Use --details for expanded output and --json for complete metadata; Distribution inspection accepts --trust-file for publisher verification. See inspection and metadata.

pyplines-builder version
pyplines-builder check ./hello-world
pyplines-builder keygen ./signing --source https://example.com --publisher example
pyplines-builder build ./hello-world --registry registry.example.com/example \
  --signing-key ./signing/publisher.pem --output ./hello-world-1.0.0.tar.gz
pyplines-builder inspect ./hello-world-1.0.0.tar.gz
pyplines-builder verify ./hello-world-1.0.0.tar.gz --trust-file ./signing/trust.json
pyplines-builder action build ./action --platform linux/arm64
pyplines-builder action inspect sha256:IMAGE_ID
pyplines-builder action run sha256:IMAGE_ID --input-file ./input.yaml

Every command supports --json. Data goes to stdout, failures to stderr; failures have nonzero exit codes. No prompts, animations, or credential values in diagnostics. Local image IDs are reported by action build. action run reports runtime logs and the terminal InvocationResult; it does not replace testing against the Server. --allow-network explicitly enables test egress.

Directory authoring

See Hello World. The root procedure.yaml uses kind: https://v1alpha1.pyplines.dev/schemas/Procedure.json, family, version, optional name/description, named inputs, actions, steps, and outputs. There is no build manifest and authors do not coordinate UUIDs. Source and published documents both use the authoritative Procedure.json contract; there is no separate authoring schema.

Action aliases reference ./actions/greet or oci://registry.example.com/actions/greet:1.0.0 (or @sha256:...). Bare names are not searched. Local paths resolve relative to the declaring Procedure and must remain within the source directory. Supporting Procedure aliases are declared under procedures, pointing to local files. A type: pypline step uses spec.procedure: '${{ procedures.alias }}'; installation resolves that included Procedure into a child Pypline. It is not a reference to an arbitrary existing deployment. Only reachable resources may enter the signed archive.

Steps use type, optional retry, and spec. Action steps use size (xs, sm, md, lg, xl; default sm), resolved through Workspace execution settings, and spec.action/spec.inputs. Delay, approval and switch specs contain their canonical durationSeconds, message, and expression/cases/default fields respectively. Steps execute sequentially in declaration order; needs is not supported. References must name earlier steps. Source output expressions use steps.NAME.output.FIELD. Published documents retain this syntax. Internal execution records are private implementation details. Published documents include order to preserve execution order across canonical JSON key sorting.

Input field definitions are JSON Schema fragments with a top-level optional boolean required (default true unless a default value exists). Nested object properties use standard JSON Schema, including nested required arrays. Output fields additionally require value, compiled into output bindings. Arbitrary Python is never executed to compile Procedures.

Stable UUIDv5 identities use the fixed builder namespace and the canonical tuple [publication authority, resource kind, family]; version is deliberately excluded. Remote Action identity uses its OCI repository as authority. Changing publisher authority or moving a remote dependency to another repository changes identity.

keygen writes publisher.yaml beside publisher.pem and the public trust file. Existing keys need this three-field configuration (source, publisher, key_id); use --publisher-config to select another file. Keep credentials out of source YAML. Registry access does not establish publisher trust.

--registry is the destination namespace: local Actions publish to NAMESPACE/actions/NAME:VERSION, and the Distribution publishes to NAMESPACE/distributions/FAMILY:VERSION. External Actions are not mirrored. --output optionally saves the identical signed archive. It is not an offline image bundle. The result includes an immutable Distribution reference.

Remote tag resolutions are recorded in pyplines.lock. Commit it for repeatable builds; --refresh resolves tags again and --frozen refuses missing resolutions without rewriting the lock. Lock entries are platform-specific. Local Action builds use Docker caching; image reproducibility still depends on pinned inputs. Identical existing images may be reused; conflicting version tags are rejected. Registry-side immutability is required to prevent concurrent publication races. Failed builds never delete published images. A local-write failure following publication reports the successfully published Distribution reference.

For a local HTTP registry, explicitly set PYPLINES_REGISTRY_ALLOW_HTTP to its host and port; Docker must also permit that registry. HTTPS is the default.

check performs offline structural checks; it does not import Action source or claim to validate runtime schemas. build validates the complete included resource graph. Server installation remains authoritative for semantic planning, authorization, trust and runtime compatibility.

Each Action's pyproject.toml supplies static project.name, version, requires-python, and the pyplines.action handler entry point. Include source files explicitly with [tool.pyplines.builder] include = ["hello_world.py"] (default: src). Include required README/license/package data there too. No symlinks, hidden directories, environments or private-key files are copied. Review this allowlist: an ordinary source file can still contain a hardcoded secret. Building executes author-controlled packaging code and must only be done for trusted source on a suitable Docker engine.

[tool.pyplines.action] owns network_required, retry_safety, and secret_aliases. The builder embeds them in the canonical Action image manifest; remote consumers read them from the image. Defaults are no network, unsafe retry, and no secrets. Resource size belongs to each Procedure Action step, not the Action.

Python 3.11–3.14 is supported; the newest compatible minor is selected, or use --python. --platform auto uses the Docker engine's architecture. Cross-platform builds require engine-provided emulation. A build targets one architecture; produce a separately signed Distribution for each platform. Multi-platform OCI index assembly is not provided. --base-image permits an explicitly pinned Python base. Otherwise the selected Python slim tag is resolved to a digest before building. Base, source, SDK and recipe attribution is embedded at /opt/pyplines/action/build.json and returned in the build report. Docker provides layer caching. Unlocked third-party dependencies can change between builds; pin dependencies for reproducibility. Distribution bytes are deterministic for the same resource documents and signing key; source-to-image reproducibility is not claimed for unpinned dependencies.

Released builders pin the matching published pyplines SDK. A development builder requires an explicit --sdk-wheel; it never silently includes the repository. Keep a private signing key outside Action sources. Keys are Ed25519 PKCS8 PEM, mode 0600 on POSIX. Existing keys and output archives are never overwritten. Existing registry version tags are reused only for identical images. Enable registry-side immutable tags as well to protect against concurrent publishers. The separate trust.json is public and can be supplied to Server trust configuration; the builder does not install trust or weaken verification.

Registry authentication

The builder manages registry credentials independently of Docker and the installation. One login is used for both Action images and Distribution artifacts:

pyplines-builder registry login registry.digitalocean.com
pyplines-builder registry status registry.digitalocean.com
pyplines-builder build ./hello-world \
  --registry registry.digitalocean.com/pyplines-dev \
  --signing-key ./signing/publisher.pem
pyplines-builder registry logout registry.digitalocean.com

Login prompts for username and a hidden password/token. The Registry v2 endpoint and authentication challenge are checked over HTTP before credentials are saved; failure preserves the existing login. Logout removes only that registry's builder credentials. Docker login is unchanged. Other registries remain authenticated. One identity is saved per registry hostname/port. There is no active-registry selection or source fallback.

For automation, supply a username and read the password/token from stdin:

pyplines-builder registry login registry.digitalocean.com \
  --username YOUR_USERNAME --password-stdin < /secure/registry-token

AUTOMATION_MODE=enabled and --json produce structured output and never prompt. status checks registry access, not repository push permission. Build checks the destination's connectivity/authentication before expensive Action builds, but actual push remains the final permission check. Source Action references select their own credentials; --registry selects exactly one publishing destination.

Credentials live in ~/.pyplines/registry/credentials.json (or beneath PYPLINES_CONFIG_DIRECTORY). Writes are locked, atomic and private on POSIX; the file is not encrypted. Protect the home directory with the operating system's account permissions on all platforms. --registry-credentials or PYPLINES_REGISTRY_CONFIG_FILE explicitly selects an inline Docker-compatible auths JSON file, without fallback. Credential helpers and credsStore are rejected; no Docker CLI or helper subprocess is launched. DOCKER_CONFIG does not select builder credentials. Unset the explicit override to manage saved logins. Never put credentials in source documents or image references.

External HTTPS token authorities require explicit approval before receiving credentials. Interactive login asks for that approval; automation supplies --authentication-host issuer.example (repeatable). No provider-specific mappings or fallback registries exist. --ca-file company-ca.pem adds private CA roots; TLS verification cannot be disabled. Anonymous HTTPS token requests do not send saved credentials. Use registry login HOST --http or PYPLINES_REGISTRY_ALLOW_HTTP=HOST:PORT only for explicit local HTTP testing. Docker never receives registry credentials: Pyplines verifies and transfers OCI image content through ImageLoad/ImageSave. The daemon does not need registry network access or insecure-registry configuration for these operations.

Local runtime tests use the same entrypoint, Unix socket transport, non-root UID, read-only root, capability restrictions and scratch mounts as the appliance. Test limits are 1 CPU, 256 MiB, 128 PIDs, 16 MiB scratch, 64 KiB test input/secrets, 4 MiB collected logs and a configurable 1–3600 second deadline. Only the test's own containers/volume are removed. Build images/cache remain for reuse.

Development

python3 tooling/contracts/sync_builder_contracts.py --check
uv run --project apps/builder pytest -q apps/builder/tests
uv run --project apps/builder black --check apps/builder/pyplines_builder/app.py
uv build --project apps/builder

The generated readers and schema snapshot come from the same sources as the operational CLI. Edit those sources and regenerate, not the bundled copies.

Shared CLI behavior

The Builder follows the same standard as pyplines: readable output by default, or JSON with AUTOMATION_MODE=enabled / --json. Automation never prompts. Help requires no Docker connection. Use version or --version for the installed package version. Common behavior is provided by pyplines-cli-common.

Publish to an OCI registry

pyplines-builder publish ./maintenance.pyplines.tar.gz \
  --to oci://ghcr.io/acme/maintenance:1.2.0 \
  --trust-file ./publisher-trust.json

Publication verifies signatures before uploading and returns an immutable reference. No Docker daemon or Index is required for publishing a built archive. Publisher authority (source in signing configuration) is a stable HTTPS URI, not a required Index endpoint or registry hostname.

Use builder registry login or --registry-credentials path/to/config.json with inline auths. PYPLINES_REGISTRY_CONFIG_FILE also selects a config file. Registry credentials are never publisher trust. Use a private file with explicit auths for CI rather than secrets in arguments. The builder refuses conflicting existing tags; enable registry-side immutable tags to protect against concurrent writers. HTTP is only enabled for exact hosts listed in PYPLINES_REGISTRY_ALLOW_HTTP, for isolated local tests.

These archives use the registry-independent v1alpha1 publication statements; rebuild older Index-signed authoring artifacts with the current builder.