continuo-validation-runner

The slim, continuo-owned validation harness. It dispatches on VALIDATION_OP (build_from_sql / clone_from_prod / build_from_columns / ensure_schema / drop_schema), fetches a node's candidate SQL or column spec from S3 when relevant, discovers the single installed engine adapter (entry-point group continuo_validation.adapters), runs the corresponding warehouse DDL through it, and prints exactly one sentinel-framed result block on stdout.

It depends on continuo-validation-contract (the port + result-block) and, at image-build time, on exactly one continuo-validation-<engine> library — Dockerfile.postgres bakes in continuo-validation-postgres. Inside the shipped images, the contract, the runner, and the engine adapter all install from this repo's own workspace paths (contract/, runner/, adapters/<engine>/) in a single pip transaction at image build time, not from PyPI — see the Dockerfile comments for why. The container entrypoint is python /validation_runner.py, which continuo's executor invokes. python -m continuo_validation_runner reaches the identical main and is convenient for local invocation, but it is not the image contract on its own — an image built around this package standalone (e.g. for an engine this repo doesn't ship) has more to satisfy than that one command. This package is also published to PyPI as continuo-validation-runner for exactly that use; see the bring-your-own-adapter section of the main README for the full image contract.

Job, not a service

This is not one of continuo's long-running services — see the service list at github.com/carolsimone/continuo. It ships as a container image (continuo-validation-<engine>) that executor-controller dispatches as a one-shot Kubernetes Job (BackoffLimit: 0, RestartPolicy: Never) per node or per schema op. The process reads VALIDATION_OP, runs it once, prints one result block, and exits — there is no process to keep alive, no gRPC/HTTP surface, and no owned datastore.

Its entire behavior is the five ops dispatched through the ValidationAdapter port (see continuo-validation-contract) plus the sentinel result-block wire format. Because that surface is small and fixed, this runner itself is very unlikely to need changes: adding support for a new warehouse engine means adding a new continuo-validation-<engine> adapter package and pointing VALIDATION_IMAGE at an image built from it, not editing this code.

The sentinel result-block format is frozen: continuo's Go package pkg/validationresult parses it, so changing it is a coordinated, versioned event across both repos, not a local edit.

build_from_columns

A python node's output table has no compiled SELECT for the harness to shape it from the way a SQL node's does — the script that actually produces the node's rows only runs at python-node runtime, after validation. So instead of build_from_sql's "materialize the candidate SQL empty," this op fetches a JSON spec from CANDIDATE_SPEC_URI (an s3:// URI, same AWS/S3 credential env as CANDIDATE_SQL_URI):

{"reads": ["<sql>", ...], "output_columns": [{"name": "...", "type": "...", "nullable": true}, ...], "config": {...}}

The body must be a JSON object; reads may be empty (a node with no upstream reads) but output_columns must be non-empty; config is optional — absent or null means {} — but when present must itself be a JSON object. Any of those three failures is a bad-input exit (2), before the adapter is ever touched. config is the node's physical-layout block (indexes, sort keys, partitioning, …); the harness checks only that it's a JSON object and hands it to the installed adapter unread — which keys are legal is the engine's call, not the harness's. Because a python node's own reads aren't implicitly validated the way a SQL node's SELECT is, the harness calls the adapter's check_binds on every reads entry, in declared order, before calling build_empty_from_columns with output_columns and config. Binds gate the build: a read that fails to bind — for example against a column an upstream node dropped — fails the op and the output table is never created.

Each reads entry must be a single read query, and adapters enforce that by parsing it (continuo_validation_contract.sql.ensure_single_read) before any of it reaches the warehouse: an entry that parses as more than one statement, or as a statement that isn't a query, fails the op rather than executing. Parsing is the enforcement — an adapter that only wrapped the read in a subquery would still let a read that balances the wrap's own parentheses smuggle a statement through on any driver that batches ;-separated statements.