Daylily Ephemeral Cluster

Daylily Ephemeral Cluster, usually called DYEC or DayEC, is the CLI control plane for short-lived AWS ParallelCluster bioinformatics work. Release 19.0.91 creates and configures clusters, mounts sequencing-run data into FSx, launches pinned workflow repositories on the headnode, monitors exact analysis roots, moves files between local and headnode storage, and exports finished results to S3 with receipts.

DYEC is not an identity service and not a workflow engine. It does not require or contact a metadata or identity service. It consumes explicit local configuration, explicit manifests, explicit S3 paths, and explicit command-catalog entries. DayOA owns its workflow rules and dy-r execution. DYEC owns cluster/headnode orchestration and the launch/export envelope.

Agent and operator starting point: read docs/agent_cli_guide.md before operating a cluster or DayOA analysis. It maps the common DYEC CLI paths, the interactive DayOA contract, and the stop conditions. dyec agent guidance is the matching compact terminal reminder.

Current operator model

Most work follows this shape:

cd /Users/jmajor/projects/lsmc/daylily-ephemeral-cluster
source ./activate

export AWS_PROFILE=lsmc
export REGION=us-west-2
export CLUSTER=ifx-p2-1000-120-0715
export STAGING_S3_URI=s3://<bucket>/<temporary-dyec-payload-prefix>/
export ANALYSIS_ID=<analysis-id>

Inspect the installed CLI before mutating anything:

dyec --json version
dyec --help
dyec agent guidance
dyec --json catalog list --dyec-version 19.0.91

Optional project-local context

When working repeatedly from one DYEC checkout, save the four recurring DYEC values in its ignored local context file rather than exporting a second set of environment variables:

dyec set-vars \
  --profile "$AWS_PROFILE" \
  --region "$REGION" \
  --region-az us-west-2d \
  --cluster-admin-email operator@example.org

dyec -v --json info

The project-local context loader reads only $PWD/.dyec.config.yaml; it does not use environment variables as a second local-context store. The file accepts only the four documented string keys; malformed YAML, unknown keys, and non-string values fail clearly. Blank and whitespace-only values are unset. For --profile, --region, and --region-az, an explicit flag wins over this local file, which wins over the command's existing behavior. A required option may therefore be omitted only when its matching local value is present. Use explicit flags to override the local context for one command, dyec unset-vars --region to clear one value, or dyec unset-vars to clear the file. -v prints the resolved local-context diagnostic to stderr before the subcommand, preserving JSON stdout.

Use --cluster for DYEC commands. Keep --cluster-name for tools such as pcluster that require that spelling.

Safety contracts

• Use dyec; do not launch DayOA by invoking raw snakemake.
• Upstream services use the installed dyec console script for every cluster lifecycle operation. They do not run pcluster, import DYEC Python internals, or use a module entrypoint as an alternate control path.
• Saved cluster templates keep SpotPrice: CALCULATE_MAX_SPOT_PRICE on every Spot compute resource. Root dyec create resolves the normal config and performs live spot-price calculation before provider dry-run/create; saved templates never persist a calculated numeric bid.
• New DayOA checkouts must be explicit-tag checkouts.
• A DYEC controller never mutates a pinned DayOA release: no runtime rule/script/environment/config patches, source overlays, or generated helpers in the checkout. It verifies the selected ref is clean before dispatch and after the workflow returns. Missing behavior is a hard error that must be fixed and released in DayOA, never repaired on the headnode.
• Headnode work uses a cluster-appropriate remote user selected by platform: Ubuntu/Intel DayOA headnodes use ubuntu; DRAGEN/RHEL-style headnodes use ec2-user.
• DYEC-created headnode shells must be bash login/interactive contexts and source ~/.bashrc; workflow controllers still run in persistent tmux panes.
• DYEC CLI launch helpers create the supported headnode controller for you; they do not require an interactive SSM session for standard catalog launches.
• Missing files, missing credentials, unsafe identity fields, malformed manifests, unexpected legacy input shapes, and insufficient staging permissions fail hard.
• S3 relay prefixes for upload/download and staged workflow launch must be readable/writable by both the local operator credentials and the headnode instance role.
• /fsx/analysis_results/** workflow writes, unlocks, deletes, restarts, and kills require analysis-root write-lock ownership.
• Headnode inspection commands are supported; Slurm/node administration is not implied by inspection.

Command groups

Run dyec --help for the live list. Current major groups are:

Detailed option-level examples live in docs/cli_reference.md. The task-oriented path is docs/agent_cli_guide.md.

Guarded AWS Budget updates

dyec aws budget set-limit changes only an existing fixed monthly USD AWS Budget. The caller must provide the exact currently observed cap, and can prove the whole contract without mutating AWS by using --dry-run:

dyec aws budget set-limit "$CLUSTER" \
  --profile "$AWS_PROFILE" \
  --region "$REGION" \
  --expected-current-monthly-cap-usd 200 \
  --monthly-cap-usd 300 \
  --dry-run

After the required budget-change approvals, repeat the same command without --dry-run. A stale current-cap value, an unsupported budget shape, or a readback mismatch fails explicitly; requesting the already-current cap is a successful no-op.

Quick cluster and queue checks

Use the aggregate read-only view to see which clusters are ready and how many Slurm jobs each has. A provisioning or teardown cluster is shown as CLUSTER_NOT_READY, not as an idle queue.

dyec cluster jobs --profile "$AWS_PROFILE" --region "$REGION"

For the exact job list on one cluster, use the drill-down command:

dyec headnode jobs --profile "$AWS_PROFILE" --region "$REGION" --cluster "$CLUSTER"

Read-only overview, prompt, and cockpit

Collect one explicit region as typed JSON, or inspect one exact cluster's two cost controls separately:

dyec --json overview snapshot --profile "$AWS_PROFILE" --region "$REGION"
dyec --json cost-controls status \
  --profile "$AWS_PROFILE" --region "$REGION" \
  --cluster "$CLUSTER" --cluster-budget-name "$CLUSTER" \
  --cost-center "${CLUSTER}-ccenter"

The overview's cluster burn/hour includes active EC2, EBS, FSx, and public IPv4. It excludes S3, transfer, NAT, and CUR-delayed costs. An unavailable required component makes the total unknown (null in JSON and ? in text), never a false zero.

Configure an ordered, cached shell summary and opt into the generated hook:

dyec prompt configure --profile "$AWS_PROFILE" --region "$REGION" --label UW2
dyec prompt refresh
eval "$(dyec shell init zsh --prompt)"  # use bash for Bash

Prompt rendering reads only $XDG_CACHE_HOME/daylily/prompt-v1.json (or ~/.cache/daylily/prompt-v1.json). At the next prompt, a cache at least 15 minutes old starts one locked refresh asynchronously; shell startup and prompt rendering never wait on AWS.

For the full-screen read-only view and its DYEC-local themes:

dyec cockpit --profile "$AWS_PROFILE" --region "$REGION"
dyec theme list
dyec theme preview daylight
dyec theme set daylight

The cockpit shows its underlying commands and JSON and has no mutation keys. Terminal tint is a separate explicit dyec theme terminal apply <theme> / restore action. See overview_prompt_and_cost_controls.md and cockpit_and_themes.md.

CLI-first catalog launch

Catalog launch is the preferred path for known DayOA commands because it renders the exact dyec workflow launch command before it starts anything. The cluster AWS Budget is --cluster-budget-name and must equal --cluster; the Slurm attribution identity is --cost-center. If omitted, DYEC resolves them to the exact cluster and <cluster>-ccenter, respectively. See docs/cost_context.md for the receipt, hash, and narrowly supported migration aliases.

Render first:

dyec --json catalog render hybrid_ilmn_ont_hiomr_kitchensink \
  --profile "$AWS_PROFILE" \
  --region "$REGION" \
  --cluster "$CLUSTER" \
  --analysis-id "$ANALYSIS_ID" \
  --executing-entity "$CLUSTER" \
  --manifest-dir ./config \
  --payload-staging-s3-uri "$STAGING_S3_URI" \
  --session-name "${ANALYSIS_ID}-dryrun" \
  --cluster-budget-name "$CLUSTER" \
  --cost-center "${CLUSTER}-ccenter" \
  --dry-run

Launch the rendered dry run:

dyec --json catalog launch hybrid_ilmn_ont_hiomr_kitchensink \
  --profile "$AWS_PROFILE" \
  --region "$REGION" \
  --cluster "$CLUSTER" \
  --analysis-id "$ANALYSIS_ID" \
  --executing-entity "$CLUSTER" \
  --manifest-dir ./config \
  --payload-staging-s3-uri "$STAGING_S3_URI" \
  --session-name "${ANALYSIS_ID}-dryrun" \
  --cluster-budget-name "$CLUSTER" \
  --cost-center "${CLUSTER}-ccenter" \
  --dry-run

If the dry-run plan is bounded and correct and the production contract requires a same-root continuation, do not issue a second catalog launch against the existing root. Start a new dyec workflow launch controller with --reuse-existing-analysis-dir --input-contract none --no-input-staging, the exact ref/commit, and the rendered live dy-r argv with only -n removed. The full continuation command and its lock boundary are in docs/agent_cli_guide.md.

For scalar DayOA runtime config, pass explicit key=value overrides. DYEC appends them to the dy-r ... --config section and does not reinterpret their workflow-specific meaning:

dyec --json catalog render hybrid_ilmn_ont_hiomr_kitchensink \
  --analysis-id hg003-hg004-ds025-ont0to6 \
  --profile "$AWS_PROFILE" --region "$REGION" --cluster "$CLUSTER" \
  --manifest-dir ./config \
  --payload-staging-s3-uri "$STAGING_S3_URI" \
  --dry-run \
  --dy-config use_fq_data_starting_hrs=0 \
  --dy-config use_fq_data_up_to_hrs=6 \
  --dy-config global_sr_subsample_pct=0.25 \
  --dy-config global_ont_subsample_pct=0.25

Typed YAML analysis configuration must be materialized inside the cloned DayOA analysis directory, normally under config/, and passed to dy-r through an in-clone --configfile path. Never retain a controller-specific YAML/config under /home/ubuntu, /tmp, or another path outside the clone. Together with the in-clone manifests and saved command/receipts, the clone is the rerunnable analysis capsule; only the explicitly declared source reads, CRAMs, references, licenses, and runtime assets remain external.

Large local payloads are staged through S3 with --payload-staging-s3-uri. DYEC uploads a tarball containing input manifests, a payload manifest, and the controller launch script. The headnode downloads and expands that tarball into the workflow run directory, starts the tmux controller, and then saves the exact executed script under <analysis-root>/bin/dyec-controller-launch.sh after day-clone creates the analysis root. This avoids SSM document-size limits without pre-creating the analysis root.

Immutable command shapes

Catalog version 6 requires an exact numeric --dyec-version for every public catalog action. The 19.0.26 snapshot is immutable; there is no mutable current alias or implicit catalog selection:

dyec --json catalog list --dyec-version 19.0.26 --type prod
dyec --json catalog render <command-id> --dyec-version 19.0.26 ...

A build may also declare one-hop, same-build aliases. An alias inherits one direct command, applies typed metadata overrides, and either extends its DayOA targets/config or supplies a complete replacement of targets, dy_command, and dryrun_dy_command. Alias chains, cycles, cross-build references, missing bases, duplicate IDs, mixed extension/replacement modes, and partial command replacements fail catalog validation. Existing catalog APIs return aliases as fully resolved AnalysisCommand records.

The 19.0.26 catalog preserves the 19.0.25 command shapes and targets DayOA 16.0.6. Public catalog output includes a derived validation_pending field: true means the command now targets a different DayOA tag than its retained validated_version. It is a visibility signal only; it does not relabel older validation receipts or block a launch.

Each command may declare an explicit validation_evidence_s3_uri_prefix. The prefix must contain command_registry.json and summary.json from a successful dyec tests command-catalog run. Compare it read-only with:

dyec --json catalog validation-compare <command-id> \
  --dyec-version 19.0.26 --profile "$AWS_PROFILE" --region "$REGION"

The comparison fails hard when no prefix is declared, the receipts are missing, the captured DayOA pin differs, or the recorded command phase did not succeed.

Workflow launch without the catalog shortcut

Use dyec workflow launch when you already know the exact DayOA command string or are launching a non-catalog repository command:

dyec workflow launch \
  --profile "$AWS_PROFILE" \
  --region "$REGION" \
  --cluster "$CLUSTER" \
  --analysis-id "$ANALYSIS_ID" \
  --executing-entity "$CLUSTER" \
  --git-tag 16.0.75 \
  --manifest-dir ./config \
  --payload-staging-s3-uri "$STAGING_S3_URI" \
  --session-name "$ANALYSIS_ID" \
  --cluster-budget-name "$CLUSTER" \
  --cost-center "$COST_CENTER" \
  --dry-run \
  --rerun-triggers mtime \
  --dy-command "dy-r produce_sentdhiomr_snv_vcf produce_sentdhiomr_sv produce_sentdhiomr_cnv -j 100 -p -T 0 --rerun-incomplete"

--dry-run appends -n to the effective dy-r command even when --dy-command is supplied; do not duplicate -n merely to make a direct launch dry. Use repeatable --rerun-triggers for explicit Snakemake trigger selection (for example, mtime). When that public option is present, --dy-command must not also contain --rerun-triggers. For the same-root live continuation, remove only --dry-run and retain the same rerun-trigger option, analysis ID, DayOA commit, manifests, and in-clone configuration.

Six-manifest DayOA inputs

Current DayOA sample-analysis launches use exactly six manifest files:

specimens.tsv
samples.tsv
libraries.tsv
sequencing_inputs.tsv
analysis_units.tsv
analysis_unit_inputs.tsv

Validate them locally before launch:

dyec --json identities validate --manifest-dir ./config
dyec --json identities status --manifest-dir ./config

The identity commands are local and provider-neutral. They never create or resolve production identities. EUID fields may be blank for ordinary analysis. Test-only EUID-like values must use the reserved Z- prefix and are not valid customer-release identities.

Bjuice prevalence manifest helper

dyec catalog config-bjuice-preval builds a six-manifest directory from reviewed Bjuice evidence files. It is deliberately specific to that reviewed evidence contract; it is not a general identity resolver.

dyec --json catalog config-bjuice-preval \
  --sample HG003 \
  --sample HG004 \
  --output-dir ./config-hg003-hg004 \
  --source-manifest-json /path/to/source_manifest_resolved.json \
  --run-evidence-json /path/to/run_evidence_v2.json \
  --library-run-matrix-tsv /path/to/bjuice_preval_library_run_matrix.tsv \
  --sample-metadata-tsv /path/to/samples.tsv \
  --legacy-units-tsv /path/to/units.tsv \
  --sr-subsample-pct 0.25 \
  --ont-subsample-pct 0.25 \
  --profile "$AWS_PROFILE" \
  --region "$REGION"

The helper writes the six manifest TSVs plus bjuice_preval_config_receipt.json, validates the manifest set, and records file hashes. It fails if reviewed sample metadata, legacy unit metadata, S3 listings, or expected ILMN/ONT source groups are absent or ambiguous.

Bjuice v2 HG002 full-prevalence multi-AU helper

dyec catalog config-bjuice-v2-hg002-multi-au is a separate, fixed contract for the seven HG002 analysis units p5xp5, 1x1, 3x3, 5x5, 10x5, 15x5, and 15x10. It accepts one direct Illumina coverage denominator and requires a matching terminal receipt; it never derives coverage from total or hybrid evidence.

dyec --json catalog config-bjuice-v2-hg002-multi-au \
  --output-dir ./config-hg002-bjuice-v2 \
  --source-manifest-json /path/to/source_manifest_resolved.json \
  --run-evidence-json /path/to/run_evidence_v2.json \
  --library-run-matrix-tsv /path/to/bjuice_preval_library_run_matrix.tsv \
  --sample-metadata-tsv /path/to/samples.tsv \
  --legacy-units-tsv /path/to/units.tsv \
  --direct-ilmn-coverage-x "$C_ILMN" \
  --direct-ilmn-coverage-evidence /path/to/direct_ilmn_terminal_receipt.json \
  --profile "$AWS_PROFILE" \
  --region "$REGION"

The receipt must use dyec.bjuice_v2_direct_ilmn_coverage_receipt.v1, identify HG002, have terminal status, and provide a matching ilmn_direct_coverage_x. The output has blank nullable live EUID fields, writes per-AU ONT_FQ_START_HOUR/ONT_FQ_END_HOUR, and calculates SUBSAMPLE_PCT = target_x / C_ILMN at 12 decimal places with ROUND_DOWN. It fails before writing output if the receipt is missing or ambiguous, coverage is non-positive, or a requested target exceeds the verified denominator.

Headnode commands and transfer

Open a full interactive shell when you need one:

dyec headnode connect \
  --profile "$AWS_PROFILE" \
  --region "$REGION" \
  --cluster "$CLUSTER"

Run a bounded command non-interactively and get stdout/stderr back:

dyec --json headnode run \
  --profile "$AWS_PROFILE" \
  --region "$REGION" \
  --cluster "$CLUSTER" \
  'hostname; command -v aws; aws sts get-caller-identity --output json'

Copy files through an explicit S3 relay:

dyec --json headnode upload -r ./config \
  /tmp/dyec-staged-config \
  --profile "$AWS_PROFILE" \
  --region "$REGION" \
  --cluster "$CLUSTER" \
  --staging-s3-uri "$STAGING_S3_URI"

dyec --json headnode download -r \
  /fsx/analysis_results/"$CLUSTER"/"$ANALYSIS_ID"/daylily-omics-analysis/results/day/hg38/reports \
  ./downloaded-reports \
  --profile "$AWS_PROFILE" \
  --region "$REGION" \
  --cluster "$CLUSTER" \
  --staging-s3-uri "$STAGING_S3_URI"

The relay prefix is intentionally explicit and auditable. DYEC does not delete relay objects automatically.

Monitoring exact analyses

Cluster monitor: current CLI

dyec cluster monitor \
  --profile "$AWS_PROFILE" --region "$REGION" \
  --clusters "$CLUSTER" [--clusters "$ANOTHER_CLUSTER"] \
  --monitor-type <cluster|headnode|controller|jobs|au> \
  --activity <active|inactive|all> \
  --detail-level <summary|high> \
  --output-mode <ascii|ansi|table|json|fancy|html-server> \
  [--server-port <1024-65535>] [--tail] [--frequency <duration>]

Repeat --clusters for clusters in the same region. --activity applies to the controller, jobs, and au monitor types. The default mode is ascii; ansi and fancy require an interactive terminal. html-server additionally requires an explicit non-root --server-port.

Jobs: complete sq table

For a full active-queue view, use the monitor directly. It returns every active sq row in sq field order, with CLUSTER added as the first column:

source ./activate

dyec cluster monitor \
  --profile "$AWS_PROFILE" --region "$REGION" \
  --clusters "$CLUSTER" \
  --monitor-type jobs --activity active --detail-level high \
  --output-mode table

The result uses lossless SSM transport: DYEC does not apply a monitor row or receipt-size cap. At high detail, ascii, ansi, table, and fancy all preserve this complete table rather than replacing it with a summary. Use --output-mode json for the same complete typed frame.

Fancy terminal overview

Use fancy for an interactive, compact visual overview of a monitor type such as controllers. For high-detail jobs, use the table above so no queue field or row is hidden.

dyec cluster monitor \
  --profile "$AWS_PROFILE" --region "$REGION" \
  --clusters "$CLUSTER" \
  --monitor-type controller --activity active --detail-level high \
  --output-mode fancy

Live HTML dashboard

html-server starts a local, loopback-only dashboard and refreshes its typed frame at the selected cadence. It needs no root privileges; open the printed http://127.0.0.1:<port>/ URL in a browser and use Ctrl-C to stop it.

dyec cluster monitor \
  --profile "$AWS_PROFILE" --region "$REGION" \
  --clusters "$CLUSTER" \
  --monitor-type jobs --activity active --detail-level high \
  --output-mode html-server --server-port 8891 --frequency 5m

The page shows the complete active sq table with CLUSTER first and updates it in place. The typed monitor frame remains available in the page for direct inspection.

For a workflow launched by DYEC, inspect the exact run-state receipt, controller target, active Snakemake log, progress, submitted/finished jobs, and current Slurm states together:

dyec --json workflow status \
  --profile "$AWS_PROFILE" \
  --region "$REGION" \
  --cluster "$CLUSTER" \
  --session "$ANALYSIS_ID"

dyec --json workflow status \
  --profile "$AWS_PROFILE" --region "$REGION" --cluster "$CLUSTER" \
  --session "$ANALYSIS_ID" --rule sentdhiomr2_hybrid_cli172i_core

dyec workflow logs \
  --profile "$AWS_PROFILE" \
  --region "$REGION" \
  --cluster "$CLUSTER" \
  --session "$ANALYSIS_ID" \
  --stream snakemake \
  --lines 200

dyec workflow logs \
  --profile "$AWS_PROFILE" --region "$REGION" --cluster "$CLUSTER" \
  --session "$ANALYSIS_ID" --stream snakemake \
  --match "Error in rule" --before-lines 40 --after-lines 80 --max-matches 1

workflow status emits exactly one derived state: RUNNING, SUCCEEDED, FAILED, or UNKNOWN. SUCCEEDED and a terminal exit code require the matching DYEC status.json; a successful status-inspection command is never the workflow exit code. CONFIGURING and RUNNING Slurm jobs are ongoing work, and an empty queue is never success. Failure detection uses anchored Snakemake terminal markers and deliberately ignores generic ERROR text and printed shell bodies.

Submitted, finished, and active Slurm record collections are complete in the transport payload. An exact --rule selector adds job and unique-AU counts for that rule, including finished work and each current Slurm state; returned AU names are complete as well.

The Snakemake stream attributes and reads the log in one remote probe. Its requested tail is compressed, integrity-checked, and decoded locally. The selected --lines tail is returned in full through the lossless transport. Literal --match mode searches the complete exact attributed log and returns the complete user-selected context, allowing old errors to be retrieved after a verbose -p log has rolled beyond the requested tail.

For a controller started manually during recovery, provide its identity explicitly; DYEC does not discover a checkout or guess the newest log:

dyec --json workflow status \
  --profile "$AWS_PROFILE" --region "$REGION" --cluster "$CLUSTER" \
  --repo-path /fsx/analysis_results/<owner>/<analysis-id>/daylily-omics-analysis \
  --controller-pid <pid> \
  --session <exact-tmux-session>

If the controller is no longer live or descriptor correlation is unavailable, add the exact --snakemake-log <repo-path>/.snakemake/log/<timestamp>.snakemake.log. Manual inspection can prove RUNNING or a high-signal FAILED state, but it cannot prove SUCCEEDED or invent a terminal RC without a matching DYEC launch receipt.

DYEC-launched controllers redirect output directly to the regular .dyec/controller.log file; they never put dy-r behind tee. Immediately after dy-r returns, the launcher atomically records workflow_completed_at and workflow_exit_code in the matching status.json, before DAG evidence, export, or other post-processing. Final controller completed_at/exit_code fields take precedence if later post-processing changes the launch outcome. For an exceptional manual recovery, likewise redirect dy-r directly to a regular file and follow it from a separate tail -f process. Do not use dy-r ... | tee ...: orphaned workflow helpers can inherit the pipe and delay the shell from persisting its RC. A manual printed RETURN CODE remains non-authoritative to workflow status; use the standard DYEC launcher when a terminal success receipt is required.

Record visits before analysis-root reads:

dyec analysis visit \
  --analysis-root /fsx/analysis_results/"$CLUSTER"/"$ANALYSIS_ID" \
  --mode read \
  --intent "status check"

Use the exact-root status reporter:

dyec analysis status slim \
  --analysis-root /fsx/analysis_results/"$CLUSTER"/"$ANALYSIS_ID" \
  --profile "$AWS_PROFILE" \
  --region "$REGION" \
  --cluster "$CLUSTER"

dyec analysis status full \
  --analysis-root /fsx/analysis_results/"$CLUSTER"/"$ANALYSIS_ID" \
  --profile "$AWS_PROFILE" \
  --region "$REGION" \
  --cluster "$CLUSTER" \
  --tail-lines 1000

For HIOMR command-family progress and optional DAG PNG download:

dyec --json command sample-stats hiomr-kitchensink \
  --name hg003_hiomrs \
  --analysis-root /fsx/analysis_results/"$CLUSTER"/"$ANALYSIS_ID" \
  --profile "$AWS_PROFILE" \
  --region "$REGION" \
  --cluster "$CLUSTER" \
  --dag-output ./tmp/hg003_hiomrs_dag.png

Queue emptiness is never success. Success requires controller exit code plus expected terminal artifacts for the workflow.

Run-directory QC commands

Run QC catalog entries are run-context commands. Mount the run directory first, then pass a run-context TSV to catalog render or catalog launch.

dyec --json mounts create s3://<sequencing-run-bucket>/<run-prefix>/ \
  --profile "$AWS_PROFILE" \
  --region "$REGION" \
  --cluster "$CLUSTER" \
  --platform ILMN \
  --read-only \
  --wait \
  --timeout-seconds 5400

run_context requires all of these exact TSV headers. For a run_dra_required command, RUN_DIR is the verified mounted path and MOUNT_ID is the explicit mount identifier; do not substitute the old three-column shape:

RUNID	PLATFORM	RUN_DIR	SOURCE_S3_URI	MOUNT_ID	SAMPLE_SHEET	BASECALLING_STATE	RUN_STATUS	OUTPUT_ROOT	REGION	PROFILE

Render each supported run-QC catalog command before launching it:

dyec --json catalog render illumina_run_qc \
  --analysis-id ilmn-runqc-20260722 \
  --profile "$AWS_PROFILE" --region "$REGION" --cluster "$CLUSTER" \
  --run-context-file ./runs.tsv \
  --dry-run

dyec --json catalog render ont_run_qc \
  --analysis-id ont-runqc-20260722 \
  --profile "$AWS_PROFILE" --region "$REGION" --cluster "$CLUSTER" \
  --run-context-file ./runs.tsv \
  --dry-run

dyec --json catalog render ultima_run_qc \
  --analysis-id ultima-runqc-20260722 \
  --profile "$AWS_PROFILE" --region "$REGION" --cluster "$CLUSTER" \
  --run-context-file ./runs.tsv \
  --dry-run

The catalog enforces each command’s required PLATFORM value.

Export

Export one completed analysis directory:

dyec analysis visit \
  --analysis-root /fsx/analysis_results/"$CLUSTER"/"$ANALYSIS_ID" \
  --mode export \
  --intent "export completed pipeline results to $DESTINATION_S3_URI without FSx cleanup"

dyec export \
  --profile "$AWS_PROFILE" \
  --region "$REGION" \
  --cluster "$CLUSTER" \
  --source-path /fsx/analysis_results/"$CLUSTER"/"$ANALYSIS_ID" \
  --destination-s3-uri "$DESTINATION_S3_URI" \
  --destination-policy new \
  --output-dir ./export-receipts/"$ANALYSIS_ID"

dyec catalog list, show, and render expose the same result_export contract. DayOA never exports: after the controller succeeds, run the displayed DYEC visit and DRA export commands from the analysis root. The default --destination-policy new requires an empty prefix. Explicit --destination-policy update-existing adds new objects and replaces objects for modified FSx files in an existing exact destination. It never deletes S3 objects for files absent from FSx and is not a mirror or two-way sync. dyec export records a local receipt and uses an explicit DRA/export path. Verify status=success, phase=complete, task_lifecycle=SUCCEEDED, detached=true, s3_delete_requested=false, and the expected S3 objects. FSx data is preserved unless a separately approved FSx cleanup is explicitly run; such cleanup never deletes S3 data.

Development and tests

Use the repo environment:

cd /Users/jmajor/projects/lsmc/daylily-ephemeral-cluster
source ./activate
python -m pytest tests/test_cli_registry_v2.py -q
git diff --check

The active catalog has release-agnostic local contract coverage. Historical catalog snapshots remain in immutable Git tags rather than the ordinary test suite:

python -m pytest \
  tests/test_current_catalog_contract.py \
  tests/test_tests_runner.py \
  -q

Release tags are numeric, annotated semver tags with no leading v. Do not move pushed tags. The checked-in release target is 19.0.26; tag only the exact clean release commit after required acceptance.