vibedata-dlt-redshift-studio

The dlt runtime execution unit for an Amazon Redshift Domain, installed in the Studio-local intent venv.

import dlt
from vibedata.dlt.redshift import setup_environment, finalize

setup_environment()
pipeline = dlt.pipeline(
    pipeline_name="my_pipeline", destination="redshift", dataset_name="analytics"
)
try:
    pipeline.run(source)
except Exception as e:
    finalize(pipeline, error_message=str(e))
    raise
finalize(pipeline)

setup_environment() registers the Domain's Secret Store provider and mints the Ephemeral database's password into this process, under dlt's own DESTINATION__REDSHIFT__CREDENTIALS__PASSWORD. Studio publishes the other four coordinates — HOST, PORT, DATABASE, USERNAME — into the Intent environment under the matching names, and dlt reads those natively, so the pipeline names no VD_* variable itself.

The password is separate because Redshift authenticates with a credential minted per invocation, scoped to one database and valid for at most an hour.

Without one, dlt refuses to configure the destination and names DESTINATION__REDSHIFT__CREDENTIALS__PASSWORD — which no authored pipeline sets, so this unit fails at the mint and names the Broker instead. A blank password is the case worth guarding: dlt resolves it cleanly and the load then reaches Redshift as a real login attempt, so an empty Broker answer is refused rather than set.

The minted password lives in os.environ for the life of the process, as the MotherDuck unit's token does — inherited by any child it spawns.

finalize() persists the audit tables. A Redshift load writes directly, so there is nothing to sync afterwards.