Hightouch Provider for Airflow
Apache Airflow Provider for Hightouch
Provides an Airflow Operator and Hook for Hightouch. This allows the user to initiate a run for a sync from Airflow.
Pre-requisites: An environment running
apache-airflow >= 1.10, including >= 2.
pip install airflow-provider-hightouch
In the Airflow Connections UI, create a new connection for Hightouch.
Password: enter the API key for your workspace. You can generate an API key from your Workspace Settings
The Operator uses the
hightouch_default connection id by default, but
if needed, you can create additional Airflow Connections and reference them
in the operator
Starts a Hightouch Sync Run. Requires the
sync_id or the
sync_slug for the sync you wish to
sync_run_id of the sync it triggers.
The run is synchronous by default, and the task will be marked complete once the sync is successfully completed.
However, you can request a asynchronous request instead by passing
to the operator.
If the API key is not authorized or if the request is invalid the task will fail. If a run is already in progress, a new run will be triggered following the completion of the existing run.
Monitors a Hightouch Sync Run. Requires the
sync_id and the
sync_run_id of the sync you wish to monitor.
To obtain the
sync_run_id of a sync triggered in Airflow, we recommend using XComs to pass the return value
Creating a run is as simple as importing the operator and providing it with a sync_id. An example dag is available as well.
from airflow_provider_hightouch.operators.hightouch import HightouchTriggerSyncOperator with DAG(....) as dag: ... my_task = HightouchTriggerSyncOperator(task_id="run_my_sync", sync_id="123") my_other_task = HightouchTriggerSyncOperator(task_id="run_my_sync", sync_slug="my-sync-slug")
Please submit issues and pull requests in our official repo: https://github.com/hightouchio/airflow-provider-hightouch
We are happy to hear from you, for any feedback please email the authors at email@example.com.
Release history Release notifications | RSS feed
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Hashes for airflow-provider-hightouch-4.0.0.tar.gz
Hashes for airflow_provider_hightouch-4.0.0-py3-none-any.whl