rudderstack-airflow-provider 1.0.0

airflow provider for rudderstack

Customer Data Platform for Developers

rudder-airflow-provider

Apache airflow provider for rudderstack.

Questions? Start a conversation on our Slack channel.

Why Use rudder-airflow-provider

Trigger wh-action or cloud extract syncs for rudderstack from apache airflow.

Installation

pip install rudderstack-airflow-provider

Usage

A simple DAG for triggering sync for rudderstack source. For complete code, refer example

with DAG('rudderstack-sample',
    default_args=default_args,
    description='A simple tutorial DAG',
    schedule_interval=timedelta(days=1),
    start_date=datetime(2021, 1, 1),
    catchup=False,
    tags=['rs']) as dag:
    rs_operator = RudderstackOperator(source_id='<source-id>', task_id='<any-task-id>', connection_id='rudderstack_conn')

Operator Params

parameter	description	type	default
source_id	valid rudderstack source id	string	None
task_id	a unique task id within a dag	string	None
wait_for_completion	if true task will wait for sync to complete	boolean	False
connection_id	airflow connection to use for connecting to rudderstack api	string	rudderstack_default

Rudderstack operator supports all the parameters supported by airflow base operator

For more detailed guide on how to run the DAG in airflow, refer to this guide

License

The rudderstack-airflow-provider is released under the MIT License.

Contribute

We would love to see you contribute to RudderStack. Get more information on how to contribute here.

Follow Us

• RudderStack blog
• Slack
• Twitter
• LinkedIn
• dev.to
• Medium
• YouTube
• HackerNews
• Product Hunt