An Apache Airflow provider package for Datazone
Project description
Airflow Datazone Provider
A provider package for Apache Airflow that provides a set of operators and hooks to interact with Datazone.
Installation
You can install this package using pip
:
pip install airflow-datazone-provider
Usage
Creating a connection
To integrate with Datazone on Airflow, you can create a connection in the Airflow UI with the following parameters:
- Conn Id:
<your_conn_id>
- Conn Type:
Datazone
- Host:
<your_datazone_host>
- API Key:
<your_datazone_api_key>
- Extra: Optional extra parameters in JSON format. For example, to specify a compute, you can use the following JSON:
{
"compute": "XSMALL"
}
DatazoneRunExecutionOperator
The DatazoneRunExecutionOperator
is an operator that allows you to run an execution on Datazone.
from airflow import DAG
from datazone_provider.operators.datazone_run_execution import DatazoneRunExecutionOperator
from datetime import datetime
dag = DAG(
"my_dag",
start_date=datetime(2021, 1, 1),
schedule_interval=None
)
dz_task = DatazoneRunExecutionOperator(
dag=dag,
task_id="run_datazone_execution",
conn_id="<your_conn_id>",
pipeline_id="<your_pipeline_id>",
run_config={
"compute": "XSMALL",
"env_vars": {
"DEMO_KEY": "DEMO_VALUE"
}
}
)
dz_task
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for airflow_provider_datazone-1.0.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 930886a816da5ff65530de5a0a019db5a8d616e70cbdee874ae7a897efaea5e1 |
|
MD5 | 60354da302bd5838649e0e11f2e8917d |
|
BLAKE2b-256 | 5e7e00b37b8205391a20cb0c009328eab58346cc0b69b016b46bbf2cd48dd948 |
Close
Hashes for airflow_provider_datazone-1.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 308676c5923d34ae884db491552fc0c1a37c9d387b8f78225be6418948186532 |
|
MD5 | 57aff6994d8e78ef1ea834ce76ce3521 |
|
BLAKE2b-256 | da0a329ba04c854f19f656a878616bd7d745ebb1220346786c5c8b7705a42966 |