Airflow package provider to reload apps/task/automation from Qlik Sense Cloud.
Project description
Airflow: Qlik Sense Cloud Provider
Qlik Sense Cloud Provider to reload application, automation from Airflow.
This repository provides basic qlik sense cloud hook and operators to trigger reloads of applications, automations, or tasks available in a Qlik Sense Cloud tenant.
Road Map
☑ Adding Report Operator (Added in beta test)
Requirements
The package has been tested with Python 3.7, Python 3.8.
Package | Version |
---|---|
apache-airflow | >2.0 |
qlik-sdk | >= 0.14.0 |
You also need a Qlik Sense Cloud tenant with API key activated. To get more informations about how to activate API key on Qlik Sense Cloud Tenant, see this section.
How to install it ?
To install it, download and unzip source and launch the following pip install command:
By using Pypi
pip install airflow-provider-qlik-sense-cloud
By Local Install
pip install .
You can also use
python setup.py install
How to use it ?
To create connection, you need to get a API Token and your Qlik Cloud Tenant Activation
1. Generating API Keys
Follow these steps to get API Keys:
Step 1: Login to your Qlik Sense Cloud Tenant.
Step 2: Click on your account logo and go into Profile.
Step 3: Go into API keys section.
Step 4: Generate a new key.
Step 5: Give a description and an expiration date ( ⚠️⚠️⚠️ Choose an expiration date with enough delay to avoid refresh all the time the token API ⚠️⚠️⚠️).
Step 6: The API key is displaying, copy it and save it.
2. Get tenant id
Step 1: Login to your Qlik Sense Cloud Tenant and go into About Section by clicking on your profile icon.
Step 2: Get the value of your tenant hostname
3. Creating Qlik Sense Cloud Connection in Airflow
Step 1: Login to your Airflow Webserver. Go into Admin > Connection and creation a new connection by cliking on add icon.
Step 2: Give a name to your connection id and selection Qlik Sense Cloud in Connection Type. Add tenant id hostname (without https) and Token API Key that you copy in Section 1 and 2.
4. Example: Creating a DAG with Qlik Sense Cloud Operator to reload App
You can now use the operators in your dags to trigger action in Qlik Sense Cloud from Airflow
Example:
from airflow import DAG
from airflow.providers.qlik_sense_cloud.operators.qlik_sense_cloud_reload import QlikSenseCloudReloadOperator
from airflow.utils.dates import days_ago
from datetime import timedelta
from textwrap import dedent
# [START default_args]
# These args will get passed on to each operator
# You can override them on a per-task basis during operator initialization
default_args = {
'owner': 'airflow',
'depends_on_past': False,
'email': ['airflow@example.com'],
'email_on_failure': False,
'email_on_retry': False,
'retries': 1,
'retry_delay': timedelta(minutes=5),
# 'queue': 'bash_queue',
# 'pool': 'backfill',
# 'priority_weight': 10,
# 'end_date': datetime(2016, 1, 1),
# 'wait_for_downstream': False,
# 'dag': dag,
# 'sla': timedelta(hours=2),
# 'execution_timeout': timedelta(seconds=300),
# 'on_failure_callback': some_function,
# 'on_success_callback': some_other_function,
# 'on_retry_callback': another_function,
# 'sla_miss_callback': yet_another_function,
# 'trigger_rule': 'all_success'
}
# [END default_args]
# [START instantiate_dag]
with DAG('test-airflow',default_args=default_args,description='A simple tutorial DAG to try',schedule_interval=timedelta(days=1),start_date=days_ago(2),tags=['example'],) as dag:
t1 = QlikSenseCloudReloadOperator(task_id='reload_app',appId='4d5ad6d0-92a1-47c3-b57d-5a07945377f8',conn_id='qliksensecloud')
t1
# [END instantiate_dag]
5. Example: Creating a DAG with Qlik Sense Cloud Operator to reload Qlik Automation
Here's an example of DAG using operator to reload automation
from airflow import DAG
from airflow.providers.qlik_sense_cloud.operators.qlik_sense_cloud_automation import QlikSenseCloudAutomationOperator
from airflow.providers.qlik_sense_cloud.operators.qlik_sense_cloud_report import QlikSenseCloudReportOperator
from airflow.utils.dates import days_ago
from datetime import timedelta
from textwrap import dedent
# [START default_args]
# These args will get passed on to each operator
# You can override them on a per-task basis during operator initialization
default_args = {
'owner': 'airflow',
'depends_on_past': False,
'email': ['airflow@example.com'],
'email_on_failure': False,
'email_on_retry': False,
'retries': 1,
'retry_delay': timedelta(minutes=5),
# 'queue': 'bash_queue',
# 'pool': 'backfill',
# 'priority_weight': 10,
# 'end_date': datetime(2016, 1, 1),
# 'wait_for_downstream': False,
# 'dag': dag,
# 'sla': timedelta(hours=2),
# 'execution_timeout': timedelta(seconds=300),
# 'on_failure_callback': some_function,
# 'on_success_callback': some_other_function,
# 'on_retry_callback': another_function,
# 'sla_miss_callback': yet_another_function,
# 'trigger_rule': 'all_success'
}
# [END default_args]
# [START instantiate_dag]
with DAG('test-automation-report',default_args=default_args,description='A simple tutorial DAG to try',schedule_interval=timedelta(days=1),start_date=days_ago(2),tags=['example'],) as dag:
t1 = QlikSenseCloudAutomationOperator(task_id='reload_automation',automationId='bab86470-578a-11ed-bee3-db20e15c9fd8',conn_id='qliksensecloud')
t2 = QlikSenseCloudReportOperator(task_id='reload_report',reportId='65f810a9e0f4697cb54b6e87',conn_id='qliksensecloud')
t1 >> t2
6. (Appendix) Activate API Key on Qlik Sense Cloud Tenant
To get more informations about API Key in Qlik Sense Cloud, you can follow these topic:
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Hashes for airflow-provider-qlik-sense-cloud-0.0.4.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | bc58ae1866db0a79632ed44f1a523fe4044ad2f82744a9bc0cc44825a73c05c7 |
|
MD5 | 87f99924455bbe55ee09147030c47387 |
|
BLAKE2b-256 | f07659046f56c833ad56a65f30259b2b4a44c3d6d71212be789327433a435f50 |