Apache Airflow provider for KDBAirflowOperator
Project description
Airflow KDB Provider
A lightweight KDB provider for Apache Airflow, featuring the KDBAirflowOperator
. This provider allows for seamless integration between Airflow and KDB+/q, making it easier to automate data pipelines that involve KDB+/q.
Installation
You can install the Airflow KDB Provider package from PyPI using the following command:
pip install airflow-kdb-provider
Usage
To use the KDBAirflowOperator in your Airflow DAG, you must first import it and create an instance of the operator. Here is an example:
from airflow_kdb_provider.operators.kdb_operator import KDBOperator
kdb_operator = KDBOperator(
task_id='run_kdb_script',
command='/path/to/kdb_script.q',
params={'param1': 'value1', 'param2': 'value2'},
conn_id='kdb_conn',
dag=dag)
In this example, we create an instance of the KDBOperator and specify the following parameters:
task_id: the task ID for this operator command: the path to the KDB+/q script that we want to execute params: a dictionary of parameters that will be passed to the KDB+/q script as command-line arguments conn_id: the connection ID for the KDB+/q server that we want to use (this should be defined in Airflow's Connections interface) dag: the DAG that this operator belongs to Once you have created an instance of the KDBOperator, you can add it to your DAG like any other Airflow operator:
some_other_operator >> kdb_operator >> some_other_operator2
In this example, we have added the kdb_operator to our DAG and specified that it should be executed after some_other_operator and before some_other_operator2.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for airflow-kdb-provider-0.1.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 892c6f7a9f8b0b018fcc6be7cc4062495e6f7e796a0380ee76c6566f5e6f5ef5 |
|
MD5 | 5ddcd73a093749ea3361f01be18950c8 |
|
BLAKE2b-256 | 2a717adaf91d862d7ad458ca8b298e95436e41e4b05c0cbe1ad2968eebb1957e |
Hashes for airflow_kdb_provider-0.1.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b312b73eb985c06ba9f526e5dd6c59964285ef75049ceab820f0eb20e3066c58 |
|
MD5 | 20f4499d4e591b9bc21892cc2029d92d |
|
BLAKE2b-256 | df7cdd45eb5f1b65a7eb9e2df324f58329a66a391eadf0aa2bdc7b698c1dd800 |