An Airflow plugin for interacting with IOMETE platform.
Project description
IOMETE Airflow Plugin
This Plugin helps to trigger/run Spark Jobs created in IOMETE platform.
Installation
This plugin could be easily installed with pip
. Just run the following in your aiflow server:
pip install iomete-airflow-plugin
Restart you server and check the plugins page on Airflow's admin panel to make sure plugin loaded successfully.
Configuration
You need to add iomete_access_token
and iomete_host
keys to airflow's Variables page.
Check our documentation page on how to generate personal access token.
Usage
Here are sample DAG:
from airflow import DAG
from iomete_airflow_plugin.iomete_operator import IometeOperator
dag = DAG(dag_id="...", default_args={}, schedule_interval=None)
task = IometeOperator(
task_id="random_task_id",
job_id="1b0fc29b-5491-4c0a-94ea-48e304c3c72e", # Spark Job ID or Name in IOMETE platform.
dag=dag,
)
Resources
For more information check:
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file iomete_airflow_plugin-1.0.2.tar.gz
.
File metadata
- Download URL: iomete_airflow_plugin-1.0.2.tar.gz
- Upload date:
- Size: 4.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.14.0 pkginfo/1.9.6 requests/2.28.2 setuptools/67.0.0 requests-toolbelt/0.10.1 tqdm/4.66.1 CPython/3.9.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 32db38442a9ff72046edb6b3c970ec8c73da7502214a5a4225af7302bf621e51 |
|
MD5 | 48f5b94210b962f3381747ea4ae2fff5 |
|
BLAKE2b-256 | 8cc144bba89f277eb3b061277b372dde49464b709469096f1a6085eca90842d4 |
File details
Details for the file iomete_airflow_plugin-1.0.2-py3-none-any.whl
.
File metadata
- Download URL: iomete_airflow_plugin-1.0.2-py3-none-any.whl
- Upload date:
- Size: 5.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.14.0 pkginfo/1.9.6 requests/2.28.2 setuptools/67.0.0 requests-toolbelt/0.10.1 tqdm/4.66.1 CPython/3.9.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 214be3db1037b9af81c02a940f32a9e72518aa32ba29d7b5b34df5791a918c6a |
|
MD5 | 01da7a73c424f31d621ebb5e2571a631 |
|
BLAKE2b-256 | 1034b87eb4b980093ac41491b557e648a931424883b2c052d9097e6b3438576d |