iomete-airflow-plugin 0.1.0

An Airflow plugin for interacting with IOMETE platform.

IOMETE Airflow Plugin

This Plugin helps to trigger/run Spark Jobs created in IOMETE platform.

Installation

This plugin could be easily installed with pip. Just run the following in your aiflow server:

pip install iomete-airflow-plugin

Restart you server and check the plugins page on Airflow's admin panel to make sure plugin loaded successfully.

Configuration

You need to add iomete_access_token key to airflow's Variables module.
Check our documentation page on how to generate personal access token.

Usage

Here are sample DAG:

from airflow import DAG
from iomete_airflow_plugin.iomete_operator import IometeOperator

dag = DAG(dag_id="...", default_args={}, schedule_interval=None)

task = IometeOperator(
    task_id="random_task_id",
    job_id="1b0fc29b-5491-4c0a-94ea-48e304c3c72e", # Spark Job ID or Name in IOMETE platform.
    workspace_id="pceh7-816", # Your workspace ID in IOMETE platform
    dag=dag,
)

Resources

For more information check:

1. Github repository
2. IOMETE Docs