Skip to main content

An Airflow plugin for interacting with IOMETE platform.

Project description

IOMETE Airflow Plugin

This Plugin helps to trigger/run Spark Jobs created in IOMETE platform.

Installation

This plugin could be easily installed with pip. Just run the following in your aiflow server:

pip install iomete-airflow-plugin

Restart you server and check the plugins page on Airflow's admin panel to make sure plugin loaded successfully.

Configuration

You need to add iomete_access_token and iomete_host keys to airflow's Variables page.
Check our documentation page on how to generate personal access token.

Usage

Here are sample DAG:

from airflow import DAG
from iomete_airflow_plugin.iomete_operator import IometeOperator

dag = DAG(dag_id="...", default_args={}, schedule_interval=None)

task = IometeOperator(
    task_id="random_task_id",
    job_id="1b0fc29b-5491-4c0a-94ea-48e304c3c72e", # Spark Job ID or Name in IOMETE platform.
    dag=dag,
)

Resources

For more information check:

  1. Github repository
  2. IOMETE Docs

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

iomete_airflow_plugin-1.0.2.tar.gz (4.4 kB view details)

Uploaded Source

Built Distribution

iomete_airflow_plugin-1.0.2-py3-none-any.whl (5.4 kB view details)

Uploaded Python 3

File details

Details for the file iomete_airflow_plugin-1.0.2.tar.gz.

File metadata

  • Download URL: iomete_airflow_plugin-1.0.2.tar.gz
  • Upload date:
  • Size: 4.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.14.0 pkginfo/1.9.6 requests/2.28.2 setuptools/67.0.0 requests-toolbelt/0.10.1 tqdm/4.66.1 CPython/3.9.6

File hashes

Hashes for iomete_airflow_plugin-1.0.2.tar.gz
Algorithm Hash digest
SHA256 32db38442a9ff72046edb6b3c970ec8c73da7502214a5a4225af7302bf621e51
MD5 48f5b94210b962f3381747ea4ae2fff5
BLAKE2b-256 8cc144bba89f277eb3b061277b372dde49464b709469096f1a6085eca90842d4

See more details on using hashes here.

File details

Details for the file iomete_airflow_plugin-1.0.2-py3-none-any.whl.

File metadata

  • Download URL: iomete_airflow_plugin-1.0.2-py3-none-any.whl
  • Upload date:
  • Size: 5.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.14.0 pkginfo/1.9.6 requests/2.28.2 setuptools/67.0.0 requests-toolbelt/0.10.1 tqdm/4.66.1 CPython/3.9.6

File hashes

Hashes for iomete_airflow_plugin-1.0.2-py3-none-any.whl
Algorithm Hash digest
SHA256 214be3db1037b9af81c02a940f32a9e72518aa32ba29d7b5b34df5791a918c6a
MD5 01da7a73c424f31d621ebb5e2571a631
BLAKE2b-256 1034b87eb4b980093ac41491b557e648a931424883b2c052d9097e6b3438576d

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page