Skip to main content

Machine Learning Orchestration

Project description

Dbnd Airflow Export plugin

The plugin

dbnd-airflow-export-plugin is a plugin for Airflow system, enables you to fetch data from Airflow database and DAG folder. This Airflow side module is one of two components allows you to observe your Airflow data with Databand system.

What does it do?

The plugin exposes a REST Api within GET /export_data which, expects since (utc). This api returns json with all the relevant information scraped from airflow system.

Configuring Airflow instance

To improve performance and reduce the runtime overhead on airflow database, we advise to add an index on end_date column to task_instance and dag_run tables i.e.

   CREATE INDEX dbnd_ti_end_date_ind ON task_instance (end_date);
   CREATE INDEX dbnd_dr_end_date_ind ON dag_run (end_date);

Installation

In order to install dbnd-airflow-export-plugin we are using Airflow plugin system.

pip install databand[airflow-export]

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dbnd-airflow-export-0.39.3.tar.gz (20.2 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

dbnd_airflow_export-0.39.3-py2.py3-none-any.whl (25.1 kB view details)

Uploaded Python 2Python 3

File details

Details for the file dbnd-airflow-export-0.39.3.tar.gz.

File metadata

  • Download URL: dbnd-airflow-export-0.39.3.tar.gz
  • Upload date:
  • Size: 20.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.6.13

File hashes

Hashes for dbnd-airflow-export-0.39.3.tar.gz
Algorithm Hash digest
SHA256 e8784f7f2785f1ff766cf20b08a153d0580494adabc50ef506c96ce3e58cee68
MD5 5fa35dfefd528926f9087e6e790dd839
BLAKE2b-256 9f41a3bf4c8c55070c21dd2411d63d8f09ecbc1b84e615c12b25325243851c29

See more details on using hashes here.

File details

Details for the file dbnd_airflow_export-0.39.3-py2.py3-none-any.whl.

File metadata

  • Download URL: dbnd_airflow_export-0.39.3-py2.py3-none-any.whl
  • Upload date:
  • Size: 25.1 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.6.13

File hashes

Hashes for dbnd_airflow_export-0.39.3-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 890ab66156773f606d61f214d7928494d36421a6b155fc83d222610e023c1cfd
MD5 288464764e8eaed543e386f024b7dba7
BLAKE2b-256 f4cad34a02eea35df84538b08a9ade87971f4f862d2dfe6aef46b4013c70e19d

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page