Skip to main content

Machine Learning Orchestration

Project description

Dbnd Airflow Export plugin

The plugin

dbnd-airflow-export-plugin is a plugin for Airflow system, enables you to fetch data from Airflow database and DAG folder. This Airflow side module is one of two components allows you to observe your Airflow data with Databand system.

What does it do?

The plugin exposes a REST Api within GET /export_data which, expects since (utc). This api returns json with all the relevant information scraped from airflow system.

Configuring Airflow instance

To improve performance and reduce the runtime overhead on airflow database, we advise to add an index on end_date column to task_instance and dag_run tables i.e.

   CREATE INDEX dbnd_ti_end_date_ind ON task_instance (end_date);
   CREATE INDEX dbnd_dr_end_date_ind ON dag_run (end_date);

Installation

In order to install dbnd-airflow-export-plugin we are using Airflow plugin system.

pip install databand[airflow-export]

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dbnd-airflow-export-0.46.5.tar.gz (20.9 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

dbnd_airflow_export-0.46.5-py2.py3-none-any.whl (26.7 kB view details)

Uploaded Python 2Python 3

File details

Details for the file dbnd-airflow-export-0.46.5.tar.gz.

File metadata

  • Download URL: dbnd-airflow-export-0.46.5.tar.gz
  • Upload date:
  • Size: 20.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.5.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.0 CPython/3.6.13

File hashes

Hashes for dbnd-airflow-export-0.46.5.tar.gz
Algorithm Hash digest
SHA256 fea390f7196ad5b5a4d543279e4bf8d00efcbd356a96e4c228ad47064d05148a
MD5 7fa76d66b32d9c7cb9d635f29718947c
BLAKE2b-256 e870a16a7d9349745f07a88e85deb0f673f0cbc8a299ca4df3e9bdeeb41a0d43

See more details on using hashes here.

File details

Details for the file dbnd_airflow_export-0.46.5-py2.py3-none-any.whl.

File metadata

  • Download URL: dbnd_airflow_export-0.46.5-py2.py3-none-any.whl
  • Upload date:
  • Size: 26.7 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.5.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.0 CPython/3.6.13

File hashes

Hashes for dbnd_airflow_export-0.46.5-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 09e7b9f065ff298ea0fe76db4137450a5fb7b383c5adcb0b4257553c149d0abf
MD5 c6cf6db55be5cc5f0907693152f30f26
BLAKE2b-256 10b43b0c48b90db3c3c0abdf1d7f67112931c666e4628c7d91cd81a8c11eecb7

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page