Skip to main content

Machine Learning Orchestration

Project description

Dbnd Airflow Export plugin

The plugin

dbnd-airflow-export-plugin is a plugin for Airflow system, enables you to fetch data from Airflow database and DAG folder. This Airflow side module is one of two components allows you to observe your Airflow data with Databand system.

What does it do?

The plugin exposes a REST Api within GET /export_data which, expects since (utc). This api returns json with all the relevant information scraped from airflow system.

Configuring Airflow instance

To improve performance and reduce the runtime overhead on airflow database, we advise to add an index on end_date column to task_instance and dag_run tables i.e.

   CREATE INDEX dbnd_ti_end_date_ind ON task_instance (end_date);
   CREATE INDEX dbnd_dr_end_date_ind ON dag_run (end_date);

Installation

In order to install dbnd-airflow-export-plugin we are using Airflow plugin system.

pip install databand[airflow-export]

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dbnd-airflow-export-0.48.3.tar.gz (16.8 kB view details)

Uploaded Source

Built Distribution

dbnd_airflow_export-0.48.3-py2.py3-none-any.whl (19.8 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file dbnd-airflow-export-0.48.3.tar.gz.

File metadata

  • Download URL: dbnd-airflow-export-0.48.3.tar.gz
  • Upload date:
  • Size: 16.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.6.4 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.1 CPython/3.6.14

File hashes

Hashes for dbnd-airflow-export-0.48.3.tar.gz
Algorithm Hash digest
SHA256 1af1fcbb39f0c3bb68644d2b0d0d24986dff4b76e113575c691c13e003ae8a9c
MD5 d37c70f9a8a9a98b6fa722f36c2ad59f
BLAKE2b-256 3fe907f3757ea92e254aacd1ae1dd0342c65ce490995bf3943f2175491515fcd

See more details on using hashes here.

File details

Details for the file dbnd_airflow_export-0.48.3-py2.py3-none-any.whl.

File metadata

  • Download URL: dbnd_airflow_export-0.48.3-py2.py3-none-any.whl
  • Upload date:
  • Size: 19.8 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.6.4 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.1 CPython/3.6.14

File hashes

Hashes for dbnd_airflow_export-0.48.3-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 624faec22c557d89ee4497fe23cf8271752fdd3463ca9ec0a5d8bb8080aa8ce1
MD5 de4e1638a7869d0b01c952b523fbeaed
BLAKE2b-256 5e77a4596b0bfc2a180ceb1e87dd3bdbb5e41db5ef94d5ac52cbe066a9314f46

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page