Skip to main content

Machine Learning Orchestration

Project description

Dbnd Airflow Export plugin

The plugin

dbnd-airflow-export-plugin is a plugin for Airflow system, enables you to fetch data from Airflow database and DAG folder. This Airflow side module is one of two components allows you to observe your Airflow data with Databand system.

What does it do?

The plugin exposes a REST Api within GET /export_data which, expects since (utc). This api returns json with all the relevant information scraped from airflow system.

Configuring Airflow instance

To improve performance and reduce the runtime overhead on airflow database, we advise to add an index on end_date column to task_instance and dag_run tables i.e.

   CREATE INDEX dbnd_ti_end_date_ind ON task_instance (end_date);
   CREATE INDEX dbnd_dr_end_date_ind ON dag_run (end_date);

Installation

In order to install dbnd-airflow-export-plugin we are using Airflow plugin system.

pip install databand[airflow-export]

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dbnd-airflow-export-0.37.3.tar.gz (19.7 kB view details)

Uploaded Source

Built Distribution

dbnd_airflow_export-0.37.3-py2.py3-none-any.whl (24.4 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file dbnd-airflow-export-0.37.3.tar.gz.

File metadata

  • Download URL: dbnd-airflow-export-0.37.3.tar.gz
  • Upload date:
  • Size: 19.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.6.13

File hashes

Hashes for dbnd-airflow-export-0.37.3.tar.gz
Algorithm Hash digest
SHA256 941d4a5455ed6f9498a14fa59e04568440ea20732a378185fe7ff0ff4dfcb547
MD5 7fc3e7b40a6edb7970ac4e81a2bf918b
BLAKE2b-256 ce11bcb3ed0a2441d9e5dd1d7df2a234446838e1217cd3e58d4869a4dfefc618

See more details on using hashes here.

File details

Details for the file dbnd_airflow_export-0.37.3-py2.py3-none-any.whl.

File metadata

  • Download URL: dbnd_airflow_export-0.37.3-py2.py3-none-any.whl
  • Upload date:
  • Size: 24.4 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.6.13

File hashes

Hashes for dbnd_airflow_export-0.37.3-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 6f33e24912f48b4a6809726ae1ca05f227da0c2c7db6f9185594cd680a9bdbab
MD5 5cb8d18a4ebdfb3e2e7287b95d39d359
BLAKE2b-256 9dbda73a4459260c899565d044b17f6bb05beaadf0611fb13afd96fa940183ee

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page