Skip to main content

Machine Learning Orchestration

Project description

Dbnd Airflow Export plugin

The plugin

dbnd-airflow-export-plugin is a plugin for Airflow system, enables you to fetch data from Airflow database and DAG folder. This Airflow side module is one of two components allows you to observe your Airflow data with Databand system.

What does it do?

The plugin exposes a REST Api within GET /export_data which, expects since (utc). This api returns json with all the relevant information scraped from airflow system.

Configuring Airflow instance

To improve performance and reduce the runtime overhead on airflow database, we advise to add an index on end_date column to task_instance and dag_run tables i.e.

   CREATE INDEX dbnd_ti_end_date_ind ON task_instance (end_date);
   CREATE INDEX dbnd_dr_end_date_ind ON dag_run (end_date);

Installation

In order to install dbnd-airflow-export-plugin we are using Airflow plugin system.

pip install databand[airflow-export]

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dbnd-airflow-export-0.48.1.tar.gz (16.8 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

dbnd_airflow_export-0.48.1-py2.py3-none-any.whl (19.8 kB view details)

Uploaded Python 2Python 3

File details

Details for the file dbnd-airflow-export-0.48.1.tar.gz.

File metadata

  • Download URL: dbnd-airflow-export-0.48.1.tar.gz
  • Upload date:
  • Size: 16.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.6.4 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.1 CPython/3.6.14

File hashes

Hashes for dbnd-airflow-export-0.48.1.tar.gz
Algorithm Hash digest
SHA256 6211188175209c2e42e966add33b41f716f09cd42b1aec72fc2891b658f2d8d6
MD5 73cdbea1487d2f0a6f678a63bdc291f9
BLAKE2b-256 ecdb73f4663d585e8e20f48704b57a5634659b57ba1545651f4379e060fdd0b4

See more details on using hashes here.

File details

Details for the file dbnd_airflow_export-0.48.1-py2.py3-none-any.whl.

File metadata

  • Download URL: dbnd_airflow_export-0.48.1-py2.py3-none-any.whl
  • Upload date:
  • Size: 19.8 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.6.4 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.1 CPython/3.6.14

File hashes

Hashes for dbnd_airflow_export-0.48.1-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 d98ffe0e9183d18ab1608a5ae993a0b271312cf60d135acb0f296e99545d25df
MD5 b923783abe7b5e8d3f0f48695f80da2d
BLAKE2b-256 297752c09f58e9a769a5fda452dba43cd940fd2bed5e21dd74024222054f24d8

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page