Machine Learning Orchestration
Project description
Dbnd Airflow Export plugin
The plugin
dbnd-airflow-export-plugin
is a plugin for Airflow system, enables you to fetch data from Airflow database and DAG folder.
This Airflow side module is one of two components allows you to observe your Airflow data with Databand
system.
What does it do?
The plugin exposes a REST Api within GET
/export_data
which, expects since
(utc).
This api returns json with all the relevant information scraped from airflow system.
Configuring Airflow instance
To improve performance and reduce the runtime overhead on airflow database, we advise to add an index on end_date column to task_instance and dag_run tables i.e.
CREATE INDEX dbnd_ti_end_date_ind ON task_instance (end_date);
CREATE INDEX dbnd_dr_end_date_ind ON dag_run (end_date);
Installation
In order to install dbnd-airflow-export-plugin
we are using Airflow plugin system.
pip install databand[airflow-export]
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for dbnd-airflow-export-0.36.4.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | c4e4d8cd05167ac1a1ab1bd9ef4281576213d179ec2700c43486e5dd98ef2117 |
|
MD5 | 5ec9f436c27ad600c1e05dbf8d37e172 |
|
BLAKE2b-256 | 3e4f2ce762157b8a7ae1d3ae7aef9316dc5b6f7c223dc1d55c48801fd4b91256 |
Hashes for dbnd_airflow_export-0.36.4-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ef5a30e2c999489e77ef5d0f339ec35938a94f9c4c2362db4ddda55b8e345f17 |
|
MD5 | b116f63316fa1c07f33b575a50f9f93a |
|
BLAKE2b-256 | 5c3546a2440a6c6ad32f5553b75d69ac64d683cf93ff9e5f20594f2a93f3f969 |