Airflow plugin to export dag and task based metrics to Prometheus.
Project description
Airflow prometheus exporter
Exposes dag and task based metrics from Airflow to a Prometheus compatible endpoint.
Discussion
You can ask questions in Gitter channel: https://gitter.im/epoch8/airflow-exporter
Screenshots
Compatibility
Note: this version is compatible with Airflow 1.10.3+ only, see #46 for details
For compatibility with previous versions of Airflow use older version: v0.5.4
- Airflow: airflow1.10.3+
- Python: python2, python3
- DataBase: postgresql, mysql
Install
pip install airflow-exporter
That's it. You're done.
Exporting extra labels to Prometheus
It is possible to add extra labels to DAG-related metrics by providing labels
dict to DAG params
.
Example
dag = DAG(
'dummy_dag',
schedule_interval=timedelta(hours=5),
default_args=default_args,
catchup=False,
params={
'labels': {
'env': 'test'
}
}
)
Label env
with value test
will be added to all metrics related to dummy_dag
:
airflow_dag_status{dag_id="dummy_dag",env="test",owner="owner",status="running"} 12.0
Metrics
Metrics will be available at
http://<your_airflow_host_and_port>/admin/metrics/
airflow_task_status
Labels:
dag_id
task_id
owner
status
Value: number of tasks in specific status.
airflow_dag_status
Labels:
dag_id
owner
status
Value: number of dags in specific status.
airflow_dag_run_duration
Labels:
dag_id
: unique identifier for a given DAG
Value: duration in seconds of the longest DAG Run for given DAG. This metric is not available for DAGs that have already completed.
License
Distributed under the BSD license. See LICENSE for more information.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for airflow_exporter-1.2.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 850dd82bb3ce5ec8372e0fe9610b84b114fa8aa56d458bdcf89f556d0b43f670 |
|
MD5 | ef9dbd6c4165fdbe46bff9c46b1f57cb |
|
BLAKE2b-256 | 22ce7db326d25d49d907e01a0bcc4b803eafcb7875ce72d348617721a06e21cc |
Hashes for airflow_exporter-1.2.0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 856241ff8a7abc49a10cb9483f4431ba8bb1a2a1ed7c9407c083208fd81cc017 |
|
MD5 | 3af189b4c3cc2485b2e4e2dc763ec568 |
|
BLAKE2b-256 | 600eae96a03ec748b8ea5d6065003ac4b81322bcff73475630732d22ffd905a9 |