Skip to main content

Publish Airflow notification errors to SNS Topic

Project description

Python 3.7 PyPI version License: MIT Twitter Follow

Publish Airflow Notification to a SNS Topic

This package adds a callback function to use in failures to DAGs and Tasks in a Airflow project.

Installation

pip install airflow-notify-sns

Usage

from datetime import timedelta

# Airflow native imports to create a DAG
from airflow import DAG, utils
from airflow.operators.bash_operator import BashOperator

# Here is function import
from airflow_notify_sns import airflow_notify_sns

# Dag Definition
dag = DAG(
    dag_id='test_dag',
    default_args={
        'owner': 'airflow',
        'depends_on_past': False,
        'start_date': utils.dates.days_ago(1),
        'retries': 3,
        'retry_delay': timedelta(minutes=5)
    },
    schedule_interval="@daily",
    dagrun_timeout=timedelta(minutes=60),
    sla_miss_callback=airflow_notify_sns,
    on_failure_callback=airflow_notify_sns
)

# Add your tasks here
t = BashOperator(
    dag=dag,
    task_id='test_env',
    bash_command='/tmp/test.sh',
    env={'EXECUTION_DATE': '{{ ds }}'},
    on_failure_callback=airflow_notify_sns
)

When DAG or tasks ends in error, a notification will be send to a SNS Topic using AWS default connection (aws_default).

Required Variable

This module will try to find a variable named airflow_notify_sns_arn in your Airflow environment, containing SNS Topic ARN where message will be published to.

If variable is not found, function will abort execution with no error.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

airflow-notify-sns-0.0.2.tar.gz (2.9 kB view hashes)

Uploaded source

Built Distribution

airflow_notify_sns-0.0.2-py3-none-any.whl (4.1 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page