airflow-provider-sifflet 1.0.2

Provider package airflow-provider-sifflet for Apache Airflow

Sifflet Provider for Apache Airflow

This package provides operators and hook that integrate Sifflet into Apache Airflow. All classes for this provider package are in the sifflet_provider Python package.

Installation

You can install this package on top of an existing Airflow 2.1+ installation

pip install airflow-provider-sifflet

The package supports the following python versions: 3.7, 3.8, 3.9, 3.10

Configuration

In the Airflow user interface, you can configure a Connection for Sifflet in Admin -> Connections -> Add a new record.

You will need to fill out the following:

Connection Id: sifflet_default
Connection Type: Sifflet
Sifflet Tenant: <your_tenant_name>
Sifflet Token: <your_sifflet_access_token>

<your_tenant_name>: if you access to Sifflet with "https://abcdef.siffletdata.com", then your tenant would be abcdef <your_sifflet_access_token>: you can find more information on how to generate it here

Modules

Operators

SiffletDbtIngestOperator

SiffletDbtIngestOperator sends your DBT artifacts to the Sifflet application.

Example usage:

from sifflet_provider.operators.dbt import SiffletDbtIngestOperator

sifflet_dbt_ingest = SiffletDbtIngestOperator(
    task_id="sifflet_dbt_ingest",
    input_folder="<path to dbt project folder>",
    target="prod",
    project_name="<dbt project name>",
)

SiffletRunRuleOperator

SiffletRunRuleOperator Run one or several Sifflet rules - requires rule id(s).

Example usage:

from sifflet_provider.operators.rule import SiffletRunRuleOperator

sifflet_run_rule = SiffletRunRuleOperator(
    task_id="sifflet_run_rule",
    rule_ids=[
        "3e2e2687-cd20-11ec-b38b-06bb20181849",
        "3e19eb3e-cd20-11ec-b38b-06bb20181849",
        "3e1a86f1-cd20-11ec-b38b-06bb20181849",
        "3e2e1fc3-cd20-11ec-b38b-06bb20181849",
    ],
    error_on_rule_fail=True
)