Skip to main content

Provider package airflow-provider-sifflet for Apache Airflow

Project description

Sifflet Provider for Apache Airflow

This package provides operators and hook that integrate Sifflet into Apache Airflow. All classes for this provider package are in the sifflet_provider Python package.

Installation

You can install this package on top of an existing Airflow 2.1+ installation

pip install airflow-provider-sifflet

The package supports the following python versions: 3.7, 3.8, 3.9, 3.10

Configuration

In the Airflow user interface, you can configure a Connection for Sifflet in Admin -> Connections -> Add a new record.

You will need to fill out the following:

Connection Id: sifflet_default
Connection Type: Sifflet
Sifflet Tenant: <your_tenant_name>
Sifflet Token: <your_sifflet_access_token>

<your_tenant_name>: if you access to Sifflet with "https://abcdef.siffletdata.com", then your tenant would be abcdef <your_sifflet_access_token>: you can find more information on how to generate it here

Modules

Operators

SiffletDbtIngestOperator

SiffletDbtIngestOperator sends your DBT artifacts to the Sifflet application.

Example usage:

from sifflet_provider.operators.dbt import SiffletDbtIngestOperator

sifflet_dbt_ingest = SiffletDbtIngestOperator(
    task_id="sifflet_dbt_ingest",
    input_folder="<path to dbt project folder>",
    target="prod",
    project_name="<dbt project name>",
)

SiffletRunRuleOperator

SiffletRunRuleOperator Run one or several Sifflet rules - requires rule id(s).

Example usage:

from sifflet_provider.operators.rule import SiffletRunRuleOperator

sifflet_run_rule = SiffletRunRuleOperator(
    task_id="sifflet_run_rule",
    rule_ids=[
        "3e2e2687-cd20-11ec-b38b-06bb20181849",
        "3e19eb3e-cd20-11ec-b38b-06bb20181849",
        "3e1a86f1-cd20-11ec-b38b-06bb20181849",
        "3e2e1fc3-cd20-11ec-b38b-06bb20181849",
    ],
    error_on_rule_fail=True
)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

airflow-provider-sifflet-1.0.2.tar.gz (4.7 kB view hashes)

Uploaded Source

Built Distribution

airflow_provider_sifflet-1.0.2-py3-none-any.whl (6.0 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page