airflow-provider-dqlens

Apache Airflow provider for DQLens. Add auto-generated data quality checks to your DAGs with one task.

Install

pip install airflow-provider-dqlens

Usage

from dqlens_airflow import DQLensOperator

quality_check = DQLensOperator(
    task_id="dqlens_quality_check",
    conn_id="my_postgres",
    schema="public",
    focus="high",
)

load_data >> quality_check >> downstream_tasks

If DQLens finds problems, the task fails and downstream tasks don't run.

Parameters

What it does

1. Reads connection from your Airflow connection
2. Profiles all tables (null rates, uniqueness, patterns, FKs, freshness)
3. Compares against previous profile (drift detection)
4. Fails the task if findings exceed your severity threshold
5. Pushes results to XCom for downstream use

XCom output

{
    "findings_count": 3,
    "total_findings": 3,
    "passed_count": 47,
    "tables_profiled": 6,
    "findings": [
        {"table": "public.orders", "column": "email", "severity": "HIGH", "message": "..."},
    ]
}

Supported databases

PostgreSQL, DuckDB, SQLite, MySQL (via Airflow connection types).

License

MIT