apache-airflow-providers-dpone

The formal Apache Airflow provider facade for dpone self-service DAGs.

Install

Use a supported Airflow matrix cell. This example uses Python 3.12, Airflow 3.2.0, and the CNCF Kubernetes provider 10.14.0. Use Apache Airflow's official constraints for the Airflow install only, then pin Airflow again while adding this provider:

AIRFLOW_VERSION=3.2.0
PYTHON_VERSION=3.12
AIRFLOW_CONSTRAINTS_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"

python -m pip install \
  "apache-airflow[cncf.kubernetes]==${AIRFLOW_VERSION}" \
  --constraint "${AIRFLOW_CONSTRAINTS_URL}"
python -m pip install \
  "apache-airflow==${AIRFLOW_VERSION}" \
  "apache-airflow-providers-cncf-kubernetes==10.14.0" \
  "apache-airflow-providers-dpone==0.73.25"
python -m pip check
python -c "from airflow.providers.dpone import load_dpone_dags; print(load_dpone_dags.__module__)"

The provider installs dpone-airflow-pack==0.73.25; it does not require the full dpone runtime or connector drivers in the scheduler image.

Load DAGs

from airflow.providers.dpone import load_dpone_dags, write_dpone_loader_ack

index_path = "/opt/airflow/dags/.dpone-cache/current/airflow-index.json"
load_report = load_dpone_dags(
    globals(),
    index_path=index_path,
)
write_dpone_loader_ack(
    load_report,
    index_path=index_path,
    ack_path="/opt/airflow/dags/.dpone-cache/status/loader-ack.json",
)
if load_report.fatal:
    error_code = (
        load_report.errors[0].get("code", "DPONE_AIRFLOW_INDEX_INVALID")
        if load_report.errors
        else "DPONE_AIRFLOW_INDEX_INVALID"
    )
    raise RuntimeError(
        f"{error_code}: dpone Airflow deployment index could not be loaded"
    )

write_dpone_loader_ack(...) performs bounded local filesystem I/O only. It atomically binds the completed parse report to the exact deployment and index digest, allowing an external controller to prove that visible DAG metadata is fresh. Keep the acknowledgement under the cache status/ directory.

The distribution owns Airflow provider discovery and the typed canonical namespace. The dependency-light dpone-airflow-pack package owns static pack reading and DAG construction. Importing the provider performs no remote cache, Airflow metadata, Connection, Variable, Vault, Kubernetes, or database I/O.

The deployment index read is bounded to 8 MiB and each listed artifact to 64 MiB by default. An index-level error returns fatal=True; do not represent that as a successful empty parse. After a trusted index is loaded, DAG spec and workload-pack size/SHA-256 failures are isolated to the affected DAG under the default skip_and_report policy. Recover by rematerializing the immutable release/deployment from trusted storage, never by editing cache artifacts in place.

DponeDag.from_spec(...) and DponeTaskGroup.from_pack(...) are typed escape hatches for hybrid DAGs. The recommended path is load_dpone_dags(...).

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.80.0)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for paulkov007 from gravatar.com paulkov007

Credits

Author/Maintainer: PaulKov

License expression

Apache-2.0
View SPDX License List

Requires

Python <3.13, >=3.11

Tags

airflow provider dpone gitops data-engineering

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

apache_airflow_providers_dpone-0.73.25.tar.gz (3.3 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

apache_airflow_providers_dpone-0.73.25-py3-none-any.whl (5.0 kB view details)

Uploaded Python 3

File details

Details for the file apache_airflow_providers_dpone-0.73.25.tar.gz.

File metadata

File hashes

Hashes for apache_airflow_providers_dpone-0.73.25.tar.gz
Algorithm Hash digest
SHA256 f0234684b0d3332d7691c27a2075a649ada04cb911417bb56a65270e8a60b8e3
MD5 61e5e87f98395e286e9fca524567390f
BLAKE2b-256 87c7aa068fb24d4cea19574fe1e13db50d00b7838d92186af1ce03ecf1bcdf6e

See more details on using hashes here.

File details

Details for the file apache_airflow_providers_dpone-0.73.25-py3-none-any.whl.

File metadata

File hashes

Hashes for apache_airflow_providers_dpone-0.73.25-py3-none-any.whl
Algorithm Hash digest
SHA256 4c98ef3fc4811e3037353b31770db5b7316c1da4575e9ff0b33f33d7894f7237
MD5 6e7408b2f7f02317f718f7b06d39a9d0
BLAKE2b-256 e03d5b9e98b4265757d0ae12b91595a4d47ba38134d20aaf94143946e4016134

See more details on using hashes here.

Release history Release notifications | RSS feed

0.80.0

2 files

0.79.2

2 files

0.79.1

2 files

0.79.0

2 files

0.78.0

2 files

0.77.1

2 files

0.77.0

2 files

0.76.0

2 files

0.75.0

2 files

0.74.36

2 files

0.74.35

2 files

0.74.34

2 files

0.74.33

2 files

0.74.32

2 files

0.74.31

2 files

0.74.30

2 files

0.74.29

2 files

0.74.28

2 files

0.74.27

2 files

0.74.26

2 files

0.74.25

2 files

0.74.24

2 files

0.74.23

2 files

0.74.22

2 files

0.74.20

2 files

0.74.19

2 files

0.74.17

2 files

0.74.16

2 files

0.74.15

2 files

0.74.14

2 files

0.74.13

2 files

0.74.12

2 files

0.74.11

2 files

0.74.10

2 files

0.74.9

2 files

0.74.8

2 files

0.74.7

2 files

0.74.6

2 files

0.74.5

2 files

0.74.4

2 files

0.74.3

2 files

0.74.2

2 files

0.74.0

2 files

0.73.42

2 files

0.73.38

2 files

0.73.36

2 files

0.73.34

2 files

0.73.31

2 files

0.73.30

2 files

0.73.29

2 files

0.73.28

2 files

This release

0.73.25 This release

2 files

0.73.24

2 files

0.73.23

2 files

0.73.22

2 files

0.73.21

2 files

0.73.20

2 files

0.73.19

2 files

0.73.17

2 files

0.73.16

2 files

0.73.15

2 files

0.73.14

2 files

0.73.13

2 files

0.73.10

2 files

0.73.9

2 files

0.73.6

2 files

0.73.2

2 files

0.73.1

2 files

0.73.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page