dpone-airflow-pack

dpone-airflow-pack is the lightweight, Airflow-independent reader and DAG construction library for dpone GitOps packs. The formal apache-airflow-providers-dpone distribution owns provider discovery and the canonical airflow.providers.dpone namespace.

It only reads a static airflow-pack.json and builds visible Airflow/Kubernetes tasks. It does not import the full dpone runtime and intentionally contains no source/sink/native transfer dependencies such as ClickHouse, MSSQL, pyodbc, pandas, polars, or ConnectorX.

Recommended zero-boilerplate Airflow DAG import:

from airflow.providers.dpone import load_dpone_dags

load_report = load_dpone_dags(
    globals(),
    index_path="/opt/airflow/dags/.dpone-cache/current/airflow-index.json",
)
if load_report.fatal:
    error_code = (
        load_report.errors[0].get("code", "DPONE_AIRFLOW_INDEX_INVALID")
        if load_report.errors
        else "DPONE_AIRFLOW_INDEX_INVALID"
    )
    raise RuntimeError(
        f"{error_code}: dpone Airflow deployment index could not be loaded"
    )

The formal provider's canonical namespace is PEP 561 typed. DponeDag.from_spec(...) and DponeTaskGroup.from_pack(...) remain typed escape hatches for hybrid DAGs. The old dpone_airflow_pack imports remain compatibility shims and emit one deprecation warning per process for provider-facade objects.

For the complete import, generated-loader, and cache-layout cutover, follow the provider and cache migration guide.

Legacy installations that still consume a mutable remote latest/pack-index.json may keep the compatibility sync temporarily:

dpone-airflow-pack-sync \
  --once \
  --index-uri s3://bucket/dpone-artifacts/prod/repo/latest/pack-index.json \
  --reader-connection-id s3_dpone_artifacts_reader \
  --cache-dir /opt/airflow/dags/.dpone-cache/airflow

dpone-airflow-pack-cache-status \
  --cache-dir /opt/airflow/dags/.dpone-cache/airflow \
  --json

The loader reads only the local composite deployment index and its listed checksum-verified DAG specs/packs. It does not scan authoring sources, refresh a remote cache, or read Airflow Variables, Connections, Vault, Kubernetes, or a metadata database during parse.

For DAGs that should run a whole GitOps domain or a named workflow group, keep the grouping in the domain catalog and resolve it through the lightweight provider:

workflow_groups:
  daily:
    workload_ids:
      - dim_customer
      - fact_order

from dpone_airflow_pack import workload_ids_from_gitops_domain

workload_ids = workload_ids_from_gitops_domain(
    repo_root,
    "sales",
    group="daily",
)

The resolver only reads YAML files and validates that every group member exists in the same domain catalog.

The full dpone[full,accel] package belongs in the KPO runtime image, not in the scheduler image.

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.74.33)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for paulkov007 from gravatar.com paulkov007

Credits

Author/Maintainer: PaulKov

License expression

Apache-2.0
View SPDX License List

Requires

Python <3.13, >=3.11

Tags

airflow gitops data-engineering scheduler kubernetes

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dpone_airflow_pack-0.73.29.tar.gz (116.9 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

dpone_airflow_pack-0.73.29-py3-none-any.whl (162.6 kB view details)

Uploaded Python 3

File details

Details for the file dpone_airflow_pack-0.73.29.tar.gz.

File metadata

  • Download URL: dpone_airflow_pack-0.73.29.tar.gz
  • Upload date:
  • Size: 116.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for dpone_airflow_pack-0.73.29.tar.gz
Algorithm Hash digest
SHA256 b9c270f1b4506065560ad7739d371fa7e46d7fba52ab22a03c1a4f5b04161f8e
MD5 021aaf7839fdb5972ff9a7614667ea95
BLAKE2b-256 bed73598ecfcefcbf84302b004dc37d819c0332c63d32dabcbdb9026ce644b5e

See more details on using hashes here.

File details

Details for the file dpone_airflow_pack-0.73.29-py3-none-any.whl.

File metadata

File hashes

Hashes for dpone_airflow_pack-0.73.29-py3-none-any.whl
Algorithm Hash digest
SHA256 762f6c6e5e0cbeeefecda8f7f8be534180654c4ec0e30ff4c876e8d0c06dcd6f
MD5 7552faa30b199ee93315cbf574a7e0b4
BLAKE2b-256 38bfb0a5d003e4d94f44e4063f44f86abec80340a2fb3554313c49a6a58963fa

See more details on using hashes here.

Release history Release notifications | RSS feed

0.74.33

2 files

0.74.32

2 files

0.74.31

2 files

0.74.30

2 files

0.74.29

2 files

0.74.28

2 files

0.74.27

2 files

0.74.26

2 files

0.74.25

2 files

0.74.24

2 files

0.74.23

2 files

0.74.22

2 files

0.74.20

2 files

0.74.19

2 files

0.74.17

2 files

0.74.16

2 files

0.74.15

2 files

0.74.14

2 files

0.74.13

2 files

0.74.12

2 files

0.74.11

2 files

0.74.10

2 files

0.74.9

2 files

0.74.8

2 files

0.74.7

2 files

0.74.6

2 files

0.74.5

2 files

0.74.4

2 files

0.74.3

2 files

0.74.2

2 files

0.74.0

2 files

0.73.42

2 files

0.73.38

2 files

0.73.36

2 files

0.73.34

2 files

0.73.31

2 files

0.73.30

2 files

This release

0.73.29 This release

2 files

0.73.28

2 files

0.73.25

2 files

0.73.24

2 files

0.73.23

2 files

0.73.22

2 files

0.73.21

2 files

0.73.20

2 files

0.73.19

2 files

0.73.17

2 files

0.73.16

2 files

0.73.15

2 files

0.73.14

2 files

0.73.13

2 files

0.73.10

2 files

0.73.9

2 files

0.73.6

2 files

0.73.2

2 files

0.73.1

2 files

0.73.0

2 files

0.72.5

2 files

0.72.4

2 files

0.72.3

2 files

0.72.2

2 files

0.72.1

2 files

0.72.0

2 files

0.70.0

2 files

0.69.0

2 files

0.68.22

2 files

0.68.21

2 files

0.68.20

2 files

0.68.19

2 files

0.68.18

2 files

0.68.17

2 files

0.68.16

2 files

0.68.15

2 files

0.68.14

2 files

0.68.12

2 files

0.68.11

2 files

0.68.10

2 files

0.68.8

2 files

0.68.7

2 files

0.68.6

2 files

0.68.5

2 files

0.68.4

2 files

0.68.2

2 files

0.68.1

2 files

0.68.0

2 files

0.67.0

2 files

0.66.17

2 files

0.66.16

2 files

0.66.15

2 files

0.66.14

2 files

0.66.13

2 files

0.66.12

2 files

0.66.11

2 files

0.66.10

2 files

0.66.9

2 files

0.66.8

2 files

0.66.7

2 files

0.66.6

2 files

0.66.5

2 files

0.66.4

2 files

0.66.3

2 files

0.66.2

2 files

0.66.1

2 files

0.66.0

2 files

0.65.2

2 files

0.65.1

2 files

0.65.0

2 files

0.64.1

2 files

0.64.0

2 files

0.63.0

2 files

0.62.1

2 files

0.62.0

2 files

0.61.3

2 files

0.61.2

2 files

0.61.1

2 files

0.61.0

2 files

0.60.2

2 files

0.60.1

2 files

0.60.0

2 files

0.59.0

2 files

0.58.0

2 files

0.57.0

2 files

0.56.0

2 files

0.55.0

2 files

0.54.0

2 files

0.53.0

2 files

0.52.0

2 files

0.51.0

2 files

0.50.0

2 files

0.49.0

2 files

0.48.1

2 files

0.48.0

2 files

0.47.0

2 files

0.46.0

2 files

0.45.0

2 files

0.44.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page