dpone-airflow-pack

dpone-airflow-pack is the lightweight, Airflow-independent reader and DAG construction library for dpone GitOps packs. The formal apache-airflow-providers-dpone distribution owns provider discovery and the canonical airflow.providers.dpone namespace.

It only reads a static airflow-pack.json and builds visible Airflow/Kubernetes tasks. It does not import the full dpone runtime and intentionally contains no source/sink/native transfer dependencies such as ClickHouse, MSSQL, pyodbc, pandas, polars, or ConnectorX.

Recommended zero-boilerplate Airflow DAG import:

from airflow.providers.dpone import load_dpone_dags

load_report = load_dpone_dags(
    globals(),
    index_path="/opt/airflow/dags/.dpone-cache/current/airflow-index.json",
)
if load_report.fatal:
    error_code = (
        load_report.errors[0].get("code", "DPONE_AIRFLOW_INDEX_INVALID")
        if load_report.errors
        else "DPONE_AIRFLOW_INDEX_INVALID"
    )
    raise RuntimeError(
        f"{error_code}: dpone Airflow deployment index could not be loaded"
    )

The formal provider's canonical namespace is PEP 561 typed. DponeDag.from_spec(...) and DponeTaskGroup.from_pack(...) remain typed escape hatches for hybrid DAGs. The old dpone_airflow_pack imports remain compatibility shims and emit one deprecation warning per process for provider-facade objects.

For the complete import, generated-loader, and cache-layout cutover, follow the provider and cache migration guide.

Legacy installations that still consume a mutable remote latest/pack-index.json may keep the compatibility sync temporarily:

dpone-airflow-pack-sync \
  --once \
  --index-uri s3://bucket/dpone-artifacts/prod/repo/latest/pack-index.json \
  --reader-connection-id s3_dpone_artifacts_reader \
  --cache-dir /opt/airflow/dags/.dpone-cache/airflow

dpone-airflow-pack-cache-status \
  --cache-dir /opt/airflow/dags/.dpone-cache/airflow \
  --json

The loader reads only the local composite deployment index and its listed checksum-verified DAG specs/packs. It does not scan authoring sources, refresh a remote cache, or read Airflow Variables, Connections, Vault, Kubernetes, or a metadata database during parse.

For DAGs that should run a whole GitOps domain or a named workflow group, keep the grouping in the domain catalog and resolve it through the lightweight provider:

workflow_groups:
  daily:
    workload_ids:
      - dim_customer
      - fact_order

from dpone_airflow_pack import workload_ids_from_gitops_domain

workload_ids = workload_ids_from_gitops_domain(
    repo_root,
    "sales",
    group="daily",
)

The resolver only reads YAML files and validates that every group member exists in the same domain catalog.

The full dpone[full,accel] package belongs in the KPO runtime image, not in the scheduler image.

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.74.33)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for paulkov007 from gravatar.com paulkov007

Credits

Author/Maintainer: PaulKov

License expression

Apache-2.0
View SPDX License List

Requires

Python <3.13, >=3.11

Tags

airflow gitops data-engineering scheduler kubernetes

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dpone_airflow_pack-0.73.14.tar.gz (93.8 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

dpone_airflow_pack-0.73.14-py3-none-any.whl (129.1 kB view details)

Uploaded Python 3

File details

Details for the file dpone_airflow_pack-0.73.14.tar.gz.

File metadata

  • Download URL: dpone_airflow_pack-0.73.14.tar.gz
  • Upload date:
  • Size: 93.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for dpone_airflow_pack-0.73.14.tar.gz
Algorithm Hash digest
SHA256 4a9c05281f5e70e1f2d7eaeb62439cc1ca64cce6eb872a10957ae3495ea169ed
MD5 be50c67cd38db7a3ea96d921c9948cc2
BLAKE2b-256 19a9e04a8b6fe2e8fc31384d0fdb906c8ca99ed81e45500adec95f132b8511ca

See more details on using hashes here.

File details

Details for the file dpone_airflow_pack-0.73.14-py3-none-any.whl.

File metadata

File hashes

Hashes for dpone_airflow_pack-0.73.14-py3-none-any.whl
Algorithm Hash digest
SHA256 456ea8509a07b348a0d8178983aef6cf91a2d000eb059e545246345bf936fd25
MD5 03faa9bfcb2386aeb96613d3f4f2ded5
BLAKE2b-256 7bee928be0135abab6b1dabe8d8377fb8ba10571ed4e48a7d9696dc48a00470f

See more details on using hashes here.

Release history Release notifications | RSS feed

0.74.33

2 files

0.74.32

2 files

0.74.31

2 files

0.74.30

2 files

0.74.29

2 files

0.74.28

2 files

0.74.27

2 files

0.74.26

2 files

0.74.25

2 files

0.74.24

2 files

0.74.23

2 files

0.74.22

2 files

0.74.20

2 files

0.74.19

2 files

0.74.17

2 files

0.74.16

2 files

0.74.15

2 files

0.74.14

2 files

0.74.13

2 files

0.74.12

2 files

0.74.11

2 files

0.74.10

2 files

0.74.9

2 files

0.74.8

2 files

0.74.7

2 files

0.74.6

2 files

0.74.5

2 files

0.74.4

2 files

0.74.3

2 files

0.74.2

2 files

0.74.0

2 files

0.73.42

2 files

0.73.38

2 files

0.73.36

2 files

0.73.34

2 files

0.73.31

2 files

0.73.30

2 files

0.73.29

2 files

0.73.28

2 files

0.73.25

2 files

0.73.24

2 files

0.73.23

2 files

0.73.22

2 files

0.73.21

2 files

0.73.20

2 files

0.73.19

2 files

0.73.17

2 files

0.73.16

2 files

0.73.15

2 files

This release

0.73.14 This release

2 files

0.73.13

2 files

0.73.10

2 files

0.73.9

2 files

0.73.6

2 files

0.73.2

2 files

0.73.1

2 files

0.73.0

2 files

0.72.5

2 files

0.72.4

2 files

0.72.3

2 files

0.72.2

2 files

0.72.1

2 files

0.72.0

2 files

0.70.0

2 files

0.69.0

2 files

0.68.22

2 files

0.68.21

2 files

0.68.20

2 files

0.68.19

2 files

0.68.18

2 files

0.68.17

2 files

0.68.16

2 files

0.68.15

2 files

0.68.14

2 files

0.68.12

2 files

0.68.11

2 files

0.68.10

2 files

0.68.8

2 files

0.68.7

2 files

0.68.6

2 files

0.68.5

2 files

0.68.4

2 files

0.68.2

2 files

0.68.1

2 files

0.68.0

2 files

0.67.0

2 files

0.66.17

2 files

0.66.16

2 files

0.66.15

2 files

0.66.14

2 files

0.66.13

2 files

0.66.12

2 files

0.66.11

2 files

0.66.10

2 files

0.66.9

2 files

0.66.8

2 files

0.66.7

2 files

0.66.6

2 files

0.66.5

2 files

0.66.4

2 files

0.66.3

2 files

0.66.2

2 files

0.66.1

2 files

0.66.0

2 files

0.65.2

2 files

0.65.1

2 files

0.65.0

2 files

0.64.1

2 files

0.64.0

2 files

0.63.0

2 files

0.62.1

2 files

0.62.0

2 files

0.61.3

2 files

0.61.2

2 files

0.61.1

2 files

0.61.0

2 files

0.60.2

2 files

0.60.1

2 files

0.60.0

2 files

0.59.0

2 files

0.58.0

2 files

0.57.0

2 files

0.56.0

2 files

0.55.0

2 files

0.54.0

2 files

0.53.0

2 files

0.52.0

2 files

0.51.0

2 files

0.50.0

2 files

0.49.0

2 files

0.48.1

2 files

0.48.0

2 files

0.47.0

2 files

0.46.0

2 files

0.45.0

2 files

0.44.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page