dpone-airflow-pack
dpone-airflow-pack is the lightweight Airflow scheduler/webserver provider for dpone GitOps packs.
It only reads a static airflow-pack.json and builds visible Airflow/Kubernetes tasks. It does not import the full
dpone runtime and intentionally contains no source/sink/native transfer dependencies such as ClickHouse, MSSQL,
pyodbc, pandas, polars, or ConnectorX.
Recommended Airflow DAG import:
from dpone_airflow_pack import build_dpone_gitops_task_group_from_pack
For remote immutable packs, keep DAG parsing cache-only:
dpone-airflow-pack-sync \
--once \
--index-uri s3://bucket/dpone-artifacts/prod/repo/latest/pack-index.json \
--reader-connection-id s3_dpone_artifacts_reader \
--cache-dir /opt/airflow/dags/.dpone-cache/airflow
dpone-airflow-pack-cache-status \
--cache-dir /opt/airflow/dags/.dpone-cache/airflow \
--json
Use load_dpone_airflow_pack_with_provenance("cached://<workload_id>") when a DAG wrapper needs to expose which
cache generation and pack checksum were used during parse. A KubernetesExecutor runtime worker is not required to mount
the scheduler or DAG processor cache.
For DAGs that should run a whole GitOps domain or a named workflow group, keep the grouping in the domain catalog and resolve it through the lightweight provider:
workflow_groups:
daily:
workload_ids:
- dim_customer
- fact_order
from dpone_airflow_pack import workload_ids_from_gitops_domain
workload_ids = workload_ids_from_gitops_domain(
repo_root,
"sales",
group="daily",
)
The resolver only reads YAML files and validates that every group member exists in the same domain catalog.
The full dpone[full,accel] package belongs in the KPO runtime image, not in the scheduler image.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file dpone_airflow_pack-0.68.19.tar.gz.
File metadata
- Download URL: dpone_airflow_pack-0.68.19.tar.gz
- Upload date:
- Size: 26.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/6.1.0 CPython/3.13.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
33926f79cb04fdf09349fa8a910196c891bdfd8da7c5ecf32a451adbed7b8bdb
|
|
| MD5 |
035e8d5fbf031e824661183df9ec807c
|
|
| BLAKE2b-256 |
9e2a8f13bd595cd8ddaf3ef705ee708f6e397eb32a49313301d9ef16931ca884
|
File details
Details for the file dpone_airflow_pack-0.68.19-py3-none-any.whl.
File metadata
- Download URL: dpone_airflow_pack-0.68.19-py3-none-any.whl
- Upload date:
- Size: 39.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/6.1.0 CPython/3.13.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
551a6bedc424f5b8871a0e7ea219a11cf24e081a68f6f8ebec85821c24d76a15
|
|
| MD5 |
f83eb418b5772bea6c9a6d1dd0093bca
|
|
| BLAKE2b-256 |
6bd5355a4454deb1b1c3eca264a7f3163e828f70da0eec78c2bbb641315bdd00
|
