Skip to main content

Lightweight Airflow scheduler-side pack provider for dpone GitOps workloads

Project description

dpone-airflow-pack

dpone-airflow-pack is the lightweight Airflow scheduler/webserver provider for dpone GitOps packs.

It only reads a static airflow-pack.json and builds visible Airflow/Kubernetes tasks. It does not import the full dpone runtime and intentionally contains no source/sink/native transfer dependencies such as ClickHouse, MSSQL, pyodbc, pandas, polars, or ConnectorX.

Recommended Airflow DAG import:

from dpone_airflow_pack import build_dpone_gitops_task_group_from_pack

The full dpone[full,accel] package belongs in the KPO runtime image, not in the scheduler image.

Airflow execution policy

airflow-pack.json can carry a scheduler-side execution policy:

{
  "airflow": {
    "execution": {
      "task_executor": "KubernetesExecutor",
      "deferrable": true,
      "on_finish_action": "delete_succeeded_pod"
    }
  }
}

The lightweight provider maps it to KubernetesPodOperator kwargs (executor, deferrable, on_finish_action) and treats those fields as pack-owned. DAG-local overrides may still set DAG-owned values such as retries, pool, get_logs, or in_cluster.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dpone_airflow_pack-0.65.2.tar.gz (14.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

dpone_airflow_pack-0.65.2-py3-none-any.whl (21.3 kB view details)

Uploaded Python 3

File details

Details for the file dpone_airflow_pack-0.65.2.tar.gz.

File metadata

  • Download URL: dpone_airflow_pack-0.65.2.tar.gz
  • Upload date:
  • Size: 14.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for dpone_airflow_pack-0.65.2.tar.gz
Algorithm Hash digest
SHA256 a31d87df41280685cf69028bdc7109cc0e8d8b15b5225f6aa4e08312a7616c33
MD5 426ebfa3a6831a63b52387e3953152ea
BLAKE2b-256 8cfb925e0d06a5e5d4ed4c4c1ef2c2fbbc3dcccfaad6e2433604bfbd5a969e5b

See more details on using hashes here.

File details

Details for the file dpone_airflow_pack-0.65.2-py3-none-any.whl.

File metadata

File hashes

Hashes for dpone_airflow_pack-0.65.2-py3-none-any.whl
Algorithm Hash digest
SHA256 b9188c4f40e074b906632f367b15c0a8ba949de9733faec9e50bf72b1f022e9d
MD5 19f678fc38e5603baa0bbc2626070f5a
BLAKE2b-256 c7a5e8248b704ff8f2ba9fc705e061560abc1747a47ef26babed051b666d9ed1

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page