Skip to main content

al-devops-kit

Airflow bootstrap tooling: a versioned installer DAG plus a CLI that deploys it.

Install

pip install al-devops-kit
al-devops-kit copy-dags /opt/airflow/dags

Python 3.10+. Install into the Airflow virtualenv. al-data-utils is pulled in automatically; apache-airflow 3.x is expected from the host environment.

CLI

al-devops-kit copy-dags <dags_dir> [--version vX.Y.Z] [--force]
al-devops-kit list-versions

copy-dags copies the installer DAG template into the given dags folder. Existing files are overwritten only when their content differs.

Installer DAG

_airflow_installer_dag.py runs daily (or on demand) and:

  1. Reads /opt/airflow/.repos/repos.yml.
  2. Clones or pulls each listed repo into /opt/airflow/.repos/<name>/.
  3. Merges the repos' requirements.txt files and pip-installs them into the Airflow venv.
  4. Installs DAGs from Python packages listed in each repo's requirements_post_install.yaml.
  5. Syncs each repo's dags_path into /opt/airflow/dags/ (copy, update, delete).

repos.yml format:

venv_pip: /opt/airflow/.venv/bin/pip
repos:
  - name: my-dags
    url: https://<token>@github.com/<org>/<repo>.git
    branch: main
    dags_path: dags/
    requirements: requirements.txt

requirements_post_install.yaml (in a repo root, optional):

package_dags:
  - name: al_data_core
    version: latest
    files:
      - data_core_dag.py
      - data_core_config_engine_dag.py

Each package listed must be installed in the Airflow venv and expose a dags_installer module with _copy_dags(target, version, force).

Repository extras

The source repository also contains an Ansible playbook for a single-VM Airflow install on Ubuntu. It is not part of the PyPI package.

Changelog

See CHANGELOG.md in the repository for release notes.

Metadata

Release files for al-devops-kit 0.4.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for al-devops-kit 0.4.0
File Size Uploaded Provenance Actions
al_devops_kit-0.4.0.tar.gz 9.9 kB Without provenance Details for al_devops_kit-0.4.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for al-devops-kit 0.4.0
File Interpreter ABI Platform Actions
al_devops_kit-0.4.0-py3-none-any.whl Python 3 none any Details for al_devops_kit-0.4.0-py3-none-any.whl

Total release size: 19.9 kB

Release files / al_devops_kit-0.4.0.tar.gz

Download URL al_devops_kit-0.4.0.tar.gz
Size 9.9 kB
Tags Source
SHA-256 checksum
How to use checksums
c0506e83da916034e5aea92578f4d8123830a99b18e21026f396399b02561fce
BLAKE2b-256 checksum
How to use checksums
a5669e31ea776004de92d227bcfb613a6bc89581c80a1dcda0b2cb50b1036c0e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / al_devops_kit-0.4.0-py3-none-any.whl

Download URL al_devops_kit-0.4.0-py3-none-any.whl
Size 10.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
041a8b32df28339d46cc2ab001fcd20b71e49a99a576711826d56d9a7de53ff5
BLAKE2b-256 checksum
How to use checksums
4666c59f7f654161a1d2a2001fb05974d6e19b99f4e6983a252e05978ae6fc6b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Security for al-devops-kit 0.4.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.4.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page