airflow-supervisor

Run and monitor supervisord-managed jobs from Apache Airflow.

Manage a dedicated supervisord instance on an Airflow worker or an SSH host. Define its programs directly in a Python DAG or in airflow-config YAML.

from datetime import datetime, timezone

from airflow import DAG
from airflow_supervisor import ProgramConfiguration, Supervisor, SupervisorAirflowConfiguration

with DAG(
    dag_id="nightly-supervisor",
    schedule="@daily",
    start_date=datetime(2025, 1, 1, tzinfo=timezone.utc),
    catchup=False,
) as dag:
    supervisor = Supervisor(
        dag=dag,
        cfg=SupervisorAirflowConfiguration(
            working_dir="/var/tmp/nightly-supervisor",
            port="127.0.0.1:9001",
            program={"nightly": ProgramConfiguration(command="/bin/sleep 5")},
        ),
    )

The lifecycle writes configuration, starts supervisord and its programs, monitors program state with airflow-ha, and stops and removes the instance on successful completion. SupervisorSSH runs configuration and daemon commands over SSH; program control and checks use the remote XML-RPC endpoint.

Documentation

Start with the tutorial to run a self-contained local job and check its cleanup. For an existing deployment, choose a how-to guide:

The API reference describes configuration fields, defaults, and DAG constraints. Why Airflow delegates process ownership explains worker placement, SSH and XML-RPC, and the Python/YAML configuration choices.

Use the log forwarding and health-check guide to collect program output in task logs and monitor retained services between runs.

Published documentation is available at airflow-laminar.github.io/airflow-supervisor.

Ecosystem

• supervisor-pydantic supplies supervisord models and lifecycle tools.
• systemd-pydantic and cron-pydantic model alternative runtimes.
• airflow-systemd provides the analogous systemd lifecycle.
• airflow-cron converts cron jobs into ordinary Airflow tasks.
• airflow-pydantic supplies declarative task, host, and connection models.
• airflow-config produces YAML-driven DAGs.