airflow-systemd

Run and monitor systemd-managed jobs from Apache Airflow.

Manage services on an Airflow worker's systemd manager or on an SSH host. Define units directly in a Python DAG or in airflow-config YAML.

from datetime import datetime, timezone
from pathlib import Path

from airflow import DAG
from airflow_systemd import ServiceConfiguration, ServiceUnitConfiguration, Systemd, SystemdAirflowConfiguration

with DAG(
    dag_id="nightly-systemd",
    schedule="@daily",
    start_date=datetime(2025, 1, 1, tzinfo=timezone.utc),
    catchup=False,
) as dag:
    systemd = Systemd(
        dag=dag,
        cfg=SystemdAirflowConfiguration(
            scope="user",
            unit_dir=Path.home() / ".config/systemd/user",
            working_dir=Path.home() / ".local/state/nightly-systemd",
            service={
                "airflow-nightly": ServiceUnitConfiguration(
                    service=ServiceConfiguration(type="exec", exec_start="/bin/sleep 5"),
                ),
            },
        ),
    )

The lifecycle writes unit files, reloads the manager, starts services, monitors state with airflow-ha, and stops and removes generated configuration on successful completion. SystemdSSH performs configuration and service operations over the configured SSH connection.

Documentation

Start with the tutorial to run a self-contained local service and check cleanup. For an existing deployment, choose a how-to guide:

The API reference describes scope, paths, monitoring defaults, and DAG constraints. Why Airflow owns the schedule explains service ownership, user managers, remote paths, and restart policies.

Published documentation is available at airflow-laminar.github.io/airflow-systemd.

Ecosystem

• systemd-pydantic supplies unit models and the systemctl client.
• supervisor-pydantic and cron-pydantic model alternative runtimes.
• airflow-supervisor provides the analogous supervisord lifecycle.
• airflow-cron converts cron jobs into ordinary Airflow tasks.
• airflow-pydantic supplies declarative task and connection models.
• airflow-config produces YAML-driven DAGs.