Migration Helper

airflow-migration-helper is a Python project for helping migrate Apache Airflow code from Airflow 2 to Airflow 3. The PyPI distribution installs the command-line tool as migration-helper.

The current implementation is focused on static analysis and migration assistance for DAG code. It includes checks based on Ruff AIR rules, Airflow configuration linting, DAG import checks, dependency compatibility checks, base reporting, shared subprocess helpers, file selection helpers, and tests.

How it works

migration-helper migrates code from Airflow 2 to Airflow 3, so it is run against a target project that is currently on Airflow 2:

• The analyzed project (DAGs, configs, dependencies) stays on Airflow 2.
• The tool runs in its own dedicated virtual environment (see Requirements and setup).

Which Airflow version the checker environment needs depends on the checker:

• ruff and dep_check analyze the target statically and do not require Airflow at all.
• airflow_config checks config keys against a built-in static rule table, so it also works without Airflow. When Airflow is installed in the checker environment, the checker additionally runs airflow config lint and reports more findings. Note that the Airflow CLI commands used by this checker operate on the active configuration of the checker environment (AIRFLOW_HOME / AIRFLOW_CONFIG), not on airflow.cfg files found in the target; only the static scan covers target files. To lint or update a specific config file through the CLI path, point AIRFLOW_CONFIG at it.
• dag_check runs airflow dags list-import-errors from the checker environment, so it parses the Airflow 2 DAGs under the Airflow installed in that environment (Airflow 3 for full compatibility checks).
• db_migrate runs the one-shot, read-only airflow db check-migrations command from the checker environment. No running Airflow processes are needed, but a reachable metadata database must be configured for that environment. The stage is only enabled with --db-migrate.

Airflow's CLI does not run on native Windows (airflow imports POSIX-only modules). On Windows, dag_check and db_migrate always report FAILED; run the tool in WSL or a container when those stages matter.

Every checker reports an execution status alongside its findings:

• success: the check ran; its findings (including zero) are trustworthy.
• partial: the check ran with reduced capability, for example airflow_config with only the static rule table because Airflow is not installed.
• failed: the check could not run or its output could not be read. A failed checker is never rendered as 0 issues.
• skipped: the stage is intentionally not run (for example db_migrate without --db-migrate) or there is nothing to check.

For full functionality (all checkers, including dag_check), install the tool together with Airflow 3 in the same virtual environment:

python -m venv .venv-mh
.venv-mh/bin/pip install "airflow-migration-helper[airflow]"

Expected Output

After running migration-helper, you get a practical migration report for the selected Airflow project. The report lists compatibility issues by file and rule, shows error/warning severity, and explains whether each finding can be handled automatically or needs manual review.

Reports can be written in JSON and HTML formats. The JSON report is intended for automation and CI, while the HTML report is intended for manual review. Both formats expose the same issue details.

The JSON report has this top-level structure:

• generated_at: UTC timestamp when the report was created.
• target: analyzed file or directory.
• summary: total number of findings, split into errors and warnings, plus checkers_failed: the list of checkers that could not run.
• checkers: execution status of every checker (see below).
• by_file: findings grouped by affected file path.
• by_type: findings grouped by rule_id.

Each checkers entry describes one checker:

• status: success, partial, failed, or skipped.
• reason: machine-readable cause for non-success, for example airflow_unavailable, output_unparseable, or disabled.
• issues: the number of findings, present only for success and partial. A failed or skipped entry carries no count, so a check that did not run can never be read as 0 issues.
• For failed entries the diagnostic detail is included: error, command, and stdout_excerpt when available.

The terminal summary prints the same information per checker:

Per-checker breakdown:
  ruff            13 issue(s)
  airflow_config  0 issue(s)
  dag_check       FAILED       airflow is not available - install with the [airflow] extra
  dep_check       SKIPPED      no dependency file found
  db_migrate      SKIPPED      not enabled - pass --db-migrate

The green No issues found. Ready for Airflow 3. message is printed only when every checker ran; if any checker failed, the summary instead reports that the results are incomplete.

Each finding contains:

• file: affected file or logical target, for example airflow.cfg or requirements.txt.
• line and col: source location when the checker can provide it; otherwise null.
• rule_id: checker-specific rule or error identifier, for example AIR301, CFG001, or DEP_RESOLUTION_FAILED.
• message: short explanation of the problem.
• severity: error, warning, or info.
• source: checker that produced the finding, for example ruff, airflow_config, dag_check, dep_check, or db_migrate.

Reports use these fields to describe fixability:

• fix_safety: "safe" means the checker knows a safe mechanical fix or safe migration action. migration-helper fix TARGET --safe-only applies safe Ruff fixes and, when Airflow is available in the checker environment, also runs airflow config update --fix on that environment's active config (see How it works). With --dry-run, Ruff applies those fixes on a temporary copy so TARGET is unchanged; config update stays in Airflow CLI dry-run mode and does not write.
• fix_safety: "unsafe" means the tool knows the likely change, but a human should review it before applying.
• fix_safety: null (or fix_safety: "manual" in HTML report) means there is no automatic fix path in the current implementation. Treat it as manual work.
• fix_description is the human-readable hint for what to change or which command/path is relevant.

Today, DAG import errors, dependency conflicts, and DB migration findings are diagnostic/manual. Source-level DAG fixes outside Ruff are still planned.

Requirements and setup

• Python >=3.10
• pip version >=22.2
• git available in PATH
• Work in a dedicated virtual environment (see below)

Install airflow-migration-helper and its dependencies (ruff and others) into a dedicated virtual environment, not into the system Python. This keeps the tool's dependencies isolated from the analyzed project.

python -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate

Install from PyPI:

pip install airflow-migration-helper[airflow]

Offline installation

To install on a machine without access to PyPI, build a wheelhouse on a machine with internet access and install from it:

# On a machine with internet access
pip download airflow-migration-helper[airflow]==1.1.0 -d wheelhouse/

# On the offline machine
pip install --no-index --find-links ./wheelhouse airflow-migration-helper

Note: ruff is a platform-specific binary package. Download the wheelhouse on a machine with the same OS and the same Python version as the target machine.

If the wheelhouse must be prepared for a different platform than the download machine, pass the target platform tags explicitly. This works only with pre-built wheels, so --only-binary=:all: is required:

# Build a wheelhouse for manylinux x86_64 + CPython 3.11 from any machine
pip download airflow-migration-helper[airflow]==1.1.0 -d wheelhouse/ \
  --only-binary=:all: \
  --platform manylinux2014_x86_64 \
  --python-version 311 \
  --implementation cp \
  --abi cp311

ruff ships platform-specific wheels (py3-none-manylinux2014_x86_64, macosx_11_0_arm64, win_amd64, ...) so the --platform tag matters most; packaging and tomli are pure Python and work everywhere. Repeat the command per target platform (for example macosx_11_0_arm64 or win_amd64) or build the wheelhouse on a machine matching the target configuration.

Alternatives: copy a prepared virtual environment as a whole (same OS / architecture / Python required), or serve the packages from an internal PyPI mirror (devpi, Nexus, Artifactory) and use pip config set global.index-url http://<mirror>/.

Offline usage notes: Airflow 3.2.1 constraints for Python 3.10-3.14 are bundled with the package, so the default dependency check works offline. Other Airflow versions require network access to download constraints, and the deep dependency check (pip install --dry-run / uv pip compile) needs access to a package index; point it at an internal one via PIP_INDEX_URL / UV_INDEX_URL if needed. git must still be available in PATH.

For development, install with test and Airflow extras instead:

pip install -e ".[dev]"

This editable install is required if you want to run the CLI as migration-helper from the repository checkout. Without it, use python -m migration_helper.cli ... from the repository root.

Usage

migration-helper COMMAND [TARGET] [OPTIONS]

Use the built-in help commands to see available options:

migration-helper --help
migration-helper analyze --help
migration-helper fix --help

Commands

• analyze: scan TARGET for Airflow 3 compatibility issues in read-only mode.
• fix: scan TARGET and apply or dry-run available auto-fix paths.

Shared Options

• TARGET: directory or file to scan. Defaults to the current directory.
• --report-dir PATH, -r PATH: write migration_report.json and/or migration_report.html into PATH.
• --output-format json|html|all: report format when --report-dir is set. Default: all.
• --modified-only: only check files reported by git status --porcelain under the target path. If git state cannot be resolved, all files are checked. With fix --dry-run, git selection runs on the original TARGET because the temporary copy has no .git.
• --target-airflow-version VERSION: Airflow version for dependency checks. Default: 3.2.1.
• --target-python-version VERSION: Python major/minor version for dependency constraints. Defaults to the current interpreter version.
• --dependency-file FILE: dependency file to check at the target root. Supports requirements-style .txt files and pyproject.toml. Default: requirements.txt.
• --use-uv: use uv instead of pip for dependency resolver checks.
• --db-migrate: enable the DB migration stage. The stage runs the read-only airflow db check-migrations command and reports a warning (DB001) when the configured metadata database has pending migrations. A check that could not run - for example an unreachable database - is reported as a checker failure (FAILED), not as a DB001 finding. By default, it uses the checker environment's own database; to check the Airflow 2 cluster metadata database, point the environment's connection string (for example AIRFLOW__DATABASE__SQL_ALCHEMY_CONN) at it. Applying migrations is not automated; run airflow db migrate yourself after reviewing the report.
• --quiet, -q: suppress progress and summary output. Exit codes are still set.

Fix Options

• --safe-only: enable safe mechanical fixes where they are currently implemented. Today this mainly drives Ruff/config fix paths; the source-level DAG fixer is still a placeholder.
• --unsafe: also enable unsafe fixes that require human review. Implies safe fixes.
• --dry-run: do not modify TARGET files or the database. Ruff still applies --safe-only / --unsafe fixes, but on a temporary copy of TARGET; dag_check then runs against that already-fixed copy. The report lists leftover issues after those fixes (closer to a real fix than to analyze), with paths rewritten back to TARGET. The copy is deleted afterwards. Config update remains a non-writing Airflow CLI dry-run. The copy includes only TARGET. Imports that resolve to files outside that tree (a sibling plugins/ package, a repo-root helper module) may fail in dag_check during dry-run even if a real fix against the original layout would succeed.
• --fix-config: request Airflow config updates explicitly. In the current implementation any fix mode (--safe-only / --unsafe) already enables the config update path, so the flag is effectively redundant. Config update still respects --dry-run.
• Where config fixes are applied: airflow config update runs against the checker environment's active config - $AIRFLOW_CONFIG, else $AIRFLOW_HOME/airflow.cfg, else ~/airflow/airflow.cfg. airflow.cfg files under TARGET are checked but never rewritten; to migrate a config, point AIRFLOW_CONFIG at it (or run inside the Airflow environment itself). Airflow regenerates the file: options whose value equals the Airflow 3 default are written commented out, and renamed keys carry a # Renamed from <section>.<key> note. Keys removed in Airflow 3 that Airflow's own updater does not drop (for example webserver.rbac, core.store_serialized_dags, core.store_dag_code on Airflow 3.2.1) are removed by migration-helper itself. Applied changes are reported as CFG004 (info); --dry-run lists the would-be changes and removals as CFG005 (warning), and findings that describe the environment config carry its full path in reports.
• Config-update resilience: after the update rewrites the checker environment's airflow.cfg, migration-helper re-reads it. If Airflow's writer produced an unparseable file (a known Airflow defect with multiline values such as dag_bundle_config_list), the file is repaired in place - semantically identical, only indentation restored - or, when that is impossible, the pre-update state is restored. The checker environment is also repaired at startup on every run, so a previously broken environment self-heals.

Exit Codes

• 0: no error-severity issues found and every checker ran. Warnings may still be present.
• 1: one or more error-severity issues found.
• 2: invalid arguments or the TARGET path does not exist.
• 3: incomplete run - at least one checker could not execute, so the report cannot be trusted as a complete answer even when it contains no findings. partial and skipped stages do not trigger this code.

Examples

# Read-only scan of all DAGs in ./dags/
migration-helper analyze ./dags/

# Scan and write HTML + JSON reports
migration-helper analyze ./dags/ --report-dir ./reports/

# Check dependencies against a specific Airflow/Python target
migration-helper analyze ./dags/ --target-airflow-version 3.2.1 --target-python-version 3.10

# Only report issues in files changed in the current git working tree
migration-helper analyze ./dags/ --modified-only

# Preview leftover issues after safe fixes without changing TARGET
migration-helper fix ./dags/ --safe-only --dry-run

# Apply currently available safe fixes in-place
migration-helper fix ./dags/ --safe-only

# Quiet mode for CI
migration-helper analyze ./dags/ --quiet
echo $?   # 0 = clean, 1 = errors

License

Licensed under the Apache License, Version 2.0. See the LICENSE file for the full license text.

Common Commands

Run unit tests:

pytest tests/unit

Run integration tests:

pytest -m integration tests/integration

Run all regular tests, excluding integration and e2e by default:

pytest

Run e2e tests explicitly:

python -m pytest tests/e2e -m e2e

If you would like to test the tool with your own DAGs, place them in test_dags or another local folder and run the tool based on the examples above.

E2E Testing

The project keeps a small Airflow 2 DAG corpus under tests/e2e/airflow2-dag-corpus/ and separate runtime provider fixtures under tests/e2e/airflow2-provider-fixtures/. The corpus is the migration-helper target; provider fixtures are import support only.

Run all e2e checks explicitly:

python -m pytest tests/e2e -m e2e

Run the Airflow 2 baseline import smoke:

cd tests/e2e
docker compose -f docker-compose.airflow2.yml run --rm airflow2-smoke

For the current e2e layout, DAG corpus contents, dry-run artifacts under tests/e2e/runs/, and the difference between analyze and fix --dry-run, see tests/e2e/README.md.

Docker

The project includes a multi-stage Dockerfile and docker-compose.yml. Two images are built:

• production: runtime dependencies used for running migration-helper commands.
• dev: development/test dependencies used for running tests.

Build

docker compose build

Prepare DAG files

Place your DAG files in test_dags/ (this directory is git-ignored):

mkdir -p test_dags
cp path/to/your_dag.py test_dags/

Run analysis (read-only)

The migration-helper-analyze service mounts ./test_dags at /dags as read-only, which is appropriate for analyze:

docker compose run --rm migration-helper-analyze

Or pass any command and options directly:

docker compose run --rm migration-helper-analyze analyze /dags
docker compose run --rm migration-helper-analyze analyze /dags --modified-only
docker compose run --rm migration-helper-analyze analyze /dags/demo.py

Apply fixes (writable)

The migration-helper-fix service mounts ./test_dags at /dags as writable so Ruff can rewrite .py files in place:

docker compose run --rm migration-helper-fix

Examples:

docker compose run --rm migration-helper-fix fix /dags --safe-only --dry-run
docker compose run --rm migration-helper-fix fix /dags/demo.py --safe-only
docker compose run --rm migration-helper-fix fix /dags/demo.py --unsafe

Run tests

Unit tests:

docker compose run --rm test

Integration tests:

docker compose run --rm test -m integration tests/integration

E2E tests:

docker compose run --rm test tests/e2e -m e2e

E2E dry-run reports are written to: tests/e2e/runs/<UTC timestamp>/ Open the latest run directory and check analyze/analyze-report/migration_report.html, fix-dry-run/fix-dry-run-report/migration_report.html, fix-safe/fix-safe-report/migration_report.html, or fix-unsafe/fix-unsafe-report/migration_report.html.

The main docker-compose.yml mounts ./tests to /app/tests, so reports created in the test container are available on the host immediately. No docker cp is needed.