Migration Helper

migration-helper is a Python project for helping migrate Apache Airflow code from Airflow 2 to Airflow 3.

The current implementation is focused on static analysis and migration assistance for DAG code. It includes checks based on Ruff AIR rules, Airflow configuration linting, DAG import checks, dependency compatibility checks, base reporting, shared subprocess helpers, file selection helpers, and tests.

Expected Output

After running migration-helper, you get a practical migration report for the selected Airflow project. The report lists compatibility issues by file and rule, shows error/warning severity, and explains whether each finding can be handled automatically or needs manual review.

Reports can be written in JSON and HTML formats. The JSON report is intended for automation and CI, while the HTML report is intended for manual review. Both formats expose the same issue details.

The JSON report has this top-level structure:

• generated_at: UTC timestamp when the report was created.
• target: analyzed file or directory.
• summary: total number of findings, split into errors and warnings.
• by_file: findings grouped by affected file path.
• by_type: findings grouped by rule_id.

Each finding contains:

• file: affected file or logical target, for example airflow.cfg or requirements.txt.
• line and col: source location when the checker can provide it; otherwise null.
• rule_id: checker-specific rule or error identifier, for example AIR301, CFG001, or DEP_RESOLUTION_FAILED.
• message: short explanation of the problem.
• severity: error, warning, or info.
• source: checker that produced the finding, for example ruff, airflow_config, dag_check, dep_check, or db_migrate.

Reports use these fields to describe fixability:

• fix_safety: "safe" means the checker knows a safe mechanical fix or safe migration action. Safe Ruff fixes can be applied with migration-helper fix TARGET --safe-only; Airflow config fixes require the config fix path and are still blocked by --dry-run.
• fix_safety: "unsafe" means the tool knows the likely change, but a human should review it before applying.
• fix_safety: null (or fix_safety: "manual" in HTML report) means there is no automatic fix path in the current implementation. Treat it as manual work.
• fix_description is the human-readable hint for what to change or which command/path is relevant.

Today, DAG import errors, dependency conflicts, and DB migration findings are diagnostic/manual. Source-level DAG fixes outside Ruff are still planned.

Requirements and setup

• Python >=3.10
• git available in PATH

Install from PyPI:

pip install migration-helper

Offline installation

To install on a machine without access to PyPI, build a wheelhouse on a machine with internet access and install from it:

# On a machine with internet access
pip download migration-helper==1.0.1 -d wheelhouse/

# On the offline machine
pip install --no-index --find-links ./wheelhouse migration-helper

Note: ruff is a platform-specific binary package. Download the wheelhouse on a machine with the same OS and the same Python version as the target machine.

If the wheelhouse must be prepared for a different platform than the download machine, pass the target platform tags explicitly. This works only with pre-built wheels, so --only-binary=:all: is required:

# Build a wheelhouse for manylinux x86_64 + CPython 3.11 from any machine
pip download migration-helper==1.0.1 -d wheelhouse/ \
  --only-binary=:all: \
  --platform manylinux2014_x86_64 \
  --python-version 311 \
  --implementation cp \
  --abi cp311

ruff ships platform-specific wheels (py3-none-manylinux2014_x86_64, macosx_11_0_arm64, win_amd64, ...) so the --platform tag matters most; packaging and tomli are pure Python and work everywhere. Repeat the command per target platform (for example macosx_11_0_arm64 or win_amd64) or build the wheelhouse on a machine matching the target configuration.

Alternatives: copy a prepared virtual environment as a whole (same OS / architecture / Python required), or serve the packages from an internal PyPI mirror (devpi, Nexus, Artifactory) and use pip config set global.index-url http://<mirror>/.

Offline usage notes: Airflow 3.2.1 constraints for Python 3.10-3.14 are bundled with the package, so the default dependency check works offline. Other Airflow versions require network access to download constraints, and the deep dependency check (pip install --dry-run / uv pip compile) needs access to a package index; point it at an internal one via PIP_INDEX_URL / UV_INDEX_URL if needed. git must still be available in PATH.

For development, install with test and Airflow extras instead:

pip install -e ".[dev]"

This editable install is required if you want to run the CLI as migration-helper from the repository checkout. Without it, use python -m migration_helper.cli ... from the repository root.

Usage

migration-helper COMMAND [TARGET] [OPTIONS]

Use the built-in help commands to see available options:

migration-helper --help
migration-helper analyze --help
migration-helper fix --help

Commands

• analyze: scan TARGET for Airflow 3 compatibility issues in read-only mode.
• fix: scan TARGET and apply or dry-run available auto-fix paths.

Shared Options

• TARGET: directory or file to scan. Defaults to the current directory.
• --report-dir PATH, -r PATH: write migration_report.json and/or migration_report.html into PATH.
• --output-format json|html|all: report format when --report-dir is set. Default: all.
• --modified-only: only check files reported by git status --porcelain under the target path. If git state cannot be resolved, all files are checked.
• --target-airflow-version VERSION: Airflow version for dependency checks. Default: 3.2.1.
• --target-python-version VERSION: Python major/minor version for dependency constraints. Defaults to the current interpreter version.
• --dependency-file FILE: dependency file to check at the target root. Supports requirements-style .txt files and pyproject.toml. Default: requirements.txt.
• --use-uv: use uv instead of pip for dependency resolver checks.
• --db-migrate: enable the DB migration stage. The current implementation is still a placeholder, so this flag is part of the CLI contract but does not yet produce a real DB migration report.
• --quiet, -q: suppress progress and summary output. Exit codes are still set.

Fix Options

• --safe-only: enable safe mechanical fixes where they are currently implemented. Today this mainly drives Ruff/config fix paths; the source-level DAG fixer is still a placeholder.
• --unsafe: also enable unsafe fixes that require human review. Implies safe fixes.
• --dry-run: guarantee that files and the database are not modified. In the current implementation, dry-run disables Ruff/config fix application before the pipeline runs, so it is mostly a scan/report safety check rather than a full diff preview.
• --fix-config: request Airflow config updates. Config update still respects --dry-run.

Exit Codes

• 0: no error-severity issues found. Warnings may still be present.
• 1: one or more error-severity issues found.
• 2: invalid arguments or the TARGET path does not exist.

Examples

# Read-only scan of all DAGs in ./dags/
migration-helper analyze ./dags/

# Scan and write HTML + JSON reports
migration-helper analyze ./dags/ --report-dir ./reports/

# Check dependencies against a specific Airflow/Python target
migration-helper analyze ./dags/ --target-airflow-version 3.2.1 --target-python-version 3.10

# Only report issues in files changed in the current git working tree
migration-helper analyze ./dags/ --modified-only

# Dry-run the current fix command path without writing files
migration-helper fix ./dags/ --safe-only --dry-run

# Apply currently available safe fixes in-place
migration-helper fix ./dags/ --safe-only

# Quiet mode for CI
migration-helper analyze ./dags/ --quiet
echo $?   # 0 = clean, 1 = errors

License

Licensed under the Apache License, Version 2.0. See the LICENSE file for the full license text.

Common Commands

Run unit tests:

pytest tests/unit

Run integration tests:

pytest -m integration tests/integration

Run all regular tests, excluding integration and e2e by default:

pytest

Run e2e tests explicitly:

python -m pytest tests/e2e -m e2e

If you would like to test the tool with your own DAGs, place them in test_dags or another local folder and run the tool based on the examples above.

E2E Testing

The project keeps a small Airflow 2 DAG corpus under tests/e2e/airflow2-dag-corpus/ and separate runtime provider fixtures under tests/e2e/airflow2-provider-fixtures/. The corpus is the migration-helper target; provider fixtures are import support only.

Run all e2e checks explicitly:

python -m pytest tests/e2e -m e2e

Run the Airflow 2 baseline import smoke:

cd tests/e2e
docker compose -f docker-compose.airflow2.yml run --rm airflow2-smoke

For the current e2e layout, DAG corpus contents, dry-run artifacts under tests/e2e/runs/, and the difference between analyze and fix --dry-run, see tests/e2e/README.md.

Docker

The project includes a multi-stage Dockerfile and docker-compose.yml. Two images are built:

• production: runtime dependencies used for running migration-helper commands.
• dev: development/test dependencies used for running tests.

Build

docker compose build

Prepare DAG files

Place your DAG files in test_dags/ (this directory is git-ignored):

mkdir -p test_dags
cp path/to/your_dag.py test_dags/

Run analysis (read-only)

The migration-helper-analyze service mounts ./test_dags at /dags as read-only, which is appropriate for analyze:

docker compose run --rm migration-helper-analyze

Or pass any command and options directly:

docker compose run --rm migration-helper-analyze analyze /dags
docker compose run --rm migration-helper-analyze analyze /dags --modified-only
docker compose run --rm migration-helper-analyze analyze /dags/demo.py

Apply fixes (writable)

The migration-helper-fix service mounts ./test_dags at /dags as writable so Ruff can rewrite .py files in place:

docker compose run --rm migration-helper-fix

Examples:

docker compose run --rm migration-helper-fix fix /dags --safe-only --dry-run
docker compose run --rm migration-helper-fix fix /dags/demo.py --safe-only
docker compose run --rm migration-helper-fix fix /dags/demo.py --unsafe

Run tests

Unit tests:

docker compose run --rm test

Integration tests:

docker compose run --rm test -m integration tests/integration

E2E tests:

docker compose run --rm test tests/e2e -m e2e

E2E dry-run reports are written to: tests/e2e/runs/<UTC timestamp>/ Open the latest run directory and check analyze/analyze-report/migration_report.html, fix-dry-run/fix-dry-run-report/migration_report.html, fix-safe/fix-safe-report/migration_report.html, or fix-unsafe/fix-unsafe-report/migration_report.html.

The main docker-compose.yml mounts ./tests to /app/tests, so reports created in the test container are available on the host immediately. No docker cp is needed.