python-to-yaml-dag-converter-mwaa-serverless 0.1.4

Python to Yaml Dag Converter for MWAA Serverless

Python to Yaml Dag Converter for MWAA Serverless is a command-line tool for converting Apache Airflow® Dags written in Python to dag-factory compatible Dags written in YAML.

Prerequisites

• Python 3.11+

Benefits

• Onboard existing Python Dags to declarative YAML
• Ensures compatibility with DagFactory 1.0
• Convert Python Dags to MWAA Serverless compatible YAML
• Upload converted YAML to S3

⚡ Quick Start

Step 1: Install

pip install python-to-yaml-dag-converter-mwaa-serverless

Step 2: Use

dag-converter convert <python-dag-file>

Call with the --help flag for more options

Step 3: Review

Review the converted Dags in the output_yaml folder in the current directory

Options

• --help Show help message and exit
• --output: PATH Path to output converted YAML Dag(s) to [default: output_yaml]
• --bucket: TEXT S3 bucket to upload converted Dags to. Uses local AWS credentials
• --validate/--no-validate: Validate the output YAML using DagFactory
• --debug/--no-debug Enable logging DagBag objects to .initial_task_attributes before and after conversion

Limitations

• Only AWS operators and AWS provider packages are supported
• Behavior that is not supported in dag-factory will not be converted
• Dynamic task mapping conversion is not supported

Examples

• See supported examples
• Try one out

dag-converter convert airflow_s3.py

Contributing and Bug Reporting

See CONTRIBUTING for more information.

Security

See CONTRIBUTING for more information.

License

This project is licensed under the Apache-2.0 License.