Apache Airflow Upgrade Check
This package aims to easy the upgrade journey from Apache Airflow 1.10 to 2.0.
While we have put a lot of effort in to making this upgrade as painless as possible, with many changes providing upgrade path (where the old code continues to work and prints out a deprecation warning) there were unfortunately some breaking changes where we couldn't provide a compatibility shim.
The recommended upgrade path to get to Airflow 2.0.0 is to first upgrade to the latest release in the 1.10 series (at the time of writing: 1.10.15) and to then run this script.
pip install apache-airflow-upgrade-check
airflow upgrade_check
This will then print out a number of action items that you should follow before upgrading to 2.0.0 or above.
The exit code of the command will be 0 (success) if no problems are reported, or 1 otherwise.
For example:
============================================= STATUS =============================================
Check for latest versions of apache-airflow and checker.................................SUCCESS
Legacy UI is deprecated by default......................................................SUCCESS
Users must set a kubernetes.pod_template_file value.....................................FAIL
Changes in import paths of hooks, operators, sensors and others.........................FAIL
Remove airflow.AirflowMacroPlugin class.................................................SUCCESS
Check versions of PostgreSQL, MySQL, and SQLite to ease upgrade to Airflow 2.0..........SUCCESS
Fernet is enabled by default............................................................FAIL
Logging configuration has been moved to new section.....................................SUCCESS
Connection.conn_id is not unique........................................................SUCCESS
GCP service account key deprecation.....................................................SUCCESS
Users must delete deprecated configs for KubernetesExecutor.............................FAIL
Changes in import path of remote task handlers..........................................SUCCESS
Chain between DAG and operator not allowed..............................................SUCCESS
SendGrid email uses old airflow.contrib module..........................................SUCCESS
Connection.conn_type is not nullable....................................................SUCCESS
Found 16 problems.
======================================== RECOMMENDATIONS =========================================
Users must set a kubernetes.pod_template_file value
---------------------------------------------------
In Airflow 2.0, KubernetesExecutor Users need to set a pod_template_file as a base
value for all pods launched by the KubernetesExecutor
Problems:
1. Please create a pod_template_file by running `airflow generate_pod_template`.
This will generate a pod using your aiflow.cfg settings
...
Additionally you can use "upgrade config" to:
- specify rules you would like to ignore
- extend the check using custom rules
For example:
airflow upgrade_check --config=/files/upgrade.yaml
the configuration file should be a proper yaml file similar to this one:
ignored_rules:
- LegacyUIDeprecated
- ConnTypeIsNotNullableRule
- PodTemplateFileRule
custom_rules:
- path.to.upgrade_module.VeryCustomCheckClass
- path.to.upgrade_module.VeryCustomCheckClass2
Changelog
1.4.0
- Add
confnot importable from airflow rule (#14400) - Upgrade rule to suggest rename
[scheduler] max_threadsto[scheduler] parsing_processes(#14913) - Fix running "upgrade_check" command in a PTY. (#14977)
- Skip
DatabaseVersionCheckRulecheck if invalid version is detected (#15122) - Fix too specific parsing of
FalseinLegacyUIDeprecated(#14967) - Fix false positives when inheriting classes that inherit
DbApiHook(#16543)
1.3.0
- Fix wrong warning about class that was not used in a dag file (#14700)
- Fill DagBag from
dag_foldersetting for upgrade rules (#14588) - Bugfix: False positives for Custom Executors via Plugins check (#14680)
- Bugfix: Fix False alarm in import changes rule (#14493)
- Use
CustomSQLAInterfaceinstead ofSQLAInterface(#14475) - Fix comparing airflow version to work with older versions of packaging library (#14435)
- Fix Incorrect warning in upgrade check and error in reading file (#14344)
- Handle possible suffix in MySQL version + avoid hard-coding (#14274)
1.2.0
- Add upgrade check option to list checks (#13392)
- Add clearer exception for read failures in macro plugin upgrade (#13371)
- Treat default value in
HostnameCallablerule as good one (#13670) - Created
CustomExecutorsRequireFullPathRuleclass (#13678) - Remove
UndefinedJinjaVariableRule - Created rule for
SparkJDBCOperatorclassconn_id(#13798) - Created
DatabaseVersionCheckRuleclass (#13955) - Add Version command for Upgrade Check (#12929)
- Use Tabular Format for the List of Upgrade Check Rules (#14139)
- Fix broken
airflow upgrade_checkcommand (#14137)
Metadata
Release files for apache-airflow-upgrade-check 1.4.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| apache-airflow-upgrade-check-1.4.0.tar.gz | 40.9 kB | Without provenance | Details for apache-airflow-upgrade-check-1.4.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| apache_airflow_upgrade_check-1.4.0-py2.py3-none-any.whl | Python 3, Python 2 | none | any | Details for apache_airflow_upgrade_check-1.4.0-py2.py3-none-any.whl |
Total release size: 99.0 kB
Release files / apache-airflow-upgrade-check-1.4.0.tar.gz
| Download URL | apache-airflow-upgrade-check-1.4.0.tar.gz |
|---|---|
| Size | 40.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
9a26c4e62ae42a42c4d8e537fdc977ff47741791da6a0ebcd54360bbba95b5ba
|
|
BLAKE2b-256 checksum How to use checksums |
ae5a482651c9007eb33b0a008f14960717c7066568a68012ab81b277c9d36f70
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.4.1 importlib_metadata/4.5.0 pkginfo/1.6.1 requests/2.24.0 requests-toolbelt/0.9.1 tqdm/4.50.2 CPython/3.8.5
|
Release files / apache_airflow_upgrade_check-1.4.0-py2.py3-none-any.whl
| Download URL | apache_airflow_upgrade_check-1.4.0-py2.py3-none-any.whl |
|---|---|
| Size | 58.1 kB |
| Tags | Python 2 Python 3 |
|
SHA-256 checksum How to use checksums |
26ce6d55fc792a116729a008d2dcceac5afceaa9f11ba0a5b77d9bc22e130f77
|
|
BLAKE2b-256 checksum How to use checksums |
5975f1920610f176c227887f34a7e0bbb01b49e1ecc04a82452af8d52a3216b9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.4.1 importlib_metadata/4.5.0 pkginfo/1.6.1 requests/2.24.0 requests-toolbelt/0.9.1 tqdm/4.50.2 CPython/3.8.5
|
Security for apache-airflow-upgrade-check 1.4.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.