Enforce Best Practices for all your Airflow DAGs. ⭐
Project description
airflint
Enforce Best Practices for all your Airflow DAGs. ⭐
⚠️ airflint is still in alpha stage and has not been tested with real world Airflow DAGs. Please report any issues you face via GitHub Issues, thank you. 🙏
🧑🏫 Rules
- Use function-level imports instead of top-level imports[^1][^2] (see Top level Python Code)
- Use jinja template syntax instead of
Variable.get
(see Airflow Variables)
[^1]: There is a PEP for Lazy Imports targeted to arrive in Python 3.12 which would supersede this rule.
[^2]: To remove top-level imports after running UseFunctionLevelImports
rule, use a tool such as autoflake.
based on official Best Practices
Requirements
airflint is tested with:
Main version (dev) | Released version (0.3.1-alpha) | |
---|---|---|
Python | 3.9, 3.10, 3.11.0-alpha - 3.11.0 | 3.9, 3.10 |
Apache Airflow | >= 2.0.0 | >= 2.3.0 |
🚀 Get started
To install it from PyPI run:
pip install airflint
NOTE: It is recommended to install airflint into your existing airflow environment with all your providers included. This way
UseJinjaVariableGet
rule can detect alltemplate_fields
and airflint works as expected.
Then just call it like this:
pre-commit
Alternatively you can add the following repo to your pre-commit-config.yaml
:
- repo: https://github.com/feluelle/airflint
rev: v0.3.1-alpha
hooks:
- id: airflint
args: ["-a"] # Use -a to apply the suggestions
additional_dependencies: # Add all package dependencies you have in your dags, preferable with version spec
- apache-airflow
- apache-airflow-providers-cncf-kubernetes
To complete the UseFunctionlevelImports
rule, please add the autoflake
hook after the airflint
hook, as below:
- repo: https://github.com/pycqa/autoflake
rev: v1.4
hooks:
- id: autoflake
args: ["--remove-all-unused-imports", "--in-place"]
This will remove unused imports.
❤️ Contributing
I am looking for contributors who are interested in..
- testing airflint with real world Airflow DAGs and reporting issues as soon as they face them
- optimizing the ast traversing for existing rules
- adding new rules based on best practices or bottlenecks you have experienced during Airflow DAGs authoring
- documenting about what is being supported in particular by each rule
- defining supported airflow versions i.e. some rules are bound to specific Airflow features and version
For questions, please don't hesitate to open a GitHub issue.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for airflint-0.3.2a0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6856ec403f1b053645b76fb4ce594239707c7e22179316eeabc73a335129135b |
|
MD5 | 65ce7c1d504f6ac6c1a860f055b06222 |
|
BLAKE2b-256 | fc06669a5abc62b585dc8404a63ff917a936ca7f8d6531fb55195d7e8dd63445 |