Enforce Best Practices for all your Airflow DAGs. ⭐
Project description
airflint
Enforce Best Practices for all your Airflow DAGs. ⭐
⚠️ airflint is still in alpha stage and has not been tested with real world Airflow DAGs. Please report any issues you face via GitHub Issues, thank you. 🙏
🧑🏫 Rules
- Use function-level imports instead of top-level imports[^1][^2] (see Top level Python Code)
- Use jinja template syntax instead of
Variable.get
(see Airflow Variables)
[^1]: There is a PEP for Lazy Imports targeted to arrive in Python 3.12 which would supersede this rule.
[^2]: To remove top-level imports after running UseFunctionLevelImports
rule, use a tool such as autoflake.
based on official Best Practices
🚀 Get started
To install it from PyPI run:
pip install airflint
NOTE: It is recommended to install airflint into your existing airflow environment with all your providers included. This way
UseJinjaVariableGet
rule can detect alltemplate_fields
and airflint works as expected.
Then just call it like this:
pre-commit
Alternatively you can add the following repo to your pre-commit-config.yaml
:
- repo: https://github.com/feluelle/airflint
rev: v0.3.1-alpha
hooks:
- id: airflint
args: ["-a"] # Use -a to apply the suggestions
additional_dependencies: # Add all package dependencies you have in your dags, preferable with version spec
- apache-airflow
- apache-airflow-providers-cncf-kubernetes
To complete the UseFunctionlevelImports
rule, please add the autoflake
hook after the airflint
hook, as below:
- repo: https://github.com/pycqa/autoflake
rev: v1.4
hooks:
- id: autoflake
args: ["--remove-all-unused-imports", "--in-place"]
This will remove unused imports.
❤️ Contributing
I am looking for contributors who are interested in..
- testing airflint with real world Airflow DAGs and reporting issues as soon as they face them
- optimizing the ast traversing for existing rules
- adding new rules based on best practices or bottlenecks you have experienced during Airflow DAGs authoring
- documenting about what is being supported in particular by each rule
- defining supported airflow versions i.e. some rules are bound to specific Airflow features and version
For questions, please don't hesitate to open a GitHub issue.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for airflint-0.3.1a0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 85eb411a72506df19a4cc8d61687dfbc94807fc26747866bac49432225ae1fcc |
|
MD5 | 97beeca93e34be1ef7a7d49316a77413 |
|
BLAKE2b-256 | db6ee7e21d2a8dcacbfd2d0fa6e80a3d203718046b5af18a53f6b425e669b582 |