Enforce Best Practices for all your Airflow DAGs. ⭐
Project description
airflint
Enforce Best Practices for all your Airflow DAGs. ⭐
⚠️ airflint is still in alpha stage and has not been tested with real world Airflow DAGs. Please report any issues you face via GitHub Issues, thank you. 🙏
🧑🏫 Rules
- Use function-level imports instead of top-level imports[^1][^2] (see Top level Python Code)
- Use jinja template syntax instead of
Variable.get
(see Airflow Variables)
[^1]: There is a PEP for Lazy Imports targeted to arrive in Python 3.12 which would supersede this rule.
[^2]: To remove top-level imports after running UseFunctionLevelImports
rule, use a tool such as autoflake.
based on official Best Practices
Requirements
airflint is tested with:
Main version (dev) | Released version (0.3.1-alpha) | |
---|---|---|
Python | 3.9, 3.10, 3.11.0-alpha - 3.11.0 | 3.9, 3.10 |
Apache Airflow | >= 2.0.0 | >= 2.3.0 |
🚀 Get started
To install it from PyPI run:
pip install airflint
NOTE: It is recommended to install airflint into your existing airflow environment with all your providers included. This way
UseJinjaVariableGet
rule can detect alltemplate_fields
and airflint works as expected.
Then just call it like this:
pre-commit
Alternatively you can add the following repo to your pre-commit-config.yaml
:
- repo: https://github.com/feluelle/airflint
rev: v0.3.1-alpha
hooks:
- id: airflint
args: ["-a"] # Use -a to apply the suggestions
additional_dependencies: # Add all package dependencies you have in your dags, preferable with version spec
- apache-airflow
- apache-airflow-providers-cncf-kubernetes
To complete the UseFunctionlevelImports
rule, please add the autoflake
hook after the airflint
hook, as below:
- repo: https://github.com/pycqa/autoflake
rev: v1.4
hooks:
- id: autoflake
args: ["--remove-all-unused-imports", "--in-place"]
This will remove unused imports.
❤️ Contributing
I am looking for contributors who are interested in..
- testing airflint with real world Airflow DAGs and reporting issues as soon as they face them
- optimizing the ast traversing for existing rules
- adding new rules based on best practices or bottlenecks you have experienced during Airflow DAGs authoring
- documenting about what is being supported in particular by each rule
- defining supported airflow versions i.e. some rules are bound to specific Airflow features and version
For questions, please don't hesitate to open a GitHub issue.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file airflint-0.3.2a0.tar.gz
.
File metadata
- Download URL: airflint-0.3.2a0.tar.gz
- Upload date:
- Size: 10.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.1.13 CPython/3.9.13 Linux/5.13.0-1031-azure
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 89f84564dc8169ac05838a121905c521a9a52088baf9f37f06dd2649c0d18411 |
|
MD5 | b9311fc0d11e66b008b1899bfc8cc0ea |
|
BLAKE2b-256 | c7e2a1ff24bdacc0c8446bb0fd78322336f3c2417ae7f520b7290eeb2703a367 |
File details
Details for the file airflint-0.3.2a0-py3-none-any.whl
.
File metadata
- Download URL: airflint-0.3.2a0-py3-none-any.whl
- Upload date:
- Size: 12.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.1.13 CPython/3.9.13 Linux/5.13.0-1031-azure
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6856ec403f1b053645b76fb4ce594239707c7e22179316eeabc73a335129135b |
|
MD5 | 65ce7c1d504f6ac6c1a860f055b06222 |
|
BLAKE2b-256 | fc06669a5abc62b585dc8404a63ff917a936ca7f8d6531fb55195d7e8dd63445 |