Skip to main content

Enforce Best Practices for all your Airflow DAGs. ⭐

Project description

airflint

pre-commit.ci status test workflow codeql-analysis workflow codecov PyPI version License PyPI - Python Version PyPI version

Enforce Best Practices for all your Airflow DAGs. ⭐

⚠️ airflint is still in alpha stage and has not been tested with real world Airflow DAGs. Please report any issues you face via GitHub Issues, thank you. 🙏

🧑‍🏫 Rules

[^1]: There is a PEP for Lazy Imports targeted to arrive in Python 3.12 which would supersede this rule.

[^2]: To remove top-level imports after running UseFunctionLevelImports rule, use a tool such as autoflake.

based on official Best Practices

Requirements

airflint is tested with:

Main version (dev) Released version (0.3.1-alpha)
Python 3.9, 3.10, 3.11.0-alpha - 3.11.0 3.9, 3.10
Apache Airflow >= 2.0.0 >= 2.3.0

🚀 Get started

To install it from PyPI run:

pip install airflint

NOTE: It is recommended to install airflint into your existing airflow environment with all your providers included. This way UseJinjaVariableGet rule can detect all template_fields and airflint works as expected.

Then just call it like this:

usage

pre-commit

Alternatively you can add the following repo to your pre-commit-config.yaml:

  - repo: https://github.com/feluelle/airflint
    rev: v0.3.1-alpha
    hooks:
      - id: airflint
        args: ["-a"]  # Use -a to apply the suggestions
        additional_dependencies:  # Add all package dependencies you have in your dags, preferable with version spec
          - apache-airflow
          - apache-airflow-providers-cncf-kubernetes

To complete the UseFunctionlevelImports rule, please add the autoflake hook after the airflint hook, as below:

  - repo: https://github.com/pycqa/autoflake
    rev: v1.4
    hooks:
      - id: autoflake
        args: ["--remove-all-unused-imports", "--in-place"]

This will remove unused imports.

❤️ Contributing

I am looking for contributors who are interested in..

  • testing airflint with real world Airflow DAGs and reporting issues as soon as they face them
  • optimizing the ast traversing for existing rules
  • adding new rules based on best practices or bottlenecks you have experienced during Airflow DAGs authoring
  • documenting about what is being supported in particular by each rule
  • defining supported airflow versions i.e. some rules are bound to specific Airflow features and version

For questions, please don't hesitate to open a GitHub issue.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

airflint-0.3.2a0.tar.gz (10.9 kB view hashes)

Uploaded Source

Built Distribution

airflint-0.3.2a0-py3-none-any.whl (12.0 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page