Skip to main content

Enforce Best Practices for all your Airflow DAGs. ⭐

Project description

airflint

pre-commit.ci status test workflow codeql-analysis workflow codecov PyPI version License PyPI - Python Version PyPI version

Enforce Best Practices for all your Airflow DAGs. ⭐

⚠️ airflint is still in alpha stage and has not been tested with real world Airflow DAGs. Please report any issues you face via GitHub Issues, thank you. 🙏

🧑‍🏫 Rules

[^1]: There is a PEP for Lazy Imports targeted to arrive in Python 3.12 which would supersede this rule.

[^2]: To remove top-level imports after running UseFunctionLevelImports rule, use a tool such as autoflake.

based on official Best Practices

Requirements

airflint is tested with:

Main version (dev) Released version (0.3.1-alpha)
Python 3.9, 3.10, 3.11.0-alpha - 3.11.0 3.9, 3.10
Apache Airflow >= 2.0.0 >= 2.3.0

🚀 Get started

To install it from PyPI run:

pip install airflint

NOTE: It is recommended to install airflint into your existing airflow environment with all your providers included. This way UseJinjaVariableGet rule can detect all template_fields and airflint works as expected.

Then just call it like this:

usage

pre-commit

Alternatively you can add the following repo to your pre-commit-config.yaml:

  - repo: https://github.com/feluelle/airflint
    rev: v0.3.1-alpha
    hooks:
      - id: airflint
        args: ["-a"]  # Use -a to apply the suggestions
        additional_dependencies:  # Add all package dependencies you have in your dags, preferable with version spec
          - apache-airflow
          - apache-airflow-providers-cncf-kubernetes

To complete the UseFunctionlevelImports rule, please add the autoflake hook after the airflint hook, as below:

  - repo: https://github.com/pycqa/autoflake
    rev: v1.4
    hooks:
      - id: autoflake
        args: ["--remove-all-unused-imports", "--in-place"]

This will remove unused imports.

❤️ Contributing

I am looking for contributors who are interested in..

  • testing airflint with real world Airflow DAGs and reporting issues as soon as they face them
  • optimizing the ast traversing for existing rules
  • adding new rules based on best practices or bottlenecks you have experienced during Airflow DAGs authoring
  • documenting about what is being supported in particular by each rule
  • defining supported airflow versions i.e. some rules are bound to specific Airflow features and version

For questions, please don't hesitate to open a GitHub issue.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

airflint-0.3.2a0.tar.gz (10.9 kB view details)

Uploaded Source

Built Distribution

airflint-0.3.2a0-py3-none-any.whl (12.0 kB view details)

Uploaded Python 3

File details

Details for the file airflint-0.3.2a0.tar.gz.

File metadata

  • Download URL: airflint-0.3.2a0.tar.gz
  • Upload date:
  • Size: 10.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.1.13 CPython/3.9.13 Linux/5.13.0-1031-azure

File hashes

Hashes for airflint-0.3.2a0.tar.gz
Algorithm Hash digest
SHA256 89f84564dc8169ac05838a121905c521a9a52088baf9f37f06dd2649c0d18411
MD5 b9311fc0d11e66b008b1899bfc8cc0ea
BLAKE2b-256 c7e2a1ff24bdacc0c8446bb0fd78322336f3c2417ae7f520b7290eeb2703a367

See more details on using hashes here.

File details

Details for the file airflint-0.3.2a0-py3-none-any.whl.

File metadata

  • Download URL: airflint-0.3.2a0-py3-none-any.whl
  • Upload date:
  • Size: 12.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.1.13 CPython/3.9.13 Linux/5.13.0-1031-azure

File hashes

Hashes for airflint-0.3.2a0-py3-none-any.whl
Algorithm Hash digest
SHA256 6856ec403f1b053645b76fb4ce594239707c7e22179316eeabc73a335129135b
MD5 65ce7c1d504f6ac6c1a860f055b06222
BLAKE2b-256 fc06669a5abc62b585dc8404a63ff917a936ca7f8d6531fb55195d7e8dd63445

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page