debussy-airflow 0.0.2

Debussy Concert provider for Airflow

Debussy Airflow

A set of Airflow Hooks, Operators and Sensors for Debussy Concert,

the core component of Debussy Framework

⚠️Not exactly a provider

This package provides operators, sensors, and hooks that enables running Debussy Concert on Apache Airflow. However, it's not a traditional Airflow provider package in the sense Debussy Concert is not a tool, but rather a framework for data pipeline development and code generation. Therefore, our hooks, operators and sensors acts as interfaces, extending other Airflow providers, and abstracting their usage.

Installation

Prerequisites

An environment running apache-airflow with the following requirements:

- Python Version >= 3.7
- Airflow Version >= 2.0.0
- apache-airflow-providers-google == 8.0.0
- facebook-business == 13.0.0
- mysql-connector-python == 8.0.24
- paramiko == 2.8.1

Modules

Hooks

• db_api_hook
• facebook_ads_hook
• http_api_hook
• http_hook
• storage_hook

Operators

• basic_operator
• db_to_storage_operator
• facebook_ads_to_storage_operator
• http_operator
• rest_api_to_storage_operator
• storage_to_rdbms_operator
• storage_to_storage_operator

Sensors

• http_sensor

Examples

See the examples directory for an example DAG.

Issues

Please submit issues and pull requests in our official repo: https://github.com/DotzInc/debussy_airflow

We are happy to hear from you. Please email any feedback to the authors at [e-mail].

Project Contributors and Maintainers

This project is built with active contributions from:

• Lawrence Fernandes
• Nilton Duarte
• Maycon Francis

This project is formatted via black:

pip install black
black .