An Apache Airflow provider concerning BigQuery reservation.
Project description
Airflow BigQuery Reservation Provider
Warning This package is a pre-released of the official apache-airflow-providers-google package. All of these operators will be integrated to the official package, soon.
This repository provides an Apache Airflow provider based on BigQuery Reservation API.
Airflow Operators
BigQueryReservationCreateOperator: Buy BigQuery slots (commitments) and assign them to a GCP project (reserve and assign).BigQueryReservationDeleteOperator: Delete BigQuery commitments and remove associated ressources (rservation and assignment).BigQueryBiEngineReservationCreateOperator: Create or Update a BI engine reservation.BigQueryBiEngineReservationDeleteOperator: Delete or Update a BI engine reservation.
You could find DAG samples here.
Requirements
- A Google Cloud connection has to be defined.
By default, all hooks and operators use
google_cloud_default. - This connection requires the following roles on the Google Cloud project(s) used in these operators:
- BigQuery Resource Admin
- BigQuery Job User - Required for
BigQueryReservationCreateOperatorbecause of the reservation attachment check.
Defining a new dedicated connection and custom GCP role could be good practices to respect the principle of least privilege.
How to install
pip install --user airflow-provider-bigquery-reservation
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file airflow_provider_bigquery_reservation-1.1.1-py3-none-any.whl.
File metadata
- Download URL: airflow_provider_bigquery_reservation-1.1.1-py3-none-any.whl
- Upload date:
- Size: 16.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.8
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
9b5cacba58eeb927947d9bcb66900002d2358f95965a26116a50164e0be5e053
|
|
| MD5 |
4a4b25aeabf1dfa8fb8ffb69b0f53086
|
|
| BLAKE2b-256 |
6ce41e0fb487177f6e84b2299a5de1bf00d2b266325de8b82cf3f71140dac9db
|
