Skip to main content

spswarehouse_airflow

This package is a clone of spswarehouse, adapted to be used in Airflow. Rather than using a credentials.py file, it pulls credentials from Airflow's Connections.

Usage

Snowflake Access

The main difference in this packages is that you do not import Warehouse directly from the package. Instead, you will import the function create_warehouse, and then call that function to create the warehouse connection when you need it. After completing the Warehouse work, you need to manually close the Warehouse connection.

At the top of your DAG file:

from spswarehouse_airflow.warehouse import create_warehouse
from spswarehouse_airflow.table_utils import *

In a function that is fed to a PythonOperator:

Warehouse = create_warehouse()
<Whatever it is you need to do in the warehouse>
Warehouse.close()

Google Access

Similar to Snowflake, rather than importing GoogleSheets or GoogleDrive, import the relevant create function.

from spswarehouse_airflow.googlesheets import create_sheets
from spswarehouse_airflow.googledrive import create_drive
from spswarehouse_airflow.googleslides import create_slides
GoogleSheets = create_sheets()
GoogleDrive = create_drive()
GoogleSlides = create_slides()

Default service account e-mail to share your files with is airflow@dops-special-projects.iam.gserviceaccount.com

If you are not using google_cloud_default as the connection name for where you are storing the credentials, pass the conn_id to the create function. E.g., GoogleSheets = create_sheets(custom_conn_id)

Metadata

Release files for spswarehouse-airflow 0.7.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for spswarehouse-airflow 0.7.1
File Size Uploaded Provenance Actions
spswarehouse_airflow-0.7.1.tar.gz 22.9 kB Without provenance Details for spswarehouse_airflow-0.7.1.tar.gz

Release files / spswarehouse_airflow-0.7.1.tar.gz

Download URL spswarehouse_airflow-0.7.1.tar.gz
Size 22.9 kB
Tags Source
SHA-256 checksum
How to use checksums
e48dc2818109dc002194bfe610b576a826b476c544e0ae77cfa442b8dd7eaae2
BLAKE2b-256 checksum
How to use checksums
d2956fd7887c9472967c9a59f4c8268dfb8b12760834f821a1e30cdad7293201
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.0.0 CPython/3.9.18

Security for spswarehouse-airflow 0.7.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page