PyPI Version PyPI - Python Version PyPI - License Code style: black CodeCov Documentation Status Security: bandit

Apache Airflow Providers containing Deferrable Operators & Sensors from Astronomer.

Installation

Install and update using pip:

pip install astronomer-providers

This only installs dependencies for core provider. To install all dependencies, run:

pip install 'astronomer-providers[all]'

To only install the dependencies for a specific provider, specify the integration name as extra argument, example to install Kubernetes provider dependencies, run:

pip install 'astronomer-providers[cncf.kubernetes]'

Extras

Extra Name

Installation Command

Dependencies

all

pip install 'astronomer-providers[all]'

All

amazon

pip install 'astronomer-providers[amazon]'

Amazon

apache.hive

pip install 'astronomer-providers[apache.hive]'

Apache Hive

apache.livy

pip install 'astronomer-providers[apache.livy]'

Apache Livy

cncf.kubernetes

pip install 'astronomer-providers[cncf.kubernetes]'

Cncf Kubernetes

databricks

pip install 'astronomer-providers[databricks]'

Databricks

google

pip install 'astronomer-providers[google]'

Google

microsoft.azure

pip install 'astronomer-providers[microsoft.azure]'

Microsoft Azure

snowflake

pip install 'astronomer-providers[snowflake]'

Snowflake

Example Usage

This repo is structured same as the Apache Airflow’s source code, so for example if you want to import Async operators, you can import it as follows:

from astronomer.providers.amazon.aws.sensors.s3 import S3KeySensorAsync as S3KeySensor

waiting_for_s3_key = S3KeySensor(
    task_id="waiting_for_s3_key",
    bucket_key="sample_key.txt",
    wildcard_match=False,
    bucket_name="sample-bucket",
)

Example DAGs for each provider is within the respective provider’s folder. For example, the Kubernetes provider’s DAGs are within the astronomer/providers/cncf/kubernetes/example_dags folder.

Principle

We will only create Async operators for the “sync-version” of operators that do some level of polling (take more than a few seconds to complete).

For example, we won’t create an async Operator for a BigQueryCreateEmptyTableOperator but will create one for BigQueryInsertJobOperator that actually runs queries and can take hours in the worst case for task completion.

Changelog

We follow Semantic Versioning for releases. Check CHANGELOG.rst for the latest changes.

Contributing Guide

All contributions, bug reports, bug fixes, documentation improvements, enhancements, and ideas are welcome.

A detailed overview on how to contribute can be found in the Contributing Guide.

As contributors and maintainers to this project, you are expected to abide by the Contributor Code of Conduct.

Goals for the project

  • Our focus is on the speed of iteration and development in this stage of the project and so we want to be able to quickly iterate with our community members and customers and cut releases as necessary

  • Airflow Providers are separate packages from the core apache-airflow package and we would like to avoid further bloating the Airflow repo

  • We want users and the community to be able to easily track features and the roadmap for individual providers that we develop

  • We would love to see the Airflow community members create, maintain and share their providers to build an Ecosystem of Providers.

License

Apache License 2.0

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (1.19.4)

Owner

PyPI data
Data sourced directly from PyPI's database.

Credits

Author: Astronomer

License

Apache Software License (Apache License 2.0)

Requires

Python >=3.7

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

astronomer-providers-1.3.1.tar.gz (99.6 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

astronomer_providers-1.3.1-py3-none-any.whl (148.0 kB view details)

Uploaded Python 3

File details

Details for the file astronomer-providers-1.3.1.tar.gz.

File metadata

  • Download URL: astronomer-providers-1.3.1.tar.gz
  • Upload date:
  • Size: 99.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.0 CPython/3.9.12

File hashes

Hashes for astronomer-providers-1.3.1.tar.gz
Algorithm Hash digest
SHA256 f6ef1f3244aac44aea032060030415a8e04e0c05673416d698e980b7fd1386ee
MD5 955572156cb71bb285532e779db4ebdb
BLAKE2b-256 d0d85e69c3d301afa20c0ce24db3bb348234edbf48033d08fb23a129ead40842

See more details on using hashes here.

File details

Details for the file astronomer_providers-1.3.1-py3-none-any.whl.

File metadata

File hashes

Hashes for astronomer_providers-1.3.1-py3-none-any.whl
Algorithm Hash digest
SHA256 32514297e91202775237bc3a68401b728d3943157a0db167f939b5d4155ece17
MD5 a9900d0de80571cf7586bf531f3f9f3e
BLAKE2b-256 c04b51745385c1cee712423f1084d7b2bcc3d5df440fe44790c681583a768edb

See more details on using hashes here.

Release history Release notifications | RSS feed

1.19.4

2 files

1.19.3

2 files

1.19.2

2 files

1.19.1

2 files

1.19.0

2 files

1.18.4

2 files

1.18.3

2 files

1.18.2

2 files

1.18.1

2 files

1.18.0

2 files

1.17.3

2 files

1.17.2

2 files

1.17.1

2 files

1.17.0

2 files

1.16.0

2 files

1.15.5

2 files

1.15.4

2 files

1.15.3

2 files

1.15.2

2 files

1.15.1

2 files

1.15.0

2 files

1.14.0

2 files

1.13.0

2 files

1.12.0

2 files

1.11.2

2 files

1.11.1

2 files

1.11.0

2 files

1.10.0

2 files

1.9.0

2 files

1.8.1

2 files

1.8.0

2 files

1.7.1

2 files

1.7.0

2 files

1.6.0

2 files

1.5.0

2 files

1.4.0

2 files

This release

1.3.1 This release

2 files

1.3.0

2 files

Pre-release

1.3.0b1

2 files

1.2.0

2 files

1.1.0

2 files

Pre-release

1.1.0a2

2 files

Pre-release

1.1.0a1

2 files

1.0.0

1 file

Pre-release

1.0.0a2

1 file

Pre-release

1.0.0a1

1 file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page