airflow-provider-skaledata
SkaleData's Airflow extensions. Pre-installed in
ghcr.io/skaledata/airflow — if
you're using the SkaleData base image you don't need to do anything.
If you're running your own Airflow image:
pip install airflow-provider-skaledata
The PyPI distribution is airflow-provider-skaledata (matches the Apache
community-provider naming convention). Imports live under the
skale.providers.* namespace.
Namespace
Mirrors Airflow's own provider layout
(airflow.providers.<name>.{hooks,operators,triggers}.<name>) under
skale.providers.*:
from skale.providers.airbyte.hooks.airbyte import AirbyteHook
from skale.providers.airbyte.operators.airbyte import AirbyteTriggerSyncOperator
from skale.providers.airbyte.sensors.airbyte import AirbyteJobSensor
from skale.providers.airbyte.triggers.airbyte import AirbyteSyncTrigger
Subsequent SkaleData providers (skale.providers.<other>...) slot in here.
What's in it today
skale.providers.airbyte — bearer-auth shim for managed Airbyte
Drop-in replacements for the upstream Airbyte provider's hook, operator,
sensor, and trigger that authenticate via a static bearer token (your SkaleData sdk_* API
key) instead of the upstream OAuth2 /applications/token flow.
SkaleData ships its managed Airbyte with global.auth.enabled: false. The Caddy
ingress in front of Airbyte validates the API key at the edge, so the standard
apache-airflow-providers-airbyte connector — which only supports OAuth2 client
credentials or no-auth — can't talk to it.
Airflow connection setup
| Field | Value |
|---|---|
| Conn Type | Airbyte |
| Host | https://<cluster>.skaledata.run/api/public/v1/ |
| Password (client secret) | sdk_... (your SkaleData API key) |
| Login / Token URL | leave blank |
| Extra (optional) | {"timeout": 60} — seconds per API request; defaults to 30 (airbyte-api 1.x) |
DAG usage
from datetime import datetime
from airflow.decorators import dag
from skale.providers.airbyte.operators.airbyte import AirbyteTriggerSyncOperator
@dag(start_date=datetime(2026, 1, 1), schedule=None, catchup=False)
def run_airbyte_sync():
AirbyteTriggerSyncOperator(
task_id="sync_postgres_to_warehouse",
airbyte_conn_id="airbyte_default",
connection_id="<your-airbyte-connection-uuid>",
deferrable=True,
)
run_airbyte_sync()
AirbyteTriggerSyncOperator is a drop-in for the upstream
AirbyteTriggerSyncOperator — same arguments, same async/deferrable semantics.
Two differences from upstream make retries safe:
- Running syncs are reused. If Airbyte rejects the submit with a 409 because a sync is already running for the connection (for example, a retry of a try that submitted one), the operator attaches to that job instead of failing.
- Status polls tolerate transient errors.
AirbyteSyncTriggerkeeps polling through up to two consecutive failed status requests (a timeout, a 5xx) and fails only on the third.
Asynchronous (idempotent) usage
With asynchronous=True the operator returns the Airbyte job id immediately
and AirbyteJobSensor waits on it as a separate task. Because the submit and
the wait are split, a sensor retry re-checks the existing job instead of
kicking off a duplicate sync — mirroring the
upstream async example:
from datetime import datetime
from airflow.decorators import dag
from skale.providers.airbyte.operators.airbyte import AirbyteTriggerSyncOperator
from skale.providers.airbyte.sensors.airbyte import AirbyteJobSensor
@dag(start_date=datetime(2026, 1, 1), schedule=None, catchup=False)
def run_airbyte_sync_async():
trigger_sync = AirbyteTriggerSyncOperator(
task_id="trigger_airbyte_sync",
airbyte_conn_id="airbyte_default",
connection_id="<your-airbyte-connection-uuid>",
asynchronous=True,
)
wait_for_sync = AirbyteJobSensor(
task_id="wait_for_airbyte_sync",
airbyte_conn_id="airbyte_default",
airbyte_job_id=trigger_sync.output,
deferrable=True,
)
trigger_sync >> wait_for_sync
run_airbyte_sync_async()
AirbyteJobSensor supports both poke mode (default) and deferrable=True,
same as upstream.
Metadata
Release files for airflow-provider-skaledata 0.8.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| airflow_provider_skaledata-0.8.0.tar.gz | 24.2 kB | With provenance | Details for airflow_provider_skaledata-0.8.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| airflow_provider_skaledata-0.8.0-py3-none-any.whl | Python 3 | none | any | Details for airflow_provider_skaledata-0.8.0-py3-none-any.whl |
Total release size: 42.4 kB
Release files / airflow_provider_skaledata-0.8.0.tar.gz
| Download URL | airflow_provider_skaledata-0.8.0.tar.gz |
|---|---|
| Size | 24.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
0e833191b4bec715b5ca9fd28a29e8ee1f566c7d2c7ecf4bf1535f458e12ba92
|
|
BLAKE2b-256 checksum How to use checksums |
17a64b159aa030049be38a1c0f5046ed71391babc484d540284e46094d9d6b67
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 16, 2026.
Transparency logRelease files / airflow_provider_skaledata-0.8.0-py3-none-any.whl
| Download URL | airflow_provider_skaledata-0.8.0-py3-none-any.whl |
|---|---|
| Size | 18.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
1b00b398683731a91c196dd971fcbd9c9f4adc8213156acf8b3d269d73678187
|
|
BLAKE2b-256 checksum How to use checksums |
2c81be0fe461ef4fc877139f0a4bf5926ca35da2f513ac2d6b6e6758c7e00b0e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 16, 2026.
Transparency logSecurity for airflow-provider-skaledata 0.8.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for airflow_provider_skaledata-0.8.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 16, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.