Skip to main content

Connector Development Kit (Python)

The Airbyte Python CDK is a framework for rapidly developing production-grade Airbyte connectors. The CDK currently offers helpers specific for creating Airbyte source connectors for:

  • HTTP APIs (REST APIs, GraphQL, etc..)
  • Singer Taps
  • Generic Python sources (anything not covered by the above)

The CDK provides an improved developer experience by providing basic implementation structure and abstracting away low-level glue boilerplate.

This document is a general introduction to the CDK. Readers should have basic familiarity with the Airbyte Specification before proceeding.

Getting Started

Generate an empty connector using the code generator. First clone the Airbyte repository then from the repository root run

cd airbyte-integrations/connector-templates/generator
./generate.sh

then follow the interactive prompt. Next, find all TODOs in the generated project directory -- they're accompanied by lots of comments explaining what you'll need to do in order to implement your connector. Upon completing all TODOs properly, you should have a functioning connector.

Additionally, you can follow this tutorial for a complete walkthrough of creating an HTTP connector using the Airbyte CDK.

Concepts & Documentation

See the concepts docs for a tour through what the API offers.

Example Connectors

HTTP Connectors:

Singer connectors:

Simple Python connectors using the barebones Source abstraction:

Contributing

First time setup

We assume python points to python >=3.9.

Setup a virtual env:

python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]" # [dev] installs development-only dependencies

Iteration

  • Iterate on the code locally
  • Run tests via pytest -s unit_tests
  • Perform static type checks using mypy airbyte_cdk. MyPy configuration is in .mypy.ini.
  • The type_check_and_test.sh script bundles both type checking and testing in one convenient command. Feel free to use it!

Testing

All tests are located in the unit_tests directory. Run pytest --cov=airbyte_cdk unit_tests/ to run them. This also presents a test coverage report.

Publishing a new version to PyPi

  1. Bump the package version in setup.py
  2. Open a PR
  3. An Airbyte member must comment /publish-cdk dry-run=true to publish the package to test.pypi.org or /publish-cdk dry-run=false to publish it to the real index of pypi.org.

Coming Soon

  • Full OAuth 2.0 support (including refresh token issuing flow via UI or CLI)
  • Airbyte Java HTTP CDK
  • CDK for Async HTTP endpoints (request-poll-wait style endpoints)
  • CDK for other protocols
  • Don't see a feature you need? Create an issue and let us know how we can help!

Metadata

Release files for airbyte-cdk-PHLAIR 0.2.3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for airbyte-cdk-PHLAIR 0.2.3
File Size Uploaded Provenance Actions
airbyte-cdk-PHLAIR-0.2.3.tar.gz 155.5 kB Without provenance Details for airbyte-cdk-PHLAIR-0.2.3.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for airbyte-cdk-PHLAIR 0.2.3
File Interpreter ABI Platform Actions
airbyte_cdk_PHLAIR-0.2.3-py3-none-any.whl Python 3 none any Details for airbyte_cdk_PHLAIR-0.2.3-py3-none-any.whl

Total release size: 315.5 kB

Release files / airbyte-cdk-PHLAIR-0.2.3.tar.gz

Download URL airbyte-cdk-PHLAIR-0.2.3.tar.gz
Size 155.5 kB
Tags Source
SHA-256 checksum
How to use checksums
d6717db7c051f5b4b464ec6ad02dfb24a213f2f27baef425cf28aa0b0ebb30b5
BLAKE2b-256 checksum
How to use checksums
b2f2ee2178cf38731f458c49741c5591890f0cb63bf64c44204a448b9dfe1605
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.1 CPython/3.9.12

Release files / airbyte_cdk_PHLAIR-0.2.3-py3-none-any.whl

Download URL airbyte_cdk_PHLAIR-0.2.3-py3-none-any.whl
Size 160.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
343af6f590941704b8eff5b6c4def83253a7d75ae55765b4a7356f0d9f53c5f9
BLAKE2b-256 checksum
How to use checksums
b5c8ed383513391251701e00cd262925059c30b42b6154c23bf4a57d25fd9cd4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.1 CPython/3.9.12

Security for airbyte-cdk-PHLAIR 0.2.3

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page