Declarative-Manifest source connector
This is the repository for the Declarative-Manifest source connector, written in Python. The declarative manifest source connector is a special connector that can create an arbitrary source connector from a declarative manifest file. This allows users to create a source connector without writing any code.
Note: This connector is managed by the Airbyte Python CDK release process. It can be run as a standalone connector in Docker and PyAirbyte, but is not yet meant to be run in the platform as a standalone connector. This source is an interface to the low-code CDK and as such, should not be modified without a corresponding CDK change.
Local development
Prerequisites
- Python (~=3.9)
- Poetry (~=1.7) - installation instructions here
Installing the connector
From this connector directory, run:
poetry install
Create credentials
The credentials for source-declarative-manifest are a little different. Your config will need to contain the
injected declarative manifest, as indicated in the spec. It will also need to contain the fields that the spec
coming out of the manifest requires. An example is available in integration_tests/pokeapi_config.json. To use
this example in the following instructions, copy this file to secrets/config.json.
Locally running the connector
poetry run source-declarative-manifest spec
poetry run source-declarative-manifest check --config secrets/config.json
poetry run source-declarative-manifest discover --config secrets/config.json
poetry run source-declarative-manifest read --config secrets/config.json --catalog sample_files/configured_catalog.json
Running unit tests
To run unit tests locally, from the connector directory run:
poetry run pytest unit_tests
Building the docker image
- Install
airbyte-ci - Run the following command to build the docker image:
airbyte-ci connectors --name=source-declarative-manifest build
An image will be available on your host with the tag airbyte/source-declarative-manifest:dev.
Running as a docker container
Then run any of the connector commands as follows:
docker run --rm airbyte/source-declarative-manifest:dev spec
docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-declarative-manifest:dev check --config /secrets/config.json
docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-declarative-manifest:dev discover --config /secrets/config.json
docker run --rm -v $(pwd)/secrets:/secrets -v $(pwd)/integration_tests:/integration_tests airbyte/source-declarative-manifest:dev read --config /secrets/config.json --catalog /integration_tests/configured_catalog.json
Running our CI test suite
You can run our full test suite locally using airbyte-ci:
airbyte-ci connectors --name=source-declarative-manifest test
This source does not currently pass the full test suite.
Dependency Management
The manifest declarative source is built to be an interface to the low-code CDK source. This means that this source should not have any production dependencies other than the Airbyte Python CDK. If for some reason you feel that a new dependency is needed, you likely want to add it to the CDK instead. It is expected that a given version of the source-declarative-manifest connector corresponds to the same version in its CDK dependency.
Publishing a new version of the connector
New versions of this connector should only be published (automatically) via the manual Airbyte CDK release process. If you want to make a change to this connector that is not a result of a CDK change and a corresponding CDK dependency bump, please reach out to the Connector Extensibility team for guidance.
1 maintainer
Metadata
1 maintainer
Release files for airbyte-source-declarative-manifest 6.5.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| airbyte_source_declarative_manifest-6.5.2.tar.gz | 4.5 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| airbyte_source_declarative_manifest-6.5.2-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 10.2 kB
Release files / airbyte_source_declarative_manifest-6.5.2.tar.gz
| Download URL | airbyte_source_declarative_manifest-6.5.2.tar.gz |
|---|---|
| Size | 4.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
4bd831add7dd71ed3f84a71b27df40420bed4eb6e4dd99bedec579a8d33d30f7
|
|
BLAKE2b-256 checksum How to use checksums |
5383d59ada22d8c13b38a096ff22bcabe78681c188f9f363bb15ce140796072a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.8.4 CPython/3.10.15 Linux/6.5.0-1025-azure
|
Release files / airbyte_source_declarative_manifest-6.5.2-py3-none-any.whl
| Download URL | airbyte_source_declarative_manifest-6.5.2-py3-none-any.whl |
|---|---|
| Size | 5.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
edbb96bd255c43eddbee2e7d0817729d282b0fd21b5f03970a0dc7a315085002
|
|
BLAKE2b-256 checksum How to use checksums |
3270d716f76bffaf522fbb867ce853a2fb5a83e0b81a2d34c72d0150aab669fa
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.8.4 CPython/3.10.15 Linux/6.5.0-1025-azure
|
