No project description provided
Project description
dagster-stitch
This library provides a Dagster integration for Stitch, a managed batch data ingestion service similar to Fivetran and Airbyte.
Disclaimer
Please note this library is under active development and should be used cautiously! Currently it supports triggering replication jobs and materializing the resulting tables as Dagster assets that can be used in downstream jobs. Reconciliation and other features are not yet included but would be great additions.
Installation
To install the library, run:
$ pip install dagster-stitch
For development, it can be installed locally and tested with:
$ pip install -e .[lint,test]
$ pytest
Configuration
To use the library, you must configure a stitch
resource in your Dagster instance. The resource requires a client_id
and client_secret
to authenticate with Stitch. You can find these values in the Stitch UI under Settings > API Keys
.
You will also need to note your Stitch account ID and the ID of the data source you want to replicate to be used in the asset or operation configuration. These can be found by navigating to the data source in the Stitch UI and looking at the URL. For example, if the URL is https://app.stitchdata.com/client/12345/pipeline/v2/sources/67890/summary
, then the account ID is 12345
and the data source ID is 67890
.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for dagster_stitch-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2f8c421416a46306652efa6c813f034d6cb060bc9e52976c623961daf171698e |
|
MD5 | 762c17520fbf435f6a0ca0324230d0b8 |
|
BLAKE2b-256 | 93c6bfe427aef38c4260e7c15973666406a18b05cb5c5ff4da5e9017126d98a1 |