The extrica adapter plugin for dbt (data build tool)
Project description
Extrica DBT Adapter
The dbt-extrica
adapter allows users to interact with Extrica's
Trino Engine, a distributed SQL query engine, using dbt.
This adapter is designed to facilitate the use of dbt for transforming and modeling data within Extrica.
Features
- Extrica Compatibility: Compatible with Extrica's Trino Query Engine, allowing users to leverage dbt within Extrica.
- JWT Authentication: Utilizes JWT for secure authentication with Extrica. The adapter handles the generation of JWT tokens behind the scenes via username and password configured in profiles.yml.
Description
Connecting to Multiple Data Sources
Challenges in Limited Connectivity
Default configurations in dbt make it challenging to connect to databases beyond the primary one associated with the project. Many organizations deal with diverse data sources, such as Oracle, Snowflake, SQL Server, Google BigQuery, Delta Lake, Iceberg, AWS Redshift, Vertica, Azure Synapse, MongoDB, AWS S3 and other data sources. Integrating and transforming data from these varied sources efficiently becomes essential for comprehensive analytics.
Solution: Leveraging Trino and Catalogs in Extrica
Extrica's Trino is an advanced query engine that excels in federated queries across multiple data sources and also allows the writeback capabilities to variety of data sources. Its ability to connect to various databases and process SQL queries at scale makes it an ideal solution for organizations dealing with diverse data sources.
Extrica, built on Trino introduces the concept of catalogs to address the challenge of connecting to multiple data sources seamlessly within dbt. Each catalog corresponds to a specific data source, enabling a unified approach to managing and transforming data across various systems.
Connecting to Extrica
Example profiles.yml
<profile-name>:
outputs:
dev:
type: extrica
method: jwt
username: [username for jwt auth]
password: [password for jwt auth]
host: [extrica hostname]
port: [port number]
schema: [dev_schema]
catalog: [catalog_name]
threads: [1 or more]
prod:
type: extrica
method: jwt
username: [username for jwt auth]
password: [password for jwt auth]
host: [extrica hostname]
port: [port number]
schema: [dev_schema]
catalog: [catalog_name]
threads: [1 or more]
target: dev
Description of Profile Fields
Parameter | Type | Description |
---|---|---|
type | string | Specifies the type of dbt adapter (Extrica). |
method | jwt | Authentication method for JWT authentication. |
username | string | Username for JWT authentication. The obtained JWT token is used to initialize a trino.auth.JWTAuthentication object. |
password | string | Password for JWT authentication. The obtained JWT token is used to initialize a trino.auth.JWTAuthentication object. |
host | string | The host parameter specifies the hostname or IP address of the Trino server. |
port | integer | The port parameter specifies the port number on which the Trino server is listening. |
schema | string | Schema or database name for the connection. |
catalog | string | Name of the catalog representing the data source. |
threads | integer | Number of threads for parallel execution of queries. (1 or more |
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for dbt_extrica-1.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9dd0062b18458875c4ead80f1f8ce4d90fd0cecc5a71ecc7ef5fa8ce9afa0f71 |
|
MD5 | 2a248bd96543e6c800bc0cbb32e93ad8 |
|
BLAKE2b-256 | 7158d348302c8ded3b5e2ca17fab5c48eb73128faa4fc8e2120a56b69188b5bb |