Skip to main content

Adapter to dbt-core for warehouse Exasol

Project description

dbt-exasol

dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.

Please see the dbt documentation on Exasol setup for more information on how to start using the Exasol adapter.

Current profile.yml settings

dbt-exasol:
  target: dev
  outputs:
    dev:
      type: exasol
      threads: 1
      dsn: HOST:PORT
      user: USERNAME
      password: PASSWORD
      dbname: db
      schema: SCHEMA

Optional login credentials using OpenID for Exasol SaaS

OpenID login through access_token or refresh_token instead of user+password

Optional parameters

  • connection_timeout: defaults to pyexasol default
  • socket_timeout: defaults to pyexasol default
  • query_timeout: defaults to pyexasol default
  • compression: default: False
  • encryption: default: True
  • protocol_version: default: v3
  • row_separator: default: CRLF for windows - LF otherwise
  • timestamp_format: default: YYYY-MM-DDTHH:MI:SS.FF6

Known isues

>=1.8 license change

As of dbt-exasol version 1.8 we have decided to switch to Apache License from GPLv3 - to be equal to dbt-core licensing.

setuptools breaking change

Due to a breaking change in setuptools and a infected dependency from dbt-core, we need to use the following workaround for poetry install.

Using encryption in Exasol 7 vs. 8

Starting from Exasol 8, encryption is enforced by default. If you are still using Exasol 7 and have trouble connecting, you can disable encryption in profiles.yaml (see optional parameters).

Materialized View & Clone operations

In Exasol materialized views and clone operations are not suported. Default behaviour from dbt-core will fail accordingly.

Null handling in test_utils null safe handling

In Exasol empty string are NULL. Due to this behaviour and as of this pull request 7776 published in dbt-core 1.6, seeds in tests that use EMPTY literal to simulate empty string have to be handled with special behaviour in exasol. See fixture for csv in exasol__seeds__data_hash_csv for tests/functional/adapter/utils/test_utils.py::TestHashExasol.

Model contracts

The following database constraints are implemented for Exasol:

Constraint Type Status
check NOT supported
not null enforced
unique NOT supported
primary key enforced
foreign key enforced

>=1.5 Incremental model update

Fallback to dbt-core implementation and supporting strategies

  • append
  • merge
  • delete+insert

>=1.3 Python model not yet supported - WIP

Breaking changes with release 1.2.2

  • Timestamp format defaults to YYYY-MM-DDTHH:MI:SS.FF6

SQL functions compatibility

split_part

There is no equivalent SQL function in Exasol for split_part.

listagg part_num

The SQL function listagg in Exasol does not support the num_part parameter.

Utilities shim package

In order to support packages like dbt-utils and dbt-audit-helper, we needed to create the shim package exasol-utils. In this shim package we need to adapt to parts of the SQL functionality that is not compatible with Exasol - e.g. when 'final' is being used which is a keyword in Exasol. Please visit Adaopter dispatch documentation of dbt-labs for more information.

Reporting bugs and contributing code

  • Please report bugs using the issues

Releases

GitHub Releases

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dbt_exasol-1.8.0.tar.gz (48.3 kB view details)

Uploaded Source

Built Distribution

dbt_exasol-1.8.0-py3-none-any.whl (70.5 kB view details)

Uploaded Python 3

File details

Details for the file dbt_exasol-1.8.0.tar.gz.

File metadata

  • Download URL: dbt_exasol-1.8.0.tar.gz
  • Upload date:
  • Size: 48.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.2 CPython/3.11.2 Linux/6.1.0-17-amd64

File hashes

Hashes for dbt_exasol-1.8.0.tar.gz
Algorithm Hash digest
SHA256 63c1ba6271a19538aa6f775fe3a83ae1b5c0139a1ffff59f1ec99c8bc7384c0f
MD5 b77bf12739ee4e497ed709f325b76b7e
BLAKE2b-256 c8fae7cf100d115dbb023df16101d9cf80a3694188974b0990ad4519d67054b8

See more details on using hashes here.

File details

Details for the file dbt_exasol-1.8.0-py3-none-any.whl.

File metadata

  • Download URL: dbt_exasol-1.8.0-py3-none-any.whl
  • Upload date:
  • Size: 70.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.2 CPython/3.11.2 Linux/6.1.0-17-amd64

File hashes

Hashes for dbt_exasol-1.8.0-py3-none-any.whl
Algorithm Hash digest
SHA256 57ef364c2f46afd32e216e3e122bc36e71abcb6b1eaece3ee3265fe029733ed7
MD5 3b6f7ac82edd74b4bc3c7b89329df6ba
BLAKE2b-256 4b4df7ddc68261acc01131ba6c6749c84f84b42057d0520dbe531559c7a7be03

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page