dbt-embrasure

Initial dbt Core 1.11 adapter for the Embrasure warehouse. Your Git project and runner stay with you. Queries go through Embrasure's public API; no AWS credentials, physical Glue names, storage paths, or backend imports are needed.

Install and connect

Install the pinned adapter:

python -m pip install 'dbt-embrasure==0.1.0'

The product repository is private; customers do not need repository access. Maintainers working from a checkout can install locally:

python -m pip install ./packages/dbt-embrasure

Create a warehouse and target database in Embrasure first. Use a workspace-scoped API token with write scope (which includes read), owned by an editor or higher. Personal access tokens are disabled by default; a workspace admin must enable them before creating an expiring token through the signed-in app. The adapter also accepts a valid short-lived session bearer token but does not refresh it. The existing workspace/token policies still apply. Never commit the token.

In ~/.dbt/profiles.yml:

embrasure:
  target: dev
  outputs:
    dev:
      type: embrasure
      endpoint: https://api.embrasure.ai
      workspace_id: "{{ env_var('EMBRASURE_WORKSPACE_ID') }}"
      token: "{{ env_var('DBT_ENV_SECRET_EMBRASURE_TOKEN') }}"
      database: analytics_dev
      schema: default
      threads: 4
      query_timeout: 1800

Set profile: embrasure in dbt_project.yml. Run dbt debug, then dbt build. The warehouse API support is deployed. Use dbt Core 1.11 and Python 3.10–3.13 (Python 3.12 recommended). dbt Cloud/Fusion are not supported by this initial package.

See the customer quickstart for separate dev/prod profiles, credential rotation, and a scheduled GitHub Actions example.

Sources and environments

Use the logical identifiers shown in Embrasure's catalog:

version: 2
sources:
  - name: crm
    database: raw
    schema: salesforce
    tables:
      - name: account

{{ source('crm', 'account') }} reads raw.salesforce.account. Model outputs use analytics_dev.default.<model>. Use distinct databases for dev, CI and production; custom output schemas and schema creation/deletion are not supported. Ingestion destinations and managed-pipeline-owned outputs cannot be mutated through this API.

Supported scope

• SQL tables (managed Iceberg), views, and ephemeral models.
• CSV seeds with bounded INSERT batches, generic/singular data tests, and docs generation.
• Incremental append (default) and merge with a required unique_key.
• Standard ref, source, Jinja, packages, and SQL hooks, subject to the warehouse's supported SQL dialect and operations. Cross-engine SQL/macros may need changes.
• Query polling, cancellation, typed paginated results, and catalog discovery.

{{ config(materialized='incremental', incremental_strategy='merge', unique_key='id') }}
select id, updated_at from {{ source('crm', 'account') }}
{% if is_incremental() %}
where updated_at > (select max(updated_at) from {{ this }})
{% endif %}

Use --full-refresh for schema changes. Automatic schema evolution, snapshots, Python models, enforced contracts, SQL grants, persist_docs, relation renaming, and multi-statement transactions are not supported. on_schema_change currently accepts only ignore; incompatible writes fail in the engine. Hooks autocommit.

Table rebuilds and seeds drop/recreate their destination; they are not atomic. A failed rebuild can leave a missing or partially populated table. Views can be replaced in place. Do not run overlapping jobs against the same outputs. Seed data is written as SQL literals, so do not put secrets in seeds; SQL appears in query history.

Scheduling and failure handling

Run dbt from your CI, cron, Airflow, Dagster, or another runner that can install this package. Store the token in its secret manager. The runner owns schedules, dependencies, job concurrency, retries, and notifications. Embrasure owns query authorization, storage, catalog reconciliation, query history, and query limits.

For unattended runs, a workspace admin must create an expiring PAT through the signed-in app with write scope (no admin scope). Keep the creating user at editor or higher. Tokens apply to the workspace, not just the configured output database. Do not rely on a short-lived CLI session for a schedule: this adapter does not refresh credentials. Replace the runner secret before expiration, verify the new token in dev, let old-token jobs finish, then revoke the old token.

query_timeout defaults to 1800 seconds; the server allows at most 3600 seconds per query and retains its scan cap. The HTTP request timeout is 30 seconds. There are no automatic query-submission retries: after a lost response, inspect query history before retrying, especially for append operations. Query failures include the public query ID. Ctrl-C and local query timeouts request cancellation; server-side timeout enforcement remains in effect if the runner disappears.

Development and verification

uv sync --project packages/dbt-embrasure --frozen
uv run --project packages/dbt-embrasure --frozen pytest -q packages/dbt-embrasure/tests
uv build --project packages/dbt-embrasure

CI installs the locked package into the API test environment and runs a real dbt project through FastAPI, the logical SQL planner, and a local SQL engine replacing AWS. The example in examples/jaffle_shop is also the live smoke project:

dbt build --project-dir packages/dbt-embrasure/examples/jaffle_shop \
  --profiles-dir packages/dbt-embrasure/examples/jaffle_shop

Use a disposable target database for live tests. Local tests do not establish live Athena compatibility. Maintainers use the manual Publish dbt adapter workflow and the release runbook at docs/dbt-release.md in the product repository. It tests installed wheels before an environment-approved PyPI publication; publishing the adapter does not deploy the API.