dbt-hotdata

Transform data in Hotdata instant databases with dbt.

Hotdata is a managed analytics engine speaking HotSQL (standard SQL with analytics extensions — if you know Postgres, you already know most of it) with no DDL surface: tables are created by loading data, not by CREATE TABLE. This adapter embraces that. Every model runs as the Chain pattern, entirely against the API:

1. the model's compiled SELECT executes server-side,
2. the result streams back as Arrow,
3. a native load (replace / append / upsert) applies it to the managed table.

No local database engine, no driver, no version matching — pure Python over HTTPS. The same project runs unchanged on a laptop, in CI, or in a serverless function.

Contents

• Requirements
• Install
• Quickstart
• How materializations work
• Feature support
• Configuration
• How it relates to hotdata-dlt-destination
• Development
• License

Requirements

• Python 3.11+, dbt-core 1.10+ (tested through 1.12)
• A Hotdata workspace, an API key, and its workspace ID — from your Hotdata dashboard or the Hotdata CLI.

Install

pip install dbt-hotdata
# or
uv add dbt-hotdata

Quickstart

profiles.yml

my_project:
  target: dev
  outputs:
    dev:
      type: hotdata
      workspace_id: your_workspace_id
      # database_id: db_abc123   # pin after the first run — see below
      schema: public
      threads: 1

Set your API key in the environment (it's a secret; the workspace ID is routing, not a credential):

export HOTDATA_API_KEY=your_api_key

dbt_project.yml — Hotdata has no views, and dbt's default materialization is view, so set the project default to table:

models:
  my_project:
    +materialized: table

Then:

dbt run

On first run, an instant database labelled dbt is created automatically and its id is printed:

hotdata: created instant database db_abc123 (name='dbt'). Pin it for future runs by
setting database_id: db_abc123 in profiles.yml.

Instant databases are addressed by id — Hotdata database names are not unique, so a name can't identify one. Pin database_id in the profile to keep building into the same database; without it, each run creates a fresh one (useful for CI or per-branch runs — databases can be set to expire).

An incremental model

-- models/events_rollup.sql
{{ config(
    materialized='incremental',
    incremental_strategy='merge',
    unique_key='event_id'
) }}

select event_id, user_id, count(*) as touches, max(occurred_at) as last_seen
from {{ source('app', 'events') }}
{% if is_incremental() %}
where occurred_at > (select max(last_seen) from {{ this }})
{% endif %}
group by event_id, user_id

merge runs as a native server-side upsert matched on unique_key — updates matches, inserts the rest, no full-table read. append (the default strategy) adds the new rows. First runs and --full-refresh load with replace.

How materializations work

Tests, dbt show, analyses, and source freshness all run as plain SELECTs on the server. dbt docs generate builds the catalog from the managed-table API plus Arrow schema probes.

Schema evolution is additive and automatic: a model that starts producing a new column just includes it in the next load — existing data is never touched, and types can widen but never silently shrink. (on_schema_change is therefore ignored.)

SQL dialect

Write models in HotSQL. It is Postgres-familiar, so SQL written for Postgres mostly runs unchanged, and the adapter overrides the cross-database macros (dateadd, datediff, convert_timezone) where HotSQL differs. Hotdata's query API also accepts the Postgres, DuckDB, and Snowflake dialects (translated to HotSQL server-side), but this adapter always submits model SQL as native HotSQL.

Feature support

Configuration

database: stays unset — inside an instant database the SQL catalog is always literally default (relations render as "default"."schema"."table"), and the adapter rejects any other value up front.

Fields left unset in the profile resolve from the platform's own HOTDATA_* environment variables (explicit profile values always win). These are the Hotdata CLI conventions — under any orchestrator that sets them, the adapter needs no profile fields at all beyond type: hotdata. A database_id adopted from the environment is logged, since it retargets the whole build.

How it relates to hotdata-dlt-destination

hotdata-dlt-destination loads external data into Hotdata (the EL); this adapter transforms it inside Hotdata (the T). They share the same conventions — HOTDATA_API_KEY from the environment, workspace_id as a plain parameter, id-first database_id addressing, the same retry classification — and the same underlying SDK (hotdata + hotdata-framework). Point dbt at the database_id your dlt pipeline prints, add sources for the loaded tables, and build models on top.

Running after a dlt load

hotdata-dlt-destination ships a dbt bridge: after a pipeline run, one helper call executes a dbt package against the exact instant database the load just wrote — no profiles.yml to author, credentials and routing reused from the pipeline. See “Transform with dbt” in that repo's README. This adapter itself knows nothing about dlt; the bridge drives it through the HOTDATA_* environment contract above.

Development

The project uses uv for dependency management.

git clone https://github.com/hotdata-dev/dbt-hotdata.git
cd dbt-hotdata

uv sync                 # install deps (including dev group)

uv run pytest           # run the test suite (offline — no credentials needed)
uv run ruff check       # lint
uv run ruff format      # format
uv run mypy             # type-check

The test suite runs entirely offline: adapter logic is exercised against an in-memory fake client, and a real dbt parse verifies plugin registration and every macro.

License

MIT © Hotdata Inc.

Resources

• Hotdata Python SDK · hotdata-framework
• hotdata-dlt-destination
• dbt adapter documentation
• Changelog · Architecture