dbt for Apache Doris

dbt-for-apache-doris enables Python dbt Core projects to transform data in Apache Doris through the Doris MySQL protocol. It is maintained by the VeloDB community.

[!IMPORTANT] This adapter is Beta. CI covers lint, Unit tests, and package validation, but not a live Doris cluster. Current-release compatibility and VeloDB verification are pending.

Installation · Quickstart · Compatibility · dbt docs · Doris docs · Issues · Releases

Supported capabilities

Status: ✅ Supported · ❌ Not supported

Feature status and database-version compatibility are separate contracts. Supported means the documented scope is implemented and tested; explicit platform boundaries are described alongside each capability.

Materializations

dbt capabilities

Compatibility

Before production use, run Functional tests against your exact release and topology. Historical results are not a current-release compatibility promise.

Installation

Install the VeloDB-maintained distribution from PyPI:

python -m venv .venv
source .venv/bin/activate
python -m pip install "dbt-for-apache-doris==1.1.0"
dbt --version

[!WARNING] The separate dbt-doris==1.0.0 distribution on PyPI does not contain the code from this repository. Install dbt-for-apache-doris as shown above.

On Windows, create the environment with py -m venv .venv, activate it using .venv\Scripts\Activate.ps1, and run the same pip install command.

The adapter declares dbt Core and the MySQL connector as dependencies, so they are installed automatically. You do not need to install dbt Core separately or download a standalone binary.

Quickstart

Add a Doris output to ~/.dbt/profiles.yml. Keep credentials outside version control; this example reads the password from an environment variable:

doris_demo:
  target: dev
  outputs:
    dev:
      type: doris
      host: 127.0.0.1
      port: 9030
      username: root
      password: "{{ env_var('DORIS_PASSWORD') }}"
      schema: analytics
      threads: 4

On Doris, schema is a database; an optional database must match it.

Create a new doris-demo directory with a models subdirectory, then add:

# dbt_project.yml
name: doris_demo
version: 1.0.0
config-version: 2
profile: doris_demo
model-paths: ["models"]

-- models/example.sql
{{ config(materialized='table', replication_num=1) }}
select 1 as id, 'hello from dbt-for-apache-doris' as message

# models/schema.yml
version: 2
models:
  - name: example
    columns:
      - name: id
        data_tests: [not_null, unique]

replication_num=1 is only for a local single-BE Quickstart.

export DORIS_PASSWORD='<your-password>'
dbt debug
dbt build

On Windows PowerShell, set the password with $env:DORIS_PASSWORD = '<your-password>', then run the same dbt commands.

Doris-specific highlights

Incremental strategies

Without an explicit strategy, unique_key selects merge; otherwise dbt uses append. delete+insert, partial-column merge, and incremental_predicates are rejected.

Materialized views

Use dbt's standard materialized_view materialization. The adapter implements it with Doris Async MV and exposes Doris-specific refresh configuration:

{{ config(
    materialized='materialized_view',
    refresh_trigger='manual',
    wait_for_refresh=true
) }}

select order_date, sum(amount) as sales
from {{ ref('orders') }}
group by order_date

Supported lifecycles include immediate/deferred build, manual/schedule/commit refresh, task waiting, configuration changes, docs, grants, and recovery. Overlapping dbt runs against the same MV target must be serialized, and a dbt wait timeout does not cancel a submitted Doris task.

Known limitations

• Microbatch execution is serial.
• Do not run multiple dbt invocations against the same Snapshot target concurrently; serialize overlapping production jobs in the scheduler.
• Serialize overlapping dbt runs against the same MV target; a dbt wait timeout does not cancel the submitted Doris task.
• Ordinary table models create Duplicate Key tables. Aggregate Key modeling, secondary indexes, and a complete Doris table abstraction are not implemented.
• delete+insert, partial-column merge, and incremental_predicates are not supported.
• A complete External Catalog namespace is unsupported.
• SSL configuration, timeout/retry, multi-FE failover, server-side cancellation, and complete query telemetry are not implemented.
• Some Table/View/MV type changes have a short canonical-name availability window rather than a zero-downtime switch.

Development and testing

Install development dependencies, then run local checks:

python -m pip install -r dev-requirements.txt
python -m pip install -e .
make lint
make test-unit

Functional tests need a dedicated non-production cluster. Edit test/doris_test.env; use an external file for private credentials:

make test
make test DORIS_TEST_CONFIG=/secure/path/doris_test.env

Preflight records live FE/BE versions, checks replication against live BEs, and requires cross_db_test to be absent. Tests create/drop databases, relations, users, and grants, so the account needs those permissions. Never use a shared or production cluster or run Functional sessions concurrently.

python scripts/run_doris_functional_tests.py --preflight-only
python scripts/run_doris_functional_tests.py -- -k snapshot -vv

The runner records evidence about the connected cluster but does not certify a release compatibility matrix.

License

The code is licensed under Apache License 2.0. See the license, notice, and migration provenance.