dbt-hodor

Documentation | Interactive guide with a run simulator | Design spec

dbt-hodor is a free, self-hosted alternative to dbt Labs' dbt State. It runs inside dbt. For every selected node it decides whether to reuse the object that is already in the warehouse or to build it again. A node is reused when its logic has not changed and its upstream data has not changed since the last build.

It needs no dbt platform account and sends nothing to external services. State lives in a Delta table on Databricks, in a Postgres database such as Databricks Lakebase, or in a local DuckDB file.

Status: early development. hodor runs inside dbt and passes end-to-end tests on dbt-duckdb. Tested live on a Databricks Unity Catalog workspace with a SQL warehouse and dbt-databricks 1.12.7 (2026-10-20 and 2026-10-21): reuse of tables, incremental models (including replace_where), views, seeds and snapshots, replayed tests, maintenance commits (OPTIMIZE, zero-row writes, SET TBLPROPERTIES) that keep nodes reused, and real new data or a changed seed file that rebuilds them. The live test runs twice, with use_materialization_v2 off and on, and also covers contracts with a primary key, a row filter, column masks, date-window and stamp models, and a table swapped in by use_safer_relation_operations. The same live suite passes on a second, independent Databricks workspace (2026-10-21). The postgres store is tested end to end against Databricks Lakebase. It has not run in a large production project yet. Start in shadow mode.

Why "hodor"

The name is a backronym: Hold Old Data, Or Rebuild. That is the only decision hodor makes, once per node.

hodor holds the door in front of your warehouse. A node gets through to a rebuild only when its code changed or new data arrived upstream. Everything else keeps the data it already has.

How it works

For each node in dbt run, dbt build, dbt test, dbt seed or dbt snapshot, hodor:

1. Hashes the logic. For a seed this is the CSV file. Otherwise the compiled SQL is normalized with sqlglot (Databricks dialect), so case, quoting, comments and table aliases do not count as changes. Values that differ on every run, such as invocation_id, are replaced first. The node config is hashed as well.
2. Checks the upstream data. Each upstream table has a data version. On Databricks it comes from the Delta commit history; maintenance commits such as OPTIMIZE or VACUUM do not count as data changes. On DuckDB it is a content fingerprint.
3. Decides. It compares both against the record of the node's last build and returns reuse or build, with a reason code.
4. Records. It appends one record per node to the state table.

A run with no code changes skips sqlglot entirely: when the compiled SQL is byte-identical to the last build, hodor reuses the stored analysis. For 10,000 compiled models, hashing then takes about 0.1 seconds instead of about 4 minutes.

Safety: fail open

Every failure leads to a build, never to a reuse. An error while deciding, a store that cannot be read, a table whose data version is unknown, an input hodor cannot trace (for example read_files(...)): in each case the node builds as plain dbt would. hodor never fails a dbt run.

Reason codes

The rules are checked in this order; the first match decides. dbt-hodor explain -s <model> shows the reason for one node, with the hashes and upstream data versions it compared. The full rules are in the design spec.

Modes

Installing the package never skips anything until someone sets on.

Configuration

Node configs

hodor reads the same state: block as dbt State, so a project can switch between the two. Set it in YAML:

# dbt_project.yml
models:
  my_project:
    +state:
      lag_tolerance: 4h
      require_fresh_data_from: any

pre_clone, allow_clones and defer_to_target are accepted and ignored, with a warning.

Settings

Set them on the profiles.yml target, or for one run with --vars '{hodor_mode: off}'. Precedence: environment variable, then --vars, then the profile key.

Name parts may contain letters, digits, _ and -. hodor always quotes them.

Where the state lives

delta has no default schema on purpose: hodor writes nothing to a catalog until someone names the schema. Without it, hodor turns itself off with a warning.

# profiles.yml
my_project:
  target: prod
  outputs:
    prod:
      type: databricks
      catalog: my_catalog
      # ... connection settings ...
      hodor_mode: shadow
      hodor_schema: ops
      # state table: my_catalog.ops.hodor_state
    local:
      type: duckdb
      path: dev.duckdb
      hodor_store: duckdb
      # state table: main.hodor_state in .hodor/state.duckdb

The state table decides what gets skipped in prod. Grant MODIFY on it only to the job service principals.

Scope of v1

Supported: models materialized as table, incremental (append, merge, delete+insert, insert_overwrite, replace_where; an unset strategy counts as merge) and view, seeds, snapshots, and data tests (generic and singular), on dbt-databricks 1.12.7 and dbt-duckdb, with dbt-core 1.12.x.

Model versions: each version is its own node with its own table and records. Moving latest_version rebuilds the models that ref() it.

Always built as plain dbt: clones and deferral, Python models, microbatch, materialized views, streaming tables, metric views, custom materializations, custom incremental strategies and unit tests.

Known limit: a change to dbt's or the adapter's own macros, for example a dbt-databricks upgrade, does not show in any hash. Run once with HODOR_MODE=off after such an upgrade. Project and package macros that a materialization or a hook runs are in the config hash.

Development

Requires uv. Local development and tests run on DuckDB, with no Databricks access.

uv sync
uv run pytest
uv run ruff check
uv run ruff format

The live Databricks test is opt-in: see docs/development.md.

Build a wheel with uv build. The package is not published to any index yet.

Contributing

Issues and pull requests are welcome. CONTRIBUTING.md has the setup, the checks to run, and what a decision change needs. A wrong reuse, where hodor skipped a node it should have rebuilt, is a security-grade bug: report it privately through the Security tab.

License

Apache-2.0. NOTICE carries the attribution for the derived files and the trademark notice; LICENSE is the full licence text.