sqlbuild 0.45.0

Typed, test-first SQL pipelines with local E2E testing

Verify early. Test properly. Deploy reversibly. SQL pipelines with the rigor of real software.

Valid isn't the same as correct. Your SQL compiles, runs, and returns rows; none of that means the number is right, and a silently-wrong number a stakeholder already trusted is the bug that actually hurts.

SQLBuild brings software-engineering rigor to SQL pipelines: catch errors before the warehouse runs them, test your logic locally, and only build what actually changed. It works as a standalone framework or points at your existing dbt project with no migration and no edits to your dbt files.

All state is persisted as append-only tables in the warehouse alongside your data: no external state database, no manifest files, no paid add-on. It keeps a low, dbt-like floor for SQL models and adds ingestion, Python nodes, and opt-in virtual environments as your project grows.

Key features

• Test your logic, not just your columns. Chained SQL unit tests resolve every intermediate model from its real SQL, plus end-to-end scenarios with local DuckDB replay for fast CI with no warehouse. Catch wrong logic before it ships, not just nulls.
• Verify early. Define models as SQL files with MODEL() headers. SQLBuild resolves references, validates SQL, infers columns, checks contracts, and computes column lineage before anything runs, all offline. It fails at compile, not halfway through a warehouse run.
• Fast and open static analysis. SQL parsing, validation, column inference, lineage, and transpilation run on Polyglot, a Rust SQL engine (MIT, 32+ dialects), so compile stays fast on large projects. The analysis is part of the Apache-2.0 core: no proprietary engine, no login, no paid tier.
• Audits that block bad data. Audits run before data reaches the target table. Full table builds materialize into a staging table and only promote if audits pass; incremental models validate each batch before DML.
• Deploy reversibly (opt-in). Virtual environments add instant low-copy branching, partial promotion, rollback, checkpoints, and reconciliation. Opt-in, not a tax you pay upfront.
• Build only what changed. Models, seeds, functions, and Python nodes are fingerprinted, source freshness is tracked, and unchanged work (including audits that already passed) is skipped. Pass --force to run everything selected.
• Works with your existing dbt project. Point SQLBuild at a dbt project and get change-aware builds with zero SQLBuild models. It reads the manifest and drives the dbt CLI as a subprocess; it never edits your dbt files. sqb dbt clone and sqb dbt diff work against a production-shaped git ref. See dbt compatibility.
• Warehouse-native state. All change-tracking state lives in append-only tables (_sqlbuild_fingerprints, _sqlbuild_source_freshness, _sqlbuild_node_results) in your warehouse schemas. No external state machine, no corruption risk.
• Cursor-based incremental processing. Automatic gap detection and resume, with microbatch mode for large ranges. No external checkpoint to maintain.
• Ingestion and Python nodes. Load external data with Python @loader functions, and run @task, @asset, and @check nodes as first-class members of the same DAG as your SQL models.

See the documentation for the full feature set, including providers, lifecycle hooks, Python macros, UDFs, custom materializations, data diffs, zero-copy cloning, and virtual environments.

Works with your existing dbt project

Point SQLBuild at a dbt project and run a sqb dbt command. The first time, it bootstraps a minimal twin project from your dbt_project.yml and profile (reusing your dbt connection), then builds your selection with state recorded:

sqb dbt build --select path:models/marts

Run it again and the models that have not changed are skipped:

dbt (3 selected resources)
  planned models: 0 run, 3 current, 0 blocked
  skipped: all planned dbt models are current

Change one model and only that model, plus whatever depends on it, rebuilds. SQLBuild fingerprints your dbt models in the warehouse and prunes everything that is already current, so a second build skips the whole run. Your --select scope is always respected, and where it matters SQLBuild warns you about stale upstreams or downstreams left outside the selection. See dbt compatibility.

Quick start

pip install sqlbuild
# or
uv pip install sqlbuild

Create and run the included playground project:

sqb playground waffle-shop
cd waffle-shop
sqb plan
sqb build
sqb test

Example

A model is a SQL file with a MODEL() header and a SELECT. References use __ref() and __source(), and configuration, schema, and audits are declared inline:

MODEL (
  materialized table,
  columns (
    order_id (audits [not_null, unique]),
  ),
  tags [marts],
);

SELECT
  o.order_id,
  o.customer_id,
  p.amount_cents AS total_cents
FROM __ref("stg_orders") o
JOIN __ref("stg_payments") p USING (order_id)

A unit test mocks sources and asserts on the model, resolving every intermediate model automatically:

TEST();

WITH
__source__raw__orders AS (
  @mock_orders()
),
__source__raw__payments AS (
  SELECT
    1 AS payment_id,
    1 AS order_id,
    1500 AS amount_cents,
    'credit_card' AS method
),
__expected__fact_orders AS (
  SELECT 1 AS order_id, 100 AS customer_id, 1500 AS total_cents
)
SELECT 1

See the documentation for incremental models, scenarios, loaders, and more.

Supported adapters

Adapter	Status
DuckDB	Supported
MotherDuck	Supported
Snowflake	Supported
BigQuery	Supported
Databricks	Supported
PostgreSQL	Supported
SQL Server	Supported

ClickHouse, Redshift, Trino, Spark, and Athena are on the way.

Documentation

Full documentation is available at docs.sqlbuild.com.

Contributing

We welcome contributions. Please see CONTRIBUTING.md for guidelines.

License

SQLBuild is licensed under the Apache License 2.0.