SQLBuild

Verify early. Test properly. Deploy reversibly. SQL pipelines with the rigor of real software.

Valid isn't the same as correct. Your SQL compiles, runs, and returns rows; none of that means the number is right, and a silently-wrong number a stakeholder already trusted is the bug that actually hurts.

SQLBuild brings software-engineering rigor to SQL pipelines: catch errors before the warehouse runs them, test your logic locally, and opt into change-aware execution when you need it. It works as a standalone framework or points at your existing dbt project with no migration and no edits to your dbt files.

All state is persisted as append-only tables in the warehouse alongside your data: no external state database, no manifest files, no paid add-on. It keeps a low, dbt-like floor for SQL models and adds ingestion, Python nodes, and opt-in virtual environments as your project grows.

Key features

  • Test your logic, not just your columns. Multi-model SQL tests resolve every intermediate model from its real SQL, plus end-to-end scenarios with local DuckDB replay for fast CI with no warehouse. Catch wrong logic before it ships, not just nulls.
  • Verify early. Define models as SQL files with MODEL() headers. SQLBuild resolves references, validates SQL, infers columns, checks contracts, and computes column lineage before anything runs, all offline. It fails at compile, not halfway through a warehouse run.
  • Fast and open static analysis. SQL parsing, validation, column inference, lineage, and transpilation run on Polyglot, a Rust SQL engine (MIT, 32+ dialects), so compile stays fast on large projects. The analysis is part of the Apache-2.0 core: no proprietary engine, no login, no paid tier.
  • Audits that block bad data. Audits run before data reaches the target table. Full table builds materialize into a staging table and only promote if audits pass; incremental models validate each batch before DML.
  • Deploy reversibly (opt-in). Virtual environments add instant low-copy branching, partial promotion, rollback, checkpoints, and reconciliation. Opt-in, not a tax you pay upfront.
  • Opt-in change-aware execution. Models, seeds, UDFs, and Python nodes are fingerprinted, and source freshness is tracked. In virtual environments, pass --changes-only or set changes_only = true to skip work that is already current; commands otherwise run the full selected scope.
  • Works with your existing dbt project. Point SQLBuild at a dbt project and run ordinary dbt selections alongside SQLBuild models. It reads the manifest and drives the dbt CLI as a subprocess; it never edits your dbt files. dbt-native --state and --defer remain available for production-shaped, state-aware selections. See dbt compatibility.
  • Warehouse-native state. All change-tracking state lives in append-only tables (_sqlbuild_fingerprints, _sqlbuild_source_freshness, _sqlbuild_node_results) in your warehouse schemas. No external state machine, no corruption risk.
  • Cursor-based incremental processing. Automatic gap detection and resume, with microbatch mode for large ranges. No external checkpoint to maintain.
  • Ingestion and Python nodes. Load external data with Python @loader functions, and run @task, @asset, and @check nodes as first-class members of the same DAG as your SQL models.

See the documentation for the full feature set, including providers, lifecycle hooks, Python macros, UDFs, custom materializations, data diffs, zero-copy cloning, and virtual environments.

Works with your existing dbt project

Point SQLBuild at a dbt project and run a sqb dbt command. The first time, it bootstraps a minimal twin project from your dbt_project.yml and profile (reusing your dbt connection), then runs your selection through dbt:

sqb dbt build --select path:models/marts

SQLBuild preserves dbt-native state and defer arguments when you need dbt's own state-aware selection:

sqb dbt build --state path/to/state --defer --select state:modified+

Ordinary sqb dbt plan, run, and build commands do not fingerprint dbt models or inspect production state automatically. Use dbt-native --state/--defer for production-shaped comparisons. See dbt compatibility.

Quick start

pip install sqlbuild
# or
uv pip install sqlbuild

Create and run the included playground project:

sqb playground waffle-shop
cd waffle-shop
sqb plan
sqb build
sqb test

Example

A model is a SQL file with a MODEL() header and a SELECT. References use __ref() and __source(), and configuration, schema, and audits are declared inline:

MODEL (
  materialized table,
  columns (
    order_id (audits [not_null, unique]),
  ),
  tags [marts],
);

SELECT
  o.order_id,
  o.customer_id,
  p.amount_cents AS total_cents
FROM __ref("stg_orders") o
JOIN __ref("stg_payments") p USING (order_id)

A unit test mocks sources and asserts on the model, resolving every intermediate model automatically:

TEST();

WITH
__source__raw__orders AS (
  @mock_orders()
),
__source__raw__payments AS (
  SELECT
    1 AS payment_id,
    1 AS order_id,
    1500 AS amount_cents,
    'credit_card' AS method
),
__expected__fact_orders AS (
  SELECT 1 AS order_id, 100 AS customer_id, 1500 AS total_cents
)
SELECT 1

See the documentation for incremental models, scenarios, loaders, and more.

Kata SQL architecture checks

Kata is SQLBuild's opt-in, error-only SQL model shape checker. It runs offline over the compiled project, reports coded faults with remediations, and never rewrites SQL. Its built-in lifecycle is native: Rust resolves rule policy, parses each model, evaluates built-ins, applies suppressions, and owns the persistent cache and deterministic result ordering.

Select the default policy namespace and opt into additional exact rules in sqlbuild_project.toml. Prefix selectors activate default-enabled rules; exact selectors also activate opt-in rules:

[kata]
select = ["SQBK", "SQBKS001", "SQBKS201", "SQBKX001", "SQBKX002"]
ignore = ["SQBKS302"]

[kata.thresholds]
min_audits_per_model = 1
min_tests_per_model = 1

Run sqb kata, inspect metadata with sqb kata rule SQBKS101, and generate agent guidance from the same active ruleset with sqb kata skills. Use sqb kata skills --check in CI to detect stale guidance. --json, --select, and --exclude are available for automation and model scoping.

Repository rules use the public API:

from sqlbuild.kata import RuleContext, kata


@kata(
    code="XSQBKP001",
    family="prices",
    slug="typed-currency",
    message="price models must declare a currency column",
    remediation="Declare currency in the MODEL columns contract at this model path.",
)
def typed_currency(*, model, ctx: RuleContext):
    return [] if any(column.name == "currency" for column in ctx.declared_columns) else [
        ctx.path_fault()
    ]

Load repository-owned files through rule_paths = ["kata/rules"] or dotted packages through rule_modules. Test each custom rule with RuleCase and evaluate_rule. Selecting custom rules disables caching unless [kata.cache] require_cacheable = true; cacheable rules may import only the supported pure modules and must access project files through RuleContext.

Python is used only for the SQLBuild compiler adapter and selected custom rules. Built-in-only runs cross into the native engine once as a compiled model batch and do not materialize or walk Python AST objects. A selected custom rule can still use the public RuleContext and raw Polyglot AST escape hatch; its findings rejoin native suppression, ordering, and cache policy.

Exact rule_exceptions require a rule, file, and reason and fail when stale. Broader rule_ignores and lone-star allowances also require reasons but are intentionally not stale-checked.

Supported adapters

Adapter Status
DuckDB Supported
MotherDuck Supported
Snowflake Supported
BigQuery Supported
Databricks Supported
PostgreSQL Supported
SQL Server Supported

ClickHouse, Redshift, Trino, Spark, and Athena are on the way.

Documentation

Full documentation is available at docs.sqlbuild.com.

Contributing

We welcome contributions. Please see CONTRIBUTING.md for guidelines.

License

SQLBuild is licensed under the Apache License 2.0.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.83.2)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for kvlonge from gravatar.com kvlonge

Requires

Python >=3.12

Provides Extra

bigquery dagster databricks dbt dlt ingestr postgres rivers snowflake sqlserver

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sqlbuild-0.54.1.tar.gz (1.0 MB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

sqlbuild-0.54.1-cp312-abi3-win_amd64.whl (4.7 MB view details)

Uploaded CPython 3.12+Windows x86-64

sqlbuild-0.54.1-cp312-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.0 MB view details)

Uploaded CPython 3.12+manylinux: glibc 2.17+ x86-64

sqlbuild-0.54.1-cp312-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (4.9 MB view details)

Uploaded CPython 3.12+manylinux: glibc 2.17+ ARM64

sqlbuild-0.54.1-cp312-abi3-macosx_11_0_arm64.whl (4.6 MB view details)

Uploaded CPython 3.12+macOS 11.0+ ARM64

sqlbuild-0.54.1-cp312-abi3-macosx_10_12_x86_64.whl (4.8 MB view details)

Uploaded CPython 3.12+macOS 10.12+ x86-64

File details

Details for the file sqlbuild-0.54.1.tar.gz.

File metadata

  • Download URL: sqlbuild-0.54.1.tar.gz
  • Upload date:
  • Size: 1.0 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for sqlbuild-0.54.1.tar.gz
Algorithm Hash digest
SHA256 da8ae8a280e7ff9b6e2942deddc2d00c2b9cd64dbf886dd8db8a4ee5698a26a8
MD5 0915e2b01c8135cd3d947fed2476d292
BLAKE2b-256 0a3a544ca726b966d8af74017e75b20d3be193e062447c140212dab2bd6cf83e

See more details on using hashes here.

Provenance

The following attestation bundles were made for sqlbuild-0.54.1.tar.gz:

Publisher: publish.yml on chio-labs/sqlbuild

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file sqlbuild-0.54.1-cp312-abi3-win_amd64.whl.

File metadata

  • Download URL: sqlbuild-0.54.1-cp312-abi3-win_amd64.whl
  • Upload date:
  • Size: 4.7 MB
  • Tags: CPython 3.12+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for sqlbuild-0.54.1-cp312-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 36bd615d8f481becb8e1e9753b02ffb4da4f371ee851c1568862c26b47d7f3ad
MD5 80e672a5d037da86385488a1a5a5e393
BLAKE2b-256 f447a2acc13eee2d3a32fabc596ccfab8a2d46de6b908813674c25e3879200b1

See more details on using hashes here.

Provenance

The following attestation bundles were made for sqlbuild-0.54.1-cp312-abi3-win_amd64.whl:

Publisher: publish.yml on chio-labs/sqlbuild

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file sqlbuild-0.54.1-cp312-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for sqlbuild-0.54.1-cp312-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 89b0ee3ca221428eb748bce11d59475aec15cb7c3061afe87eb458bc4508f017
MD5 18dd3a3583b76399507ab44646d4f516
BLAKE2b-256 239a8295362a0402016457989b0b06168fe6cdd30709c86545bd95e2e11df19e

See more details on using hashes here.

Provenance

The following attestation bundles were made for sqlbuild-0.54.1-cp312-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: publish.yml on chio-labs/sqlbuild

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file sqlbuild-0.54.1-cp312-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for sqlbuild-0.54.1-cp312-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 e4c6b5fd4e20459618d2987e3449a0102b1a62115032daec48deae4430c73ecf
MD5 0d23adb7516572a60bec2fdb0967923d
BLAKE2b-256 9208d5e35baf96187d6aa0de0fe4d4366d8f4eab1d5bd500a772df5dc61fa8d3

See more details on using hashes here.

Provenance

The following attestation bundles were made for sqlbuild-0.54.1-cp312-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: publish.yml on chio-labs/sqlbuild

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file sqlbuild-0.54.1-cp312-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for sqlbuild-0.54.1-cp312-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 4a0019c9586bbdc9354877110807b729db5a836120701af0f81f8f09daf70927
MD5 76f806ad209fc9eb288a4e2d0259b79b
BLAKE2b-256 873ab2f4d3618012894d4dcb1e8537a0b4ebe8e932bd94c7fce7d08a2c6c4df1

See more details on using hashes here.

Provenance

The following attestation bundles were made for sqlbuild-0.54.1-cp312-abi3-macosx_11_0_arm64.whl:

Publisher: publish.yml on chio-labs/sqlbuild

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file sqlbuild-0.54.1-cp312-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for sqlbuild-0.54.1-cp312-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 f6d9b312c890bd10c08f61020014c8acfeb5efb123b449509b0ad2ce488b67f3
MD5 b40d21a6775d669585ca4c145235cae6
BLAKE2b-256 1bb8d7e862e4e9279859da66d8e61fff70b643b67753fa77ed8ec87ad75a48b9

See more details on using hashes here.

Provenance

The following attestation bundles were made for sqlbuild-0.54.1-cp312-abi3-macosx_10_12_x86_64.whl:

Publisher: publish.yml on chio-labs/sqlbuild

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.83.2

6 files

0.83.1

6 files

0.83.0

6 files

0.82.6

6 files

0.82.5

6 files

0.82.4

6 files

0.82.3

6 files

0.82.2

6 files

0.82.1

6 files

0.82.0

6 files

0.81.2

6 files

0.81.1

6 files

0.81.0

6 files

0.80.2

6 files

0.80.1

6 files

0.80.0

6 files

0.79.0

6 files

0.78.0

6 files

0.77.2

6 files

0.77.1

6 files

0.77.0

6 files

0.76.9

6 files

0.76.8

6 files

0.76.7

6 files

0.76.6

6 files

0.76.5

6 files

0.76.4

6 files

0.76.3

6 files

0.76.2

6 files

0.76.1

6 files

0.76.0

6 files

0.75.1

6 files

0.75.0

6 files

0.74.4

6 files

0.74.3

6 files

0.74.2

6 files

0.74.1

6 files

0.74.0

6 files

0.73.0

6 files

0.72.2

6 files

0.72.1

6 files

0.72.0

6 files

0.71.6

6 files

0.71.5

6 files

0.71.4

6 files

0.71.3

6 files

0.71.2

6 files

0.71.1

6 files

0.71.0

6 files

0.70.0

6 files

0.69.0

6 files

0.68.0

6 files

0.67.2

6 files

0.67.1

6 files

0.67.0

6 files

0.66.6

6 files

0.66.5

6 files

0.66.4

6 files

0.66.3

6 files

0.66.2

6 files

0.66.1

6 files

0.66.0

6 files

0.65.6

6 files

0.65.5

6 files

0.65.4

6 files

0.65.3

6 files

0.65.2

6 files

0.65.1

6 files

0.65.0

6 files

0.64.0

6 files

0.63.8

6 files

0.63.7

6 files

0.63.6

6 files

0.63.5

6 files

0.63.4

6 files

0.63.3

6 files

0.63.2

6 files

0.63.1

6 files

0.63.0

6 files

0.62.2

6 files

0.62.1

6 files

0.62.0

6 files

0.61.0

6 files

0.60.0

6 files

0.59.0

6 files

0.58.0

6 files

0.57.0

6 files

0.56.2

6 files

0.56.1

6 files

0.56.0

6 files

0.55.9

6 files

0.55.8

6 files

0.55.7

6 files

0.55.6

6 files

0.55.5

6 files

0.55.4

6 files

0.55.3

6 files

0.55.2

6 files

0.55.1

6 files

0.55.0

6 files

0.54.2

6 files

This release

0.54.1 This release

6 files

0.54.0

5 files

0.53.0

2 files

0.52.0

2 files

0.51.0

2 files

0.50.0

2 files

0.49.0

2 files

0.48.7

2 files

0.48.6

2 files

0.48.5

2 files

0.48.4

2 files

0.48.3

2 files

0.48.2

2 files

0.48.1

2 files

0.48.0

2 files

0.47.0

2 files

0.46.0

2 files

0.45.5

2 files

0.45.4

2 files

0.45.3

2 files

0.45.2

2 files

0.45.1

2 files

0.45.0

2 files

0.44.4

2 files

0.41.1

2 files

0.41.0

2 files

Yanked

0.40.1

2 files

Yanked reason: Just a shitty release

Yanked

0.40.0

2 files

Yanked reason: shitty release

0.39.3

2 files

0.39.2

2 files

0.39.1

2 files

0.39.0

2 files

0.38.6

2 files

0.38.5

2 files

0.38.4

2 files

0.38.3

2 files

0.38.2

2 files

0.38.1

2 files

0.38.0

2 files

0.37.7

2 files

0.37.6

2 files

0.37.5

2 files

0.37.4

2 files

0.37.3

2 files

0.37.2

2 files

0.37.1

2 files

0.36.0

2 files

0.35.0

2 files

0.34.0

2 files

0.33.0

2 files

0.32.0

2 files

0.31.0

2 files

0.30.1

2 files

0.30.0

2 files

0.29.0

2 files

0.28.1

2 files

0.28.0

2 files

0.27.0

2 files

0.26.2

2 files

0.26.1

2 files

0.26.0

2 files

0.25.1

2 files

0.25.0

2 files

0.24.0

2 files

0.23.0

2 files

0.22.1

2 files

0.22.0

2 files

0.21.0

2 files

0.20.1

2 files

0.20.0

2 files

0.19.0

2 files

0.18.0

2 files

0.16.2

2 files

0.16.1

2 files

0.15.0

2 files

0.14.0

2 files

0.13.0

2 files

0.12.0

2 files

0.10.0

2 files

0.9.0

2 files

0.8.0

2 files

0.7.0

2 files

0.4.0

2 files

0.3.0

2 files

0.2.1

2 files

0.2.0

2 files

0.0.1

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page