Skip to main content

DVT logo

DVT — Data Virtualization Tool

Any source. Any target. One SQL project.

DVT is a cross-engine data transformation CLI built on dbt. Write SQL models that JOIN tables living on different database engines — MySQL with Snowflake, Oracle with PostgreSQL, anything with anything — and DVT handles extraction, federation, and loading automatically.

{{ config(materialized='f_table') }}   -- "f" = federated

select
    c.customer_name,
    o.order_date,
    o.total_amount
from {{ source('crm', 'customers') }} c      -- lives on MySQL
join {{ source('erp', 'orders') }} o          -- lives on Snowflake
  on c.customer_id = o.customer_id
where o.order_date >= '2024-01-01'
pip install dvt-core
dvt sync       # installs every adapter + driver your profiles.yml needs
dvt run        # dbt runs what dbt can; DVT federates the rest
dvt version    # (worth running once — you'll see)

Website & full docs: getdvt.net · Quickstart · PyPI


Architecture

DVT is a wrapper around stock dbt-core, not a fork. dbt compiles the project and runs every standard model; DVT picks up the models dbt can't express — cross-engine federation models and locally-executed Python models — and runs them through its own pipeline:

flowchart TB
    CLI([dvt run / build]) --> COMPILE["dbt compile → manifest.json"]
    COMPILE --> STD["standard models<br/>table · view · incremental"]
    COMPILE --> FED["federated models<br/>f_table · f_incremental · .py"]
    STD --> DBT["stock dbt executes them<br/>on the default target"]
    FED --> HOMO{"all sources on<br/>ONE connection?"}
    HOMO -- "yes — Sling direct" --> DIRECT["whole query transpiled to the
    source engine, streamed straight
    to the target — no staging"]
    HOMO -- "no — federation" --> DECOMP["1 · DECOMPOSE — SQLGlot maps every
    table to a profiles.yml connection"]
    DECOMP --> PUSH["2 · TRANSPILE + PUSH DOWN — minimal
    per-source queries, each engine's dialect"]
    PUSH --> EXTRACT["3 · EXTRACT — Sling → Parquet, parallel"]
    EXTRACT --> COMPUTE["4 · COMPUTE — DuckDB joins locally"]
    COMPUTE --> LOAD["5 · LOAD — Sling delivers to any
    database, warehouse, or bucket"]
    DIRECT --> TARGET[("target")]
    LOAD --> TARGET
    DBT --> TARGET

The Sling direct path (0.2.5)

When every source in an f_table model lives on one connection, steps 3–5 collapse into a single hop: the whole query — joins and aggregations included — is transpiled to that engine's dialect and submitted to Sling as a custom-SQL stream loaded straight to the target. No Parquet staging, no DuckDB. The check is automatic and per-run; any failure falls back to the standard pipeline. Add a source on another engine tomorrow and the same model silently switches to the federation pipeline — the model never changes.

Execution paths

Your model Who runs it How
table / view / incremental stock dbt natively on the default target — whole-query pushdown by definition
f_table / f_incremental, sources on one connection DVT Sling direct: full query pushed to the source engine, result streamed to the target
f_table / f_incremental, sources span connections DVT decompose → per-source pushdown extraction → DuckDB compute → load
.py models DVT locally executed; result loads to any target

Standard models that set a non-default target= are refused with guidance (dbt silently ignores the kwarg — your model would land on the wrong engine).

Design principles

  • Wrapper, not fork. DVT shells out to the dbt CLI and reads manifest.json — it never imports dbt internals. Your dbt version can move; DVT keeps working. Every dbt feature (Jinja, macros, packages, tests, snapshots, docs) works because it is dbt running it.
  • Dual-dialect by design. Federation models are written in DuckDB dialect and are transpiled per source — they never depend on any engine's syntax. Standard models compile in the default target's dialect via the official dbt adapter. Result: switching your default target (e.g. PostgreSQL → Databricks) is a profile change, not a rewrite.
  • Pushdown first. The fastest row to move is the one you never extract. Filters and column pruning push down to source engines; when every table in a model lives on one engine, the entire query pushes down and nothing moves at all.
  • Your hardware does the joining. Cross-engine compute happens in DuckDB on the machine running DVT — source databases only ever serve simple filtered SELECTs, and no warehouse bills you for the join.

What DVT adds on top of dbt

Capability How
Cross-engine models materialized='f_table' / 'f_incremental' (long aliases: federated_*, federation_*)
Incremental federation watermark-based delta extraction; persistent local cache between runs
Per-model targets config(target='other_output') on federated models — results land on any profile output (standard models are refused a foreign target)
Bucket targets materialize models to S3 / GCS / Azure as Parquet or CSV
Python models, locally dbt-style .py models run on your machine (APIs, pandas, ML) — results land on any target
Bulk seeds dvt seed loads csv / tsv / parquet / json / jsonl / avro / arrow / xlsx via Sling to any engine, with enforced snake_case columns
One-command setup dvt sync installs every adapter and driver your profiles.yml requires
Connection diagnostics dvt debug --all checks every connection on both layers (dbt adapter + Sling)
Cross-engine catalog dvt docs builds one catalog + lineage UI across all engines in your project

Everything else — profiles.yml in ~/.dbt/, sources.yml, ref/source/config, tests, snapshots — is standard dbt. Existing dbt projects work unchanged; sources gain an optional meta.connection to bind them to other engines.

Supported engines

Every engine supported by both dbt and Sling is a DVT adapter — as federation source, federation target, seed target, and default target:

PostgreSQL · MySQL · MariaDB · SQL Server · Oracle · SQLite · DuckDB · Snowflake · BigQuery · Redshift · Databricks · ClickHouse · Trino · Athena · Microsoft Fabric — plus S3, GCS, and Azure Blob as bucket storages.

Anything else (MongoDB, Elasticsearch, REST APIs, …) is reachable through locally-executed Python models. Details per adapter: getdvt.net/docs/adapters.

Learn more

License

DVT is distributed as compiled wheels on PyPI. Free tier covers local files, transactional databases, and Python models; Pro adds cloud buckets (S3/GCS/Azure); Max adds cloud warehouses. See getdvt.net/pricing.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

dvt_core-0.2.35-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (21.1 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.35-cp314-cp314-macosx_11_0_arm64.whl (3.5 MB view details)

Uploaded CPython 3.14macOS 11.0+ ARM64

dvt_core-0.2.35-cp314-cp314-macosx_10_15_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.14macOS 10.15+ x86-64

dvt_core-0.2.35-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (21.4 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.35-cp313-cp313-macosx_11_0_arm64.whl (3.5 MB view details)

Uploaded CPython 3.13macOS 11.0+ ARM64

dvt_core-0.2.35-cp313-cp313-macosx_10_13_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.13macOS 10.13+ x86-64

dvt_core-0.2.35-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (21.5 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.35-cp312-cp312-macosx_11_0_arm64.whl (3.5 MB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

dvt_core-0.2.35-cp312-cp312-macosx_10_13_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.12macOS 10.13+ x86-64

dvt_core-0.2.35-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (22.3 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.35-cp311-cp311-macosx_11_0_arm64.whl (3.5 MB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

dvt_core-0.2.35-cp311-cp311-macosx_10_9_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.11macOS 10.9+ x86-64

dvt_core-0.2.35-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (21.0 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.35-cp310-cp310-macosx_11_0_arm64.whl (3.6 MB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

dvt_core-0.2.35-cp310-cp310-macosx_10_9_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.10macOS 10.9+ x86-64

File details

Details for the file dvt_core-0.2.35-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.35-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 d22c2ea4989348157bd6af5508ce5d3f3bea5c9b9ab2b3c4e78384db37e3adae
MD5 a9b3b2e627d19a3dea2b80055bd3770a
BLAKE2b-256 3f71658919e8cecaff89224e5c88294e944e92217bf9fe0517d69059f118ccb6

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.35-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.35-cp314-cp314-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.35-cp314-cp314-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 9f5b5f7ace6d13bb5e65484a5fb7b39c17a8b56cc511b2bc624ce8b221a7fc20
MD5 a7c45d1d29d8a68e1d1b2980126a497e
BLAKE2b-256 0149b48cbccbbd4e2f8cf153ffe03efd1ca83bdc588baeb064e8e382bba4e360

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.35-cp314-cp314-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.35-cp314-cp314-macosx_10_15_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.35-cp314-cp314-macosx_10_15_x86_64.whl
Algorithm Hash digest
SHA256 81aae41307cd9b7f8c63c22e2c4069c9272a26885466d33630949255ad37cfc1
MD5 42d02bd959946c85195eb1cbfdccc4c9
BLAKE2b-256 901cea218909932ec1d2c2bb82c49653b8a8a3e3051d5f502e4025f339cfa923

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.35-cp314-cp314-macosx_10_15_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.35-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.35-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 3fc233b9119e9f8ff5900b11d693d788852f9197383c6b93e9e7ad6dd62e5ac4
MD5 d5f9956a58fb73a3708f51e4679c9acf
BLAKE2b-256 5ff5694f6c6c11e8e29f6e3892a7f886f40c979806f894c4d0b954571e66b90f

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.35-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.35-cp313-cp313-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.35-cp313-cp313-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 3d7e26389c21b01009f44d889a2a59c529f002f5764a8ee610a036b1208e27bb
MD5 1aa0416a13c313cab12042b2e3b33801
BLAKE2b-256 1e1a9c22c39504fa76b027cdeb0998dafbb85da2e47ce1ea0eee15bd2d9adfa3

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.35-cp313-cp313-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.35-cp313-cp313-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.35-cp313-cp313-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 40b1dc54c67b9fed48438a3928e193cbc92b19afd383283840dbd635cd7cc022
MD5 ade98ee92e7db764f74480a34a32a7f0
BLAKE2b-256 493e9f3725dc845199a251afb0f2e7d26b20112bd228b11bf2caa2b19b8d1c40

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.35-cp313-cp313-macosx_10_13_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.35-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.35-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 8bef9e0fba4dd3ccdc248d30f5a1b338fae6db93a9abf08f9b47bc7d069fc14f
MD5 3925dd147db4219b49c3c3b5f649c086
BLAKE2b-256 36b5183577dbe76010480fec9ae3d914fad9374d989f212800ff133d2125d30b

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.35-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.35-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.35-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 efdacf8adee640b88c27f3ffa36cd24d3d628d6565541c6e1e4e2c7a674baacc
MD5 61be3e9cb476175dea8ece8b456ab4e8
BLAKE2b-256 303d1a294e8cdc3cb050efc8eff5cae8ec8709cd9443f106d76ff660f1d24b1b

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.35-cp312-cp312-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.35-cp312-cp312-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.35-cp312-cp312-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 ff781833eaa99f2e90c9dc37d65eaa1206f0aa172e48660c6e214a0854ee466c
MD5 3f09b41b4719804565c698aa4aa20d6d
BLAKE2b-256 5249bd3cce87ddfbe28d2ece2f68bc19941b682eb5803c35a39de027de208673

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.35-cp312-cp312-macosx_10_13_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.35-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.35-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 ab79f504172e124143ad79130cd1eb457da33b8d60fdfac75b4348ba6d963c6d
MD5 50d6e482a3823caf855211a13b49c182
BLAKE2b-256 eadacddcd821ed7d69c15e26a8cb94de8c338814b628db3d4eda52d9a4821726

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.35-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.35-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.35-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 d514ecfb22f3bf6b7d169c80cb7b2990308fff3fb1cc0502d891bdee6ed689a8
MD5 4d1408d1c28172b238f2c2edde68d70e
BLAKE2b-256 f09223a62a7cfd271b5c0948919b410b9222e64db16faf7fbc36a6e3b8dc9dfb

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.35-cp311-cp311-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.35-cp311-cp311-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.35-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 fbb7b31d036c41f6acd5501f0e64ff1dd3c898a2fa43f057638bf84282f470a9
MD5 0b5a2b70e46e9ffc311e6d2ddbed0198
BLAKE2b-256 596c4cc219169aee6865e9f6291562cc7e1c9a085c70826b359be3b9c3e74e7a

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.35-cp311-cp311-macosx_10_9_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.35-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.35-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 86a1f9b8089c5f060c79baa344a8f9299c9ffe2d533db3a514c6855b7cf8893b
MD5 0f8158d7e18dc424acc539100c3a2034
BLAKE2b-256 3bc851129fc038d25e72e1a5a954a8aac9d6a38f58f13934ca638ea91fc6a977

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.35-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.35-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.35-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 23386966cd123633ceaef8ea7c0c07a50644d856dff0cd49d838a097e38813a0
MD5 148c29446256ed4da61f95cb0db57e53
BLAKE2b-256 910f63c523bb362aba840bfb74b3bd293c0913f12914e3aacf1dbde91cb487b0

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.35-cp310-cp310-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.35-cp310-cp310-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.35-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 a47016765eeed3dfeee41689b0be089e9f2768b3027d06c203334b94d5ba6347
MD5 4dce0700789f618b6b7bca2e6f5724f4
BLAKE2b-256 b7ab8dd0f35016159f7ad8568ec2cd0ce0b71e8c4c3c4909086dd92d9146cf60

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.35-cp310-cp310-macosx_10_9_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.2.52

15 files

0.2.51

15 files

0.2.50

15 files

0.2.40

15 files

0.2.39

15 files

0.2.38

15 files

0.2.37

15 files

0.2.36

15 files

This release

0.2.35 This release

15 files

0.2.34

15 files

0.2.33

15 files

0.2.32

15 files

0.2.31

15 files

0.2.30

15 files

0.2.29

15 files

0.2.28

15 files

0.2.27

15 files

0.2.26

15 files

0.2.25

15 files

0.2.24

15 files

0.2.23

15 files

0.2.22

15 files

0.2.21

15 files

0.2.20

15 files

0.2.19

15 files

0.2.18

15 files

0.2.17

15 files

0.2.16

15 files

0.2.15

15 files

0.2.14

15 files

0.2.13

15 files

0.2.12

15 files

0.2.11

15 files

0.2.10

15 files

0.2.9

19 files

0.2.8

9 files

0.2.7

9 files

0.2.5

9 files

0.2.4

9 files

0.2.3

9 files

0.2.2

9 files

0.2.1

9 files

0.2.0

9 files

0.1.54

10 files

0.1.53

10 files

0.1.52

10 files

0.1.51

10 files

0.1.50

10 files

0.0.1

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page