Skip to main content

DVT logo

DVT — Data Virtualization Tool

Any source. Any target. One SQL project.

DVT is a cross-engine data transformation CLI built on dbt. Write SQL models that JOIN tables living on different database engines — MySQL with Snowflake, Oracle with PostgreSQL, anything with anything — and DVT handles extraction, federation, and loading automatically.

{{ config(materialized='f_table') }}   -- "f" = federated

select
    c.customer_name,
    o.order_date,
    o.total_amount
from {{ source('crm', 'customers') }} c      -- lives on MySQL
join {{ source('erp', 'orders') }} o          -- lives on Snowflake
  on c.customer_id = o.customer_id
where o.order_date >= '2024-01-01'
pip install dvt-core
dvt sync       # installs every adapter + driver your profiles.yml needs
dvt run        # dbt runs what dbt can; DVT federates the rest
dvt version    # (worth running once — you'll see)

Website & full docs: getdvt.net · Quickstart · PyPI


Architecture

DVT is a wrapper around stock dbt-core, not a fork. dbt compiles the project and runs every standard model; DVT picks up the models dbt can't express — cross-engine federation models and locally-executed Python models — and runs them through its own pipeline:

flowchart TB
    CLI([dvt run / build]) --> COMPILE["dbt compile → manifest.json"]
    COMPILE --> STD["standard models<br/>table · view · incremental"]
    COMPILE --> FED["federated models<br/>f_table · f_incremental · .py"]
    STD --> DBT["stock dbt executes them<br/>on the default target"]
    FED --> HOMO{"all sources on<br/>ONE connection?"}
    HOMO -- "yes — Sling direct" --> DIRECT["whole query transpiled to the
    source engine, streamed straight
    to the target — no staging"]
    HOMO -- "no — federation" --> DECOMP["1 · DECOMPOSE — SQLGlot maps every
    table to a profiles.yml connection"]
    DECOMP --> PUSH["2 · TRANSPILE + PUSH DOWN — minimal
    per-source queries, each engine's dialect"]
    PUSH --> EXTRACT["3 · EXTRACT — Sling → Parquet, parallel"]
    EXTRACT --> COMPUTE["4 · COMPUTE — DuckDB joins locally"]
    COMPUTE --> LOAD["5 · LOAD — Sling delivers to any
    database, warehouse, or bucket"]
    DIRECT --> TARGET[("target")]
    LOAD --> TARGET
    DBT --> TARGET

The Sling direct path (0.2.5)

When every source in an f_table model lives on one connection, steps 3–5 collapse into a single hop: the whole query — joins and aggregations included — is transpiled to that engine's dialect and submitted to Sling as a custom-SQL stream loaded straight to the target. No Parquet staging, no DuckDB. The check is automatic and per-run; any failure falls back to the standard pipeline. Add a source on another engine tomorrow and the same model silently switches to the federation pipeline — the model never changes.

Execution paths

Your model Who runs it How
table / view / incremental stock dbt natively on the default target — whole-query pushdown by definition
f_table / f_incremental, sources on one connection DVT Sling direct: full query pushed to the source engine, result streamed to the target
f_table / f_incremental, sources span connections DVT decompose → per-source pushdown extraction → DuckDB compute → load
.py models DVT locally executed; result loads to any target

Standard models that set a non-default target= are refused with guidance (dbt silently ignores the kwarg — your model would land on the wrong engine).

Design principles

  • Wrapper, not fork. DVT shells out to the dbt CLI and reads manifest.json — it never imports dbt internals. Your dbt version can move; DVT keeps working. Every dbt feature (Jinja, macros, packages, tests, snapshots, docs) works because it is dbt running it.
  • Dual-dialect by design. Federation models are written in DuckDB dialect and are transpiled per source — they never depend on any engine's syntax. Standard models compile in the default target's dialect via the official dbt adapter. Result: switching your default target (e.g. PostgreSQL → Databricks) is a profile change, not a rewrite.
  • Pushdown first. The fastest row to move is the one you never extract. Filters and column pruning push down to source engines; when every table in a model lives on one engine, the entire query pushes down and nothing moves at all.
  • Your hardware does the joining. Cross-engine compute happens in DuckDB on the machine running DVT — source databases only ever serve simple filtered SELECTs, and no warehouse bills you for the join.

What DVT adds on top of dbt

Capability How
Cross-engine models materialized='f_table' / 'f_incremental' (long aliases: federated_*, federation_*)
Incremental federation watermark-based delta extraction; persistent local cache between runs
Per-model targets config(target='other_output') on federated models — results land on any profile output (standard models are refused a foreign target)
Bucket targets materialize models to S3 / GCS / Azure as Parquet or CSV
Python models, locally dbt-style .py models run on your machine (APIs, pandas, ML) — results land on any target
Bulk seeds dvt seed loads csv / tsv / parquet / json / jsonl / avro / arrow / xlsx via Sling to any engine, with enforced snake_case columns
One-command setup dvt sync installs every adapter and driver your profiles.yml requires
Connection diagnostics dvt debug --all checks every connection on both layers (dbt adapter + Sling)
Cross-engine catalog dvt docs builds one catalog + lineage UI across all engines in your project

Everything else — profiles.yml in ~/.dbt/, sources.yml, ref/source/config, tests, snapshots — is standard dbt. Existing dbt projects work unchanged; sources gain an optional meta.connection to bind them to other engines.

Supported engines

Every engine supported by both dbt and Sling is a DVT adapter — as federation source, federation target, seed target, and default target:

PostgreSQL · MySQL · MariaDB · SQL Server · Oracle · SQLite · DuckDB · Snowflake · BigQuery · Redshift · Databricks · ClickHouse · Trino · Athena · Microsoft Fabric — plus S3, GCS, and Azure Blob as bucket storages.

Anything else (MongoDB, Elasticsearch, REST APIs, …) is reachable through locally-executed Python models. Details per adapter: getdvt.net/docs/adapters.

Learn more

License

DVT is distributed as compiled wheels on PyPI. Free tier covers local files, transactional databases, and Python models; Pro adds cloud buckets (S3/GCS/Azure); Max adds cloud warehouses. See getdvt.net/pricing.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

dvt_core-0.2.36-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (21.9 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.36-cp314-cp314-macosx_11_0_arm64.whl (3.6 MB view details)

Uploaded CPython 3.14macOS 11.0+ ARM64

dvt_core-0.2.36-cp314-cp314-macosx_10_15_x86_64.whl (3.7 MB view details)

Uploaded CPython 3.14macOS 10.15+ x86-64

dvt_core-0.2.36-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (22.2 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.36-cp313-cp313-macosx_11_0_arm64.whl (3.6 MB view details)

Uploaded CPython 3.13macOS 11.0+ ARM64

dvt_core-0.2.36-cp313-cp313-macosx_10_13_x86_64.whl (3.7 MB view details)

Uploaded CPython 3.13macOS 10.13+ x86-64

dvt_core-0.2.36-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (22.4 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.36-cp312-cp312-macosx_11_0_arm64.whl (3.6 MB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

dvt_core-0.2.36-cp312-cp312-macosx_10_13_x86_64.whl (3.7 MB view details)

Uploaded CPython 3.12macOS 10.13+ x86-64

dvt_core-0.2.36-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (23.1 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.36-cp311-cp311-macosx_11_0_arm64.whl (3.6 MB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

dvt_core-0.2.36-cp311-cp311-macosx_10_9_x86_64.whl (3.7 MB view details)

Uploaded CPython 3.11macOS 10.9+ x86-64

dvt_core-0.2.36-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (21.8 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.36-cp310-cp310-macosx_11_0_arm64.whl (3.7 MB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

dvt_core-0.2.36-cp310-cp310-macosx_10_9_x86_64.whl (3.8 MB view details)

Uploaded CPython 3.10macOS 10.9+ x86-64

File details

Details for the file dvt_core-0.2.36-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.36-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 8dc11e3f6dc1648f0c6cd99b58ecb8d4a90679b9005186d275b875a574cfa7f5
MD5 c73806121efd6bedc8c7001275e1a2ca
BLAKE2b-256 04220ec9ba0fa8d438c572985643527ad53dd24384535d3e20bfa42ab9a705c3

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.36-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.36-cp314-cp314-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.36-cp314-cp314-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 08a17ab20cae632939d5a7c896596fb39c8215559cb6a4c7e1c5e7429ff5b60b
MD5 37cf99084b5838e1c9c16bf4ab6a1a9c
BLAKE2b-256 3df2b87de13254f4b871d35886cf893168a0a6b6117c4763ab28dff522f474eb

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.36-cp314-cp314-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.36-cp314-cp314-macosx_10_15_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.36-cp314-cp314-macosx_10_15_x86_64.whl
Algorithm Hash digest
SHA256 d322a3cb5e96d1040e56743ff9f099347857c549ca408f278733e6cf447d9ea0
MD5 2b645f09b206f58d10dd861884a62b36
BLAKE2b-256 ad5d35ab8f8528d1420a314d7c8fed5e5fdcae7b84eb59a3d27553189d64cc92

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.36-cp314-cp314-macosx_10_15_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.36-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.36-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 98fb6100a6c5203735b2eb507d0c2f0c8a021ae81a2c56586a13a4c9aeeafe01
MD5 e341c5e88fedf5d8fce6398ccc1bd047
BLAKE2b-256 943c81f9fbe2ffdf51a886fe2455022b396917f2e3e6a74c0889cd508b34172b

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.36-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.36-cp313-cp313-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.36-cp313-cp313-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 dd938797cd78420339a33f5d9180168313026dcc2321d5a088daf57766a75a28
MD5 ac12a3d250dc4999c1167faa522a1c2b
BLAKE2b-256 05851865558ae4d32f9f432a6e84fb868dc71c8661db84bd791cf40eea1d5386

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.36-cp313-cp313-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.36-cp313-cp313-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.36-cp313-cp313-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 755c7e3b0a793ac82ead01f5de29d991f668c53d2e1306070c8b5652b50bb68a
MD5 08f24ca6b3d6a1cd6a0321cfde4373e1
BLAKE2b-256 e41aca5de54ad31c0c226e58bf40dbcbbef5b91c75d702cc03186893bc0a4347

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.36-cp313-cp313-macosx_10_13_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.36-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.36-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 fa9f3c10862ca6fb92f45a43ec6cc7aa4b76a906894411cfe8d3eb2186581a13
MD5 1f4a63ba5b7ad40b4d81c81a1964e460
BLAKE2b-256 df2380943d8a769009319e7213815038cd1d60371eafa2d87623f17380313db8

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.36-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.36-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.36-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 8764f3be44468bfc5bb3803349cd1da1f874c2ef72088a229322dcfb998a853b
MD5 631af98f27793520e80f2bdec8c3ac77
BLAKE2b-256 7cef4dffbc43d156b38ab868279190582c9cfc78988f73f8db4462c3240f8198

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.36-cp312-cp312-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.36-cp312-cp312-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.36-cp312-cp312-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 4a5d1c45fae7c025d653ad937c03fd2a2660202e7cb029b2263a591edbad52e1
MD5 c6a530d0c031d38e54e6b47302e49b79
BLAKE2b-256 65cb10958d27629af59bfd4372144cbf052d18c51647eb104f0d7d111eab8444

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.36-cp312-cp312-macosx_10_13_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.36-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.36-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 82e4a83cdd0dd28cf006bdf4c2b0f33060b74aa3a7b9d12045466788aa94e2b7
MD5 73e7f80e7f2d00a82b6112f50d8ecdc4
BLAKE2b-256 e47353799122aea9f30a9e5c441d5072897122b5dedf77594c580125fdbd4c49

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.36-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.36-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.36-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 8edf553b33d71a5f4367aab9790cd5f420598e37afec4f5500131540197a129b
MD5 efaa13fbbfb5864fe2cb473e64a39176
BLAKE2b-256 edb318785bb486789b6193159744c96c9a7f5e1cbc6043047d7a7daea79ee5b9

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.36-cp311-cp311-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.36-cp311-cp311-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.36-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 880680e076687a5cf67b8354a6a4b2be88ab2ba2eededd5e1bf9215bc1719896
MD5 66e1f7f7047ae7865959bdbb55f7b5e2
BLAKE2b-256 e36521963fe64011c597d8b8f2d85510c2e850435d76332318acb5db727be0fa

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.36-cp311-cp311-macosx_10_9_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.36-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.36-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 bd8be65ad778bf132a023bad6506e6e55e2eb7f88fe7607dfbfd2712806db5dd
MD5 fd8c9c0fab1bfcc4d97ebcae3ad25d96
BLAKE2b-256 3f7bd0586a0ab33e60b3d473a9337b342ede70b569a9c60af654594a06e26ee8

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.36-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.36-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.36-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 1fe09225066bb645d32766b7c3192e89c3e9eda4ddfdf179fd5edefbd3a24e4f
MD5 521fe17d59f46c6932e821efe0626f7a
BLAKE2b-256 c6b5f519124392e3ca75a0559df29a52c0984156e6b43a23741fa8feb9382204

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.36-cp310-cp310-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.36-cp310-cp310-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.36-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 271b20faffe321e8dc077c15d5cb43de8c4a45a8f2e995c95e29bfa19ce44238
MD5 6eb6ca1b902d364f3307af64aaf380f4
BLAKE2b-256 8136b376545bd37600dee5f511109d2e4d5d847bb2a457d7612eed6babd53f2c

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.36-cp310-cp310-macosx_10_9_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.2.52

15 files

0.2.51

15 files

0.2.50

15 files

0.2.40

15 files

0.2.39

15 files

0.2.38

15 files

0.2.37

15 files

This release

0.2.36 This release

15 files

0.2.35

15 files

0.2.34

15 files

0.2.33

15 files

0.2.32

15 files

0.2.31

15 files

0.2.30

15 files

0.2.29

15 files

0.2.28

15 files

0.2.27

15 files

0.2.26

15 files

0.2.25

15 files

0.2.24

15 files

0.2.23

15 files

0.2.22

15 files

0.2.21

15 files

0.2.20

15 files

0.2.19

15 files

0.2.18

15 files

0.2.17

15 files

0.2.16

15 files

0.2.15

15 files

0.2.14

15 files

0.2.13

15 files

0.2.12

15 files

0.2.11

15 files

0.2.10

15 files

0.2.9

19 files

0.2.8

9 files

0.2.7

9 files

0.2.5

9 files

0.2.4

9 files

0.2.3

9 files

0.2.2

9 files

0.2.1

9 files

0.2.0

9 files

0.1.54

10 files

0.1.53

10 files

0.1.52

10 files

0.1.51

10 files

0.1.50

10 files

0.0.1

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page