Skip to main content

DVT logo

DVT — Data Virtualization Tool

Any source. Any target. One SQL project.

DVT is a cross-engine data transformation CLI built on dbt. Write SQL models that JOIN tables living on different database engines — MySQL with Snowflake, Oracle with PostgreSQL, anything with anything — and DVT handles extraction, federation, and loading automatically.

{{ config(materialized='f_table') }}   -- "f" = federated

select
    c.customer_name,
    o.order_date,
    o.total_amount
from {{ source('crm', 'customers') }} c      -- lives on MySQL
join {{ source('erp', 'orders') }} o          -- lives on Snowflake
  on c.customer_id = o.customer_id
where o.order_date >= '2024-01-01'
pip install dvt-core
dvt sync       # installs every adapter + driver your profiles.yml needs
dvt run        # dbt runs what dbt can; DVT federates the rest
dvt version    # (worth running once — you'll see)

Website & full docs: getdvt.net · Quickstart · PyPI


Architecture

DVT is a wrapper around stock dbt-core, not a fork. dbt compiles the project and runs every standard model; DVT picks up the models dbt can't express — cross-engine federation models and locally-executed Python models — and runs them through its own pipeline:

flowchart TB
    CLI([dvt run / build]) --> COMPILE["dbt compile → manifest.json"]
    COMPILE --> STD["standard models<br/>table · view · incremental"]
    COMPILE --> FED["federated models<br/>f_table · f_incremental · .py"]
    STD --> DBT["stock dbt executes them<br/>on the default target"]
    FED --> HOMO{"all sources on<br/>ONE connection?"}
    HOMO -- "yes — Sling direct" --> DIRECT["whole query transpiled to the
    source engine, streamed straight
    to the target — no staging"]
    HOMO -- "no — federation" --> DECOMP["1 · DECOMPOSE — SQLGlot maps every
    table to a profiles.yml connection"]
    DECOMP --> PUSH["2 · TRANSPILE + PUSH DOWN — minimal
    per-source queries, each engine's dialect"]
    PUSH --> EXTRACT["3 · EXTRACT — Sling → Parquet, parallel"]
    EXTRACT --> COMPUTE["4 · COMPUTE — DuckDB joins locally"]
    COMPUTE --> LOAD["5 · LOAD — Sling delivers to any
    database, warehouse, or bucket"]
    DIRECT --> TARGET[("target")]
    LOAD --> TARGET
    DBT --> TARGET

The Sling direct path (0.2.5)

When every source in an f_table model lives on one connection, steps 3–5 collapse into a single hop: the whole query — joins and aggregations included — is transpiled to that engine's dialect and submitted to Sling as a custom-SQL stream loaded straight to the target. No Parquet staging, no DuckDB. The check is automatic and per-run; any failure falls back to the standard pipeline. Add a source on another engine tomorrow and the same model silently switches to the federation pipeline — the model never changes.

Execution paths

Your model Who runs it How
table / view / incremental stock dbt natively on the default target — whole-query pushdown by definition
f_table / f_incremental, sources on one connection DVT Sling direct: full query pushed to the source engine, result streamed to the target
f_table / f_incremental, sources span connections DVT decompose → per-source pushdown extraction → DuckDB compute → load
.py models DVT locally executed; result loads to any target

Standard models that set a non-default target= are refused with guidance (dbt silently ignores the kwarg — your model would land on the wrong engine).

Design principles

  • Wrapper, not fork. DVT shells out to the dbt CLI and reads manifest.json — it never imports dbt internals. Your dbt version can move; DVT keeps working. Every dbt feature (Jinja, macros, packages, tests, snapshots, docs) works because it is dbt running it.
  • Dual-dialect by design. Federation models are written in DuckDB dialect and are transpiled per source — they never depend on any engine's syntax. Standard models compile in the default target's dialect via the official dbt adapter. Result: switching your default target (e.g. PostgreSQL → Databricks) is a profile change, not a rewrite.
  • Pushdown first. The fastest row to move is the one you never extract. Filters and column pruning push down to source engines; when every table in a model lives on one engine, the entire query pushes down and nothing moves at all.
  • Your hardware does the joining. Cross-engine compute happens in DuckDB on the machine running DVT — source databases only ever serve simple filtered SELECTs, and no warehouse bills you for the join.

What DVT adds on top of dbt

Capability How
Cross-engine models materialized='f_table' / 'f_incremental' (long aliases: federated_*, federation_*)
Incremental federation watermark-based delta extraction; persistent local cache between runs
Per-model targets config(target='other_output') on federated models — results land on any profile output (standard models are refused a foreign target)
Bucket targets materialize models to S3 / GCS / Azure as Parquet or CSV
Python models, locally dbt-style .py models run on your machine (APIs, pandas, ML) — results land on any target
Bulk seeds dvt seed loads DVT's four file formats — csv / parquet / json / jsonl — via Sling to any engine, with enforced snake_case columns
One-command setup dvt sync installs every adapter and driver your profiles.yml requires
Connection diagnostics dvt debug --all checks every connection on both layers (dbt adapter + Sling)
Cross-engine catalog dvt docs builds one catalog + lineage UI across all engines in your project

Everything else — profiles.yml in ~/.dbt/, sources.yml, ref/source/config, tests, snapshots — is standard dbt. Existing dbt projects work unchanged; sources gain an optional meta.connection to bind them to other engines.

Supported engines

Every engine supported by both dbt and Sling is a DVT adapter — as federation source, federation target, seed target, and default target:

PostgreSQL · MySQL · MariaDB · SQL Server · Oracle · SQLite · DuckDB · Snowflake · BigQuery · Redshift · Databricks · ClickHouse · Trino · Athena · Microsoft Fabric — plus S3, GCS, and Azure Blob as bucket storages.

Anything else (MongoDB, Elasticsearch, REST APIs, …) is reachable through locally-executed Python models. Details per adapter: getdvt.net/docs/adapters.

Learn more

License

DVT is distributed as compiled wheels on PyPI. Free tier covers local files, transactional databases, and Python models; Pro adds cloud buckets (S3/GCS/Azure); Max adds cloud warehouses. See getdvt.net/pricing.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

dvt_core-0.2.52-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (57.3 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.52-cp314-cp314-macosx_11_0_arm64.whl (8.6 MB view details)

Uploaded CPython 3.14macOS 11.0+ ARM64

dvt_core-0.2.52-cp314-cp314-macosx_10_15_x86_64.whl (8.8 MB view details)

Uploaded CPython 3.14macOS 10.15+ x86-64

dvt_core-0.2.52-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (58.1 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.52-cp313-cp313-macosx_11_0_arm64.whl (8.6 MB view details)

Uploaded CPython 3.13macOS 11.0+ ARM64

dvt_core-0.2.52-cp313-cp313-macosx_10_13_x86_64.whl (8.8 MB view details)

Uploaded CPython 3.13macOS 10.13+ x86-64

dvt_core-0.2.52-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (58.6 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.52-cp312-cp312-macosx_11_0_arm64.whl (8.6 MB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

dvt_core-0.2.52-cp312-cp312-macosx_10_13_x86_64.whl (8.8 MB view details)

Uploaded CPython 3.12macOS 10.13+ x86-64

dvt_core-0.2.52-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (60.1 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.52-cp311-cp311-macosx_11_0_arm64.whl (8.7 MB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

dvt_core-0.2.52-cp311-cp311-macosx_10_9_x86_64.whl (8.9 MB view details)

Uploaded CPython 3.11macOS 10.9+ x86-64

dvt_core-0.2.52-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (56.8 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.52-cp310-cp310-macosx_11_0_arm64.whl (8.7 MB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

dvt_core-0.2.52-cp310-cp310-macosx_10_9_x86_64.whl (9.0 MB view details)

Uploaded CPython 3.10macOS 10.9+ x86-64

File details

Details for the file dvt_core-0.2.52-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.52-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 8096458cceb2f7dabea24afd5cb41fad40cd9a92102725da5cc8369721eb3db6
MD5 0f70472bb5bf8bdf668770f12b61aa54
BLAKE2b-256 b6687b40dfd024d3d1ac0651ef833f59f354f40d0f293e22a29548b86ab9d380

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.52-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.52-cp314-cp314-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.52-cp314-cp314-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 e96750a3d19d305c7c0fa36b361da2ea8c497162efa508fb174ae7e7162cd15b
MD5 d33a71ec7d2bbfb35a3b337d8a266838
BLAKE2b-256 82831d501287e2083c875fcd0c0ca0c18bea4a440ba456c95e179ad465d433aa

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.52-cp314-cp314-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.52-cp314-cp314-macosx_10_15_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.52-cp314-cp314-macosx_10_15_x86_64.whl
Algorithm Hash digest
SHA256 bcb87d2ac4b987601fbd462b5cbf5d85990d0eb5191692b8bc974c7e5f1e3d3b
MD5 f17b7d19aac638c9155b1a51eadf7d7c
BLAKE2b-256 23d9e5e445ade2eb89eabf9d0403d8575cd719d99685dbde3e25b0b9af6f5377

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.52-cp314-cp314-macosx_10_15_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.52-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.52-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 d51b18f286f27f3966712b4088a81691eb2aebfa28b91a99862723e6c81b8ce1
MD5 99e3675e6a17b0aa9565b5c0c7d37a9f
BLAKE2b-256 485772394c5256789ece55f7811971536f0367b2d0fb150649d00af754ec8fc3

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.52-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.52-cp313-cp313-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.52-cp313-cp313-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 ad7d69a7639b27712926ca9d3e864ca91efd1f564ce2a62f894391d3811a23c5
MD5 1b598361a574ea39e3796c88497bb093
BLAKE2b-256 4edbfd3abc05eb61175ad85c9904cde75c00f9d78d668f90888d551cfa41bb26

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.52-cp313-cp313-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.52-cp313-cp313-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.52-cp313-cp313-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 b207d9dd6bd2d115b368bd8199a26f08f11c80e0b8ccdd6a57a160684b5908ce
MD5 dd60334bcc705edcb50ff4bee12b1bfb
BLAKE2b-256 7450ef987cf367a6c62072edbd65287bc322f3ef15c57dbcc1194474ddafa2dd

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.52-cp313-cp313-macosx_10_13_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.52-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.52-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 0aac1f520a10c948d67147d890f40043a20df9c83c8369458670e657a59ec4fb
MD5 4553697f48baec0fb3dff34eac0b3682
BLAKE2b-256 c44b70da1936415c3a1ff5c2c06dfe103bd13d58c0c7de3769fe82600480f985

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.52-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.52-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.52-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 d03c226013aa985679fcef6e92a0f537195b59162b960d85cbd6417eeb62ae65
MD5 cd5cddf20bce6f4fa5797b2293803686
BLAKE2b-256 ec98905672b24639e4af048314ef70cdf1cfe5121da09d57841ecc1dd93d6afa

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.52-cp312-cp312-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.52-cp312-cp312-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.52-cp312-cp312-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 9ea81e08e0757774d68b98092335fac215b41dd61061a231680958c58bf24c92
MD5 2f532ff385558133ad047281f51e4694
BLAKE2b-256 55dc9c2a9fb0f9887d98bcde499b2cb7d5e48b4f19ca6a98d49328bda34c8929

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.52-cp312-cp312-macosx_10_13_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.52-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.52-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 73546667ec35c3797c320e93d0011fafd661508ba9717fd4d2813d08da51d38d
MD5 2faebc088da8c2939908e56a16b5d958
BLAKE2b-256 cf7def43a8d5bc8f3262eaf3962ee354a1ba244948d33df7044320b8989f45d4

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.52-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.52-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.52-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 127fb8027c847da58bbd369de009eb3fd50645077278666eb485b42351be9bfd
MD5 eed5d69840169d6f8a1f74ee21f4313e
BLAKE2b-256 613e682b9aec3e3ef9ba52461638532b08962b3e018eb40e6f4618b9454d9039

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.52-cp311-cp311-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.52-cp311-cp311-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.52-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 69765fd1d7d1018e093d5c05115a5a2a887a4398e4319bb33a5d1724ec190e57
MD5 3b6c1938348fda8109568ce1742c24cb
BLAKE2b-256 db7475d14bfa82b8c8654ab9b5fe0bb4b9de6deff8390dac0012c23ba93113f6

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.52-cp311-cp311-macosx_10_9_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.52-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.52-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 e46dca64490f19e1c54308b487b6354ed05a33767d01a2d283d65a6791fa73c6
MD5 cc03ccf3ac8507466cc5b93c2e870a09
BLAKE2b-256 7d0afffff575444afcccb2ccaa8b6203ffca631cabf64eb8cd9bebfaf90eb92f

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.52-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.52-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.52-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 0b29ce6fed1776890d0c7a9b32e0e9993435bcff9648ad84b9a90939027c5974
MD5 2381eace6c3151f3506065047b8187f2
BLAKE2b-256 23cad678a9089df76294adbfa57a930af8e9181b3dc5a1dfed5d5ac1da4a0b61

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.52-cp310-cp310-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.52-cp310-cp310-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.52-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 1a6c8888463b91a099afd9cacdf71df8efbf267948d9273542c1c930c2debbb0
MD5 18554a6b03f1fda2dfa1ddb2e8dfbcfc
BLAKE2b-256 e4d0bab025ac0f607000f3d2a1dfa3f90a99d7868472b32821fb92987b4b356b

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.52-cp310-cp310-macosx_10_9_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.2.52 This release

15 files

0.2.51

15 files

0.2.50

15 files

0.2.40

15 files

0.2.39

15 files

0.2.38

15 files

0.2.37

15 files

0.2.36

15 files

0.2.35

15 files

0.2.34

15 files

0.2.33

15 files

0.2.32

15 files

0.2.31

15 files

0.2.30

15 files

0.2.29

15 files

0.2.28

15 files

0.2.27

15 files

0.2.26

15 files

0.2.25

15 files

0.2.24

15 files

0.2.23

15 files

0.2.22

15 files

0.2.21

15 files

0.2.20

15 files

0.2.19

15 files

0.2.18

15 files

0.2.17

15 files

0.2.16

15 files

0.2.15

15 files

0.2.14

15 files

0.2.13

15 files

0.2.12

15 files

0.2.11

15 files

0.2.10

15 files

0.2.9

19 files

0.2.8

9 files

0.2.7

9 files

0.2.5

9 files

0.2.4

9 files

0.2.3

9 files

0.2.2

9 files

0.2.1

9 files

0.2.0

9 files

0.1.54

10 files

0.1.53

10 files

0.1.52

10 files

0.1.51

10 files

0.1.50

10 files

0.0.1

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page