Skip to main content

DVT logo

DVT — Data Virtualization Tool

Any source. Any target. One SQL project.

DVT is a cross-engine data transformation CLI built on dbt. Write SQL models that JOIN tables living on different database engines — MySQL with Snowflake, Oracle with PostgreSQL, anything with anything — and DVT handles extraction, federation, and loading automatically.

{{ config(materialized='f_table') }}   -- "f" = federated

select
    c.customer_name,
    o.order_date,
    o.total_amount
from {{ source('crm', 'customers') }} c      -- lives on MySQL
join {{ source('erp', 'orders') }} o          -- lives on Snowflake
  on c.customer_id = o.customer_id
where o.order_date >= '2024-01-01'
pip install dvt-core
dvt sync       # installs every adapter + driver your profiles.yml needs
dvt run        # dbt runs what dbt can; DVT federates the rest
dvt version    # (worth running once — you'll see)

Website & full docs: getdvt.net · Quickstart · PyPI


Architecture

DVT is a wrapper around stock dbt-core, not a fork. dbt compiles the project and runs every standard model; DVT picks up the models dbt can't express — cross-engine federation models and locally-executed Python models — and runs them through its own pipeline:

flowchart TB
    CLI([dvt run / build]) --> COMPILE["dbt compile → manifest.json"]
    COMPILE --> STD["standard models<br/>table · view · incremental"]
    COMPILE --> FED["federated models<br/>f_table · f_incremental · .py"]
    STD --> DBT["stock dbt executes them<br/>on the default target"]
    FED --> HOMO{"all sources on<br/>ONE connection?"}
    HOMO -- "yes — Sling direct" --> DIRECT["whole query transpiled to the
    source engine, streamed straight
    to the target — no staging"]
    HOMO -- "no — federation" --> DECOMP["1 · DECOMPOSE — SQLGlot maps every
    table to a profiles.yml connection"]
    DECOMP --> PUSH["2 · TRANSPILE + PUSH DOWN — minimal
    per-source queries, each engine's dialect"]
    PUSH --> EXTRACT["3 · EXTRACT — Sling → Parquet, parallel"]
    EXTRACT --> COMPUTE["4 · COMPUTE — DuckDB joins locally"]
    COMPUTE --> LOAD["5 · LOAD — Sling delivers to any
    database, warehouse, or bucket"]
    DIRECT --> TARGET[("target")]
    LOAD --> TARGET
    DBT --> TARGET

The Sling direct path (0.2.5)

When every source in an f_table model lives on one connection, steps 3–5 collapse into a single hop: the whole query — joins and aggregations included — is transpiled to that engine's dialect and submitted to Sling as a custom-SQL stream loaded straight to the target. No Parquet staging, no DuckDB. The check is automatic and per-run; any failure falls back to the standard pipeline. Add a source on another engine tomorrow and the same model silently switches to the federation pipeline — the model never changes.

Execution paths

Your model Who runs it How
table / view / incremental stock dbt natively on the default target — whole-query pushdown by definition
f_table / f_incremental, sources on one connection DVT Sling direct: full query pushed to the source engine, result streamed to the target
f_table / f_incremental, sources span connections DVT decompose → per-source pushdown extraction → DuckDB compute → load
.py models DVT locally executed; result loads to any target

Standard models that set a non-default target= are refused with guidance (dbt silently ignores the kwarg — your model would land on the wrong engine).

Design principles

  • Wrapper, not fork. DVT shells out to the dbt CLI and reads manifest.json — it never imports dbt internals. Your dbt version can move; DVT keeps working. Every dbt feature (Jinja, macros, packages, tests, snapshots, docs) works because it is dbt running it.
  • Dual-dialect by design. Federation models are written in DuckDB dialect and are transpiled per source — they never depend on any engine's syntax. Standard models compile in the default target's dialect via the official dbt adapter. Result: switching your default target (e.g. PostgreSQL → Databricks) is a profile change, not a rewrite.
  • Pushdown first. The fastest row to move is the one you never extract. Filters and column pruning push down to source engines; when every table in a model lives on one engine, the entire query pushes down and nothing moves at all.
  • Your hardware does the joining. Cross-engine compute happens in DuckDB on the machine running DVT — source databases only ever serve simple filtered SELECTs, and no warehouse bills you for the join.

What DVT adds on top of dbt

Capability How
Cross-engine models materialized='f_table' / 'f_incremental' (long aliases: federated_*, federation_*)
Incremental federation watermark-based delta extraction; persistent local cache between runs
Per-model targets config(target='other_output') on federated models — results land on any profile output (standard models are refused a foreign target)
Bucket targets materialize models to S3 / GCS / Azure as Parquet or CSV
Python models, locally dbt-style .py models run on your machine (APIs, pandas, ML) — results land on any target
Bulk seeds dvt seed loads DVT's four file formats — csv / parquet / json / jsonl — via Sling to any engine, with enforced snake_case columns
One-command setup dvt sync installs every adapter and driver your profiles.yml requires
Connection diagnostics dvt debug --all checks every connection on both layers (dbt adapter + Sling)
Cross-engine catalog dvt docs builds one catalog + lineage UI across all engines in your project

Everything else — profiles.yml in ~/.dbt/, sources.yml, ref/source/config, tests, snapshots — is standard dbt. Existing dbt projects work unchanged; sources gain an optional meta.connection to bind them to other engines.

Supported engines

Every engine supported by both dbt and Sling is a DVT adapter — as federation source, federation target, seed target, and default target:

PostgreSQL · MySQL · MariaDB · SQL Server · Oracle · SQLite · DuckDB · Snowflake · BigQuery · Redshift · Databricks · ClickHouse · Trino · Athena · Microsoft Fabric — plus S3, GCS, and Azure Blob as bucket storages.

Anything else (MongoDB, Elasticsearch, REST APIs, …) is reachable through locally-executed Python models. Details per adapter: getdvt.net/docs/adapters.

Learn more

License

DVT is distributed as compiled wheels on PyPI. Free tier covers local files, transactional databases, and Python models; Pro adds cloud buckets (S3/GCS/Azure); Max adds cloud warehouses. See getdvt.net/pricing.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

dvt_core-0.2.51-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (56.8 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.51-cp314-cp314-macosx_11_0_arm64.whl (8.6 MB view details)

Uploaded CPython 3.14macOS 11.0+ ARM64

dvt_core-0.2.51-cp314-cp314-macosx_10_15_x86_64.whl (8.7 MB view details)

Uploaded CPython 3.14macOS 10.15+ x86-64

dvt_core-0.2.51-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (57.5 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.51-cp313-cp313-macosx_11_0_arm64.whl (8.5 MB view details)

Uploaded CPython 3.13macOS 11.0+ ARM64

dvt_core-0.2.51-cp313-cp313-macosx_10_13_x86_64.whl (8.7 MB view details)

Uploaded CPython 3.13macOS 10.13+ x86-64

dvt_core-0.2.51-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (58.0 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.51-cp312-cp312-macosx_11_0_arm64.whl (8.6 MB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

dvt_core-0.2.51-cp312-cp312-macosx_10_13_x86_64.whl (8.8 MB view details)

Uploaded CPython 3.12macOS 10.13+ x86-64

dvt_core-0.2.51-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (59.5 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.51-cp311-cp311-macosx_11_0_arm64.whl (8.6 MB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

dvt_core-0.2.51-cp311-cp311-macosx_10_9_x86_64.whl (8.9 MB view details)

Uploaded CPython 3.11macOS 10.9+ x86-64

dvt_core-0.2.51-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (56.3 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.51-cp310-cp310-macosx_11_0_arm64.whl (8.7 MB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

dvt_core-0.2.51-cp310-cp310-macosx_10_9_x86_64.whl (8.9 MB view details)

Uploaded CPython 3.10macOS 10.9+ x86-64

File details

Details for the file dvt_core-0.2.51-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.51-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 f0baecbead28c0eb1b742c69a43786a7262c09ca6bcbb9ed2ba72d06f46aa41d
MD5 5a97b46d393f5148ce7dccf51fbd2e66
BLAKE2b-256 304464f779361b409816f5bd32009620c06062fdd0c2cb6afbd61e219f27138c

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.51-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.51-cp314-cp314-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.51-cp314-cp314-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 75b3d387f3f0d875a1892aa46f74f1a4f7df7caf48fed40d64898ebf09536e9a
MD5 5178be46ad895981d02ef28ab8ec5f3f
BLAKE2b-256 1823bc34180c2d1eb5556e15b96c31a472a7915102bcd77c89360140b33004b4

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.51-cp314-cp314-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.51-cp314-cp314-macosx_10_15_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.51-cp314-cp314-macosx_10_15_x86_64.whl
Algorithm Hash digest
SHA256 09126a9c66a17af131c54797ae8d8eb7ca19949f90a7c9039b0d43ed94bb28ba
MD5 e95ff75fd6de47dc18286b647d8cf25a
BLAKE2b-256 4a8cb74b5b7c50c5b8e557b78fc1f819ca5f01ae066d78b6497a32eacf6d678c

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.51-cp314-cp314-macosx_10_15_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.51-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.51-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 65d2095c36f11416b78a00c4543854133daeca844a1e0361564c0c8bc3d5a435
MD5 b668b484b72be4322a1a73ddd21277c0
BLAKE2b-256 30a839767484b2b7f9eac2cbd8f75ee95fe68e58837325b7ce43e572c36e96b4

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.51-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.51-cp313-cp313-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.51-cp313-cp313-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 85ced9f51ab248cf17c1894d5e0ebaca88d370f3d1d9f41b3e116609a1b54005
MD5 8076079deeb47747c6bcbb65e1907672
BLAKE2b-256 9a5df25bf06f406e3acd2d79029d79b74ab8031ea82e1c0453ade752cdfda4c4

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.51-cp313-cp313-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.51-cp313-cp313-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.51-cp313-cp313-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 559563ab7342096d2f9c58c85e1c3a053875bf75006141c19816cb24ddd8a404
MD5 183eefa3e3e4c849e06f69d3bbc4df10
BLAKE2b-256 b84fb28d1df0290d10d740e4db6d129f3ee6ba960f3ea599a51984f4618ea54b

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.51-cp313-cp313-macosx_10_13_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.51-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.51-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 2af7024791a7f4dd5f67a0c5a7fa17e951c90fcc4db2b9b5ff9922f40103c903
MD5 ab49a707eca946680c5e4fc8b96fe35f
BLAKE2b-256 85487a58f809f4d830f9f8e1a21e21679cc7742ae0fd6673d1e71bf2cbfb6d40

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.51-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.51-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.51-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 82f883a90c7af4fd4a9dfabe0f72f488968f0e5fae447616e62c5567719425eb
MD5 930f6d3b643d7f17e58fe597b805fee5
BLAKE2b-256 25363b421528be78d24c71cade6cee0f656836b4c910ab61a5724a43ef2f96a3

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.51-cp312-cp312-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.51-cp312-cp312-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.51-cp312-cp312-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 2452a2d6b671dbaf7af6d70a176e433a1193f73533825885c947244dcd83005e
MD5 a08c12b8528986ccd2b2fd9024491695
BLAKE2b-256 068d1f3f0af86472da8300f7c4ef0c3633f75e7c4e54efcfdf657c1c6fac1875

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.51-cp312-cp312-macosx_10_13_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.51-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.51-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 0d5f05eaf2b05be8a1d6444f5259e7fca1d76d2d97fe23c176a51384962ed729
MD5 c4e6a3a7fd5d8a3f0ed464601d8cf9ee
BLAKE2b-256 59003f64105dc9791dcaf6d0f03997a96ee7367227c91a4573d0650db884e7b5

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.51-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.51-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.51-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 cb00984bae9ce7876f656f4d075d9b63789248b479524a29196d87cb5e39d9a8
MD5 2520a316e307f1a16aa62c91277d0aa8
BLAKE2b-256 fdf024c0f49ea93046a16d95cd935412c00c38ef961e70109e328949456d4bb1

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.51-cp311-cp311-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.51-cp311-cp311-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.51-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 0e275fa1899fe5b7848ac0046e57dad1ea5f74a5ca69c2b9314f7b8e678e0e49
MD5 c8645f7ab6f5d0431fae62b4c8dad32e
BLAKE2b-256 03cac2c2f7461a14a74bf3dc6ebc14025c10e656d990951cb1d1c68773d0b1de

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.51-cp311-cp311-macosx_10_9_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.51-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.51-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 604c15dbea82ac26b6fc91f8ff958bf60441f7cc69b2b20f3d352cedc528e54c
MD5 9631199f9794c13b1596ed07645553d3
BLAKE2b-256 b48dde2877fdc9f2a7d66158cb3f47e41e980517731b0b292f1e62e415ce7cfc

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.51-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.51-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.51-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 efaa286150f722da1ad9f164da256d8a26f1556c5366619bdeb6633512a4de85
MD5 f3695960b89d13ea55cf9036610eb9a1
BLAKE2b-256 976f55b3abecf89146fb036d5b07ac407ea4bc9318a22887ffeac11984414cfa

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.51-cp310-cp310-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.51-cp310-cp310-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.51-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 8ba9658b8378ddc0dfeac9971cea2b8bf3e41420c5ec53b72a74238b71caf28b
MD5 8892d65c5c4f7ab376a38701adc47be0
BLAKE2b-256 7e18dce25a67c200a0cf42a2d7a320873aa53d263c69ddcc417d000a97731cb8

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.51-cp310-cp310-macosx_10_9_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.2.52

15 files

This release

0.2.51 This release

15 files

0.2.50

15 files

0.2.40

15 files

0.2.39

15 files

0.2.38

15 files

0.2.37

15 files

0.2.36

15 files

0.2.35

15 files

0.2.34

15 files

0.2.33

15 files

0.2.32

15 files

0.2.31

15 files

0.2.30

15 files

0.2.29

15 files

0.2.28

15 files

0.2.27

15 files

0.2.26

15 files

0.2.25

15 files

0.2.24

15 files

0.2.23

15 files

0.2.22

15 files

0.2.21

15 files

0.2.20

15 files

0.2.19

15 files

0.2.18

15 files

0.2.17

15 files

0.2.16

15 files

0.2.15

15 files

0.2.14

15 files

0.2.13

15 files

0.2.12

15 files

0.2.11

15 files

0.2.10

15 files

0.2.9

19 files

0.2.8

9 files

0.2.7

9 files

0.2.5

9 files

0.2.4

9 files

0.2.3

9 files

0.2.2

9 files

0.2.1

9 files

0.2.0

9 files

0.1.54

10 files

0.1.53

10 files

0.1.52

10 files

0.1.51

10 files

0.1.50

10 files

0.0.1

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page