Skip to main content

DVT logo

DVT — Data Virtualization Tool

Any source. Any target. One SQL project.

DVT is a cross-engine data transformation CLI built on dbt. Write SQL models that JOIN tables living on different database engines — MySQL with Snowflake, Oracle with PostgreSQL, anything with anything — and DVT handles extraction, federation, and loading automatically.

{{ config(materialized='f_table') }}   -- "f" = federated

select
    c.customer_name,
    o.order_date,
    o.total_amount
from {{ source('crm', 'customers') }} c      -- lives on MySQL
join {{ source('erp', 'orders') }} o          -- lives on Snowflake
  on c.customer_id = o.customer_id
where o.order_date >= '2024-01-01'
pip install dvt-core
dvt sync       # installs every adapter + driver your profiles.yml needs
dvt run        # dbt runs what dbt can; DVT federates the rest
dvt version    # (worth running once — you'll see)

Website & full docs: getdvt.net · Quickstart · PyPI


Architecture

DVT is a wrapper around stock dbt-core, not a fork. dbt compiles the project and runs every standard model; DVT picks up the models dbt can't express — cross-engine federation models and locally-executed Python models — and runs them through its own pipeline:

flowchart TB
    CLI([dvt run / build]) --> COMPILE["dbt compile → manifest.json"]
    COMPILE --> STD["standard models<br/>table · view · incremental"]
    COMPILE --> FED["federated models<br/>f_table · f_incremental · .py"]
    STD --> DBT["stock dbt executes them<br/>on the default target"]
    FED --> HOMO{"all sources on<br/>ONE connection?"}
    HOMO -- "yes — Sling direct" --> DIRECT["whole query transpiled to the
    source engine, streamed straight
    to the target — no staging"]
    HOMO -- "no — federation" --> DECOMP["1 · DECOMPOSE — SQLGlot maps every
    table to a profiles.yml connection"]
    DECOMP --> PUSH["2 · TRANSPILE + PUSH DOWN — minimal
    per-source queries, each engine's dialect"]
    PUSH --> EXTRACT["3 · EXTRACT — Sling → Parquet, parallel"]
    EXTRACT --> COMPUTE["4 · COMPUTE — DuckDB joins locally"]
    COMPUTE --> LOAD["5 · LOAD — Sling delivers to any
    database, warehouse, or bucket"]
    DIRECT --> TARGET[("target")]
    LOAD --> TARGET
    DBT --> TARGET

The Sling direct path (0.2.5)

When every source in an f_table model lives on one connection, steps 3–5 collapse into a single hop: the whole query — joins and aggregations included — is transpiled to that engine's dialect and submitted to Sling as a custom-SQL stream loaded straight to the target. No Parquet staging, no DuckDB. The check is automatic and per-run; any failure falls back to the standard pipeline. Add a source on another engine tomorrow and the same model silently switches to the federation pipeline — the model never changes.

Execution paths

Your model Who runs it How
table / view / incremental stock dbt natively on the default target — whole-query pushdown by definition
f_table / f_incremental, sources on one connection DVT Sling direct: full query pushed to the source engine, result streamed to the target
f_table / f_incremental, sources span connections DVT decompose → per-source pushdown extraction → DuckDB compute → load
.py models DVT locally executed; result loads to any target

Standard models that set a non-default target= are refused with guidance (dbt silently ignores the kwarg — your model would land on the wrong engine).

Design principles

  • Wrapper, not fork. DVT shells out to the dbt CLI and reads manifest.json — it never imports dbt internals. Your dbt version can move; DVT keeps working. Every dbt feature (Jinja, macros, packages, tests, snapshots, docs) works because it is dbt running it.
  • Dual-dialect by design. Federation models are written in DuckDB dialect and are transpiled per source — they never depend on any engine's syntax. Standard models compile in the default target's dialect via the official dbt adapter. Result: switching your default target (e.g. PostgreSQL → Databricks) is a profile change, not a rewrite.
  • Pushdown first. The fastest row to move is the one you never extract. Filters and column pruning push down to source engines; when every table in a model lives on one engine, the entire query pushes down and nothing moves at all.
  • Your hardware does the joining. Cross-engine compute happens in DuckDB on the machine running DVT — source databases only ever serve simple filtered SELECTs, and no warehouse bills you for the join.

What DVT adds on top of dbt

Capability How
Cross-engine models materialized='f_table' / 'f_incremental' (long aliases: federated_*, federation_*)
Incremental federation watermark-based delta extraction; persistent local cache between runs
Per-model targets config(target='other_output') on federated models — results land on any profile output (standard models are refused a foreign target)
Bucket targets materialize models to S3 / GCS / Azure as Parquet or CSV
Python models, locally dbt-style .py models run on your machine (APIs, pandas, ML) — results land on any target
Bulk seeds dvt seed loads csv / tsv / parquet / json / jsonl / avro / arrow / xlsx via Sling to any engine, with enforced snake_case columns
One-command setup dvt sync installs every adapter and driver your profiles.yml requires
Connection diagnostics dvt debug --all checks every connection on both layers (dbt adapter + Sling)
Cross-engine catalog dvt docs builds one catalog + lineage UI across all engines in your project

Everything else — profiles.yml in ~/.dbt/, sources.yml, ref/source/config, tests, snapshots — is standard dbt. Existing dbt projects work unchanged; sources gain an optional meta.connection to bind them to other engines.

Supported engines

Every engine supported by both dbt and Sling is a DVT adapter — as federation source, federation target, seed target, and default target:

PostgreSQL · MySQL · MariaDB · SQL Server · Oracle · SQLite · DuckDB · Snowflake · BigQuery · Redshift · Databricks · ClickHouse · Trino · Athena · Microsoft Fabric — plus S3, GCS, and Azure Blob as bucket storages.

Anything else (MongoDB, Elasticsearch, REST APIs, …) is reachable through locally-executed Python models. Details per adapter: getdvt.net/docs/adapters.

Learn more

License

DVT is distributed as compiled wheels on PyPI. Free tier covers local files, transactional databases, and Python models; Pro adds cloud buckets (S3/GCS/Azure); Max adds cloud warehouses. See getdvt.net/pricing.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

dvt_core-0.2.37-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (21.9 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.37-cp314-cp314-macosx_11_0_arm64.whl (3.6 MB view details)

Uploaded CPython 3.14macOS 11.0+ ARM64

dvt_core-0.2.37-cp314-cp314-macosx_10_15_x86_64.whl (3.7 MB view details)

Uploaded CPython 3.14macOS 10.15+ x86-64

dvt_core-0.2.37-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (22.2 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.37-cp313-cp313-macosx_11_0_arm64.whl (3.6 MB view details)

Uploaded CPython 3.13macOS 11.0+ ARM64

dvt_core-0.2.37-cp313-cp313-macosx_10_13_x86_64.whl (3.7 MB view details)

Uploaded CPython 3.13macOS 10.13+ x86-64

dvt_core-0.2.37-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (22.4 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.37-cp312-cp312-macosx_11_0_arm64.whl (3.6 MB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

dvt_core-0.2.37-cp312-cp312-macosx_10_13_x86_64.whl (3.7 MB view details)

Uploaded CPython 3.12macOS 10.13+ x86-64

dvt_core-0.2.37-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (23.1 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.37-cp311-cp311-macosx_11_0_arm64.whl (3.6 MB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

dvt_core-0.2.37-cp311-cp311-macosx_10_9_x86_64.whl (3.7 MB view details)

Uploaded CPython 3.11macOS 10.9+ x86-64

dvt_core-0.2.37-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (21.8 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.37-cp310-cp310-macosx_11_0_arm64.whl (3.7 MB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

dvt_core-0.2.37-cp310-cp310-macosx_10_9_x86_64.whl (3.7 MB view details)

Uploaded CPython 3.10macOS 10.9+ x86-64

File details

Details for the file dvt_core-0.2.37-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.37-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 1a8030ec4b34f2a84083ca8c1b448a073ef6ff7ac1f43ebb45f2fb5139f6177a
MD5 34a451a5c2f0bc30b58c7fe224a0e1e6
BLAKE2b-256 b82bd36ecb5a185738f3cb99906e69b72128ed6dd898a0b3c8da968375f110e4

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.37-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.37-cp314-cp314-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.37-cp314-cp314-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 5b93a645a68f055bee55cc6a57b5444292f01d509b34b207368c93a8fa9df997
MD5 c77a48e93fe98484294c24752512ecca
BLAKE2b-256 0dca5c6700121053ddf99c2655e6b74854485220e9a2078fd9e051d937cbb270

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.37-cp314-cp314-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.37-cp314-cp314-macosx_10_15_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.37-cp314-cp314-macosx_10_15_x86_64.whl
Algorithm Hash digest
SHA256 8c1b85761291e3e65dfbb7c628e6eaae68b5f1ba5856eab7bde4246f7f62cbbf
MD5 08b3a80b8755857480e256c38d24f94f
BLAKE2b-256 740c9d33b193c98197f86d156cf62543812b908f6b53703df9a87f330f543213

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.37-cp314-cp314-macosx_10_15_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.37-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.37-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 31293217f22ad4ef48af7f1f452249b48f6d074e0e0a659b2021046ec5b06c47
MD5 8f5ff63dc9be1a7c867ec30d88d6787f
BLAKE2b-256 c2dccabdfb1d49634eda6ff60b740505ed0968fa538c95caf6d0ed54b6d02cc9

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.37-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.37-cp313-cp313-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.37-cp313-cp313-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 97278f66daa3456d9450238c55f4b70aa6fd37db2468775bad02abc2f7dcfb38
MD5 bf3aa90419ed74f6bfc66489141fe942
BLAKE2b-256 589edd72449e244075586eeee4ce145b95618392187bd83128f4d5839f79d051

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.37-cp313-cp313-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.37-cp313-cp313-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.37-cp313-cp313-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 b60ef5619a9a9b76a57c3cd90384aec09b902474c3746a6aa3c3be7f8ab40269
MD5 992cb9580a2fe3232b0734cc76219a52
BLAKE2b-256 22155f4100f29010756f245ed27263f25107644e6b458c9c8766c58c2ac78d79

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.37-cp313-cp313-macosx_10_13_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.37-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.37-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 1d6dd6ddfb533122a955b055f5e3bdd9beff775c87a73aa79a7d8908df028dba
MD5 6f6d17b20e67345e936650c296baabcb
BLAKE2b-256 fd9d069217183244033d872d25b6075723c43270c424ab2e1714054643b09a31

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.37-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.37-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.37-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 04894f8b4c15d982ed0079ec5d8e55ddfc75b1a770bb5067675208c10053b3ab
MD5 cd026df5aa2c0d0a82f626bbd5ad3ae7
BLAKE2b-256 6e048626ebcf534b701858519c887e7afe3da1c75037fc85b3e8f112b8a3b8c1

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.37-cp312-cp312-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.37-cp312-cp312-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.37-cp312-cp312-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 54467be3d7fc5b9d7b63709d598ababec363f9b789ae128a86c149c424afecfa
MD5 6e5739ed42fe680d9f8df940ffbc1658
BLAKE2b-256 10fd6d30f373f6048e88c5f53f1f02d56a70017584879792de533fd9cf484e80

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.37-cp312-cp312-macosx_10_13_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.37-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.37-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 5cc44c52d4fa1948edba0ec55ef76eac1f6c87fb60ef2719e937a2b93444f489
MD5 9a27819a6a751412865b6798b475f8a9
BLAKE2b-256 adea09fe17e4a4aba537ba5605af49e9e261a0a3b44e7d9b3522f760515e8cf2

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.37-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.37-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.37-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 17602527103551f4ffcf1b94ad057df9f75b24d0833bafbad4d1432ea7c5c6f9
MD5 63ef4fd3c92b3d30a35b7ddf8e85f0d9
BLAKE2b-256 7f6be52806342944ad1a2b63dc3572aff9d099f854175c3cc3e6c1b9c690e360

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.37-cp311-cp311-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.37-cp311-cp311-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.37-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 1366ebccd7fff75e10f45d4b29144fcaf301d3699ff379553c03effe4c76e7f9
MD5 116470b3e4779c5e81a37c9620f7e951
BLAKE2b-256 47d27b1aa8140233e4bad3a86d7bba4ce4620428923ae3867a534ade7ad54ceb

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.37-cp311-cp311-macosx_10_9_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.37-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.37-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 e8cd960eff6e41136d38867f476732887d68fd0c6231980eeeffc1c8e751a142
MD5 0316d662500bf66d488fb3fb47b9fe7f
BLAKE2b-256 0154af3e5bd0d454e61d5e152ded9b204f9fb0bf72f0b0ecf3aec4f8a7438ef5

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.37-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.37-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.37-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 f46173d1876c74f2041795029a005ec2d835b3923bdda67cb3b0f7297a946eee
MD5 2930c98433b46ed076119678011dbf62
BLAKE2b-256 f708c96e348a58c866cd43c03e2f67c590ad87b9b76fd63db9903a338fd2f596

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.37-cp310-cp310-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.37-cp310-cp310-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.37-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 3e0cded79712a4c9749a7b3a70851f8ca784f57018f4d24b93eccf2414db433e
MD5 398e10d737f7f82d931e8e6067ac9f7c
BLAKE2b-256 80e0afbafc31e6a2a1dd9159f15dc7eba6e8c5428fbb64849cc63f0ba651f2fd

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.37-cp310-cp310-macosx_10_9_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.2.52

15 files

0.2.51

15 files

0.2.50

15 files

0.2.40

15 files

0.2.39

15 files

0.2.38

15 files

This release

0.2.37 This release

15 files

0.2.36

15 files

0.2.35

15 files

0.2.34

15 files

0.2.33

15 files

0.2.32

15 files

0.2.31

15 files

0.2.30

15 files

0.2.29

15 files

0.2.28

15 files

0.2.27

15 files

0.2.26

15 files

0.2.25

15 files

0.2.24

15 files

0.2.23

15 files

0.2.22

15 files

0.2.21

15 files

0.2.20

15 files

0.2.19

15 files

0.2.18

15 files

0.2.17

15 files

0.2.16

15 files

0.2.15

15 files

0.2.14

15 files

0.2.13

15 files

0.2.12

15 files

0.2.11

15 files

0.2.10

15 files

0.2.9

19 files

0.2.8

9 files

0.2.7

9 files

0.2.5

9 files

0.2.4

9 files

0.2.3

9 files

0.2.2

9 files

0.2.1

9 files

0.2.0

9 files

0.1.54

10 files

0.1.53

10 files

0.1.52

10 files

0.1.51

10 files

0.1.50

10 files

0.0.1

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page