Skip to main content

DVT logo

DVT — Data Virtualization Tool

Any source. Any target. One SQL project.

DVT is a cross-engine data transformation CLI built on dbt. Write SQL models that JOIN tables living on different database engines — MySQL with Snowflake, Oracle with PostgreSQL, anything with anything — and DVT handles extraction, federation, and loading automatically.

{{ config(materialized='f_table') }}   -- "f" = federated

select
    c.customer_name,
    o.order_date,
    o.total_amount
from {{ source('crm', 'customers') }} c      -- lives on MySQL
join {{ source('erp', 'orders') }} o          -- lives on Snowflake
  on c.customer_id = o.customer_id
where o.order_date >= '2024-01-01'
pip install dvt-core
dvt sync       # installs every adapter + driver your profiles.yml needs
dvt run        # dbt runs what dbt can; DVT federates the rest
dvt version    # (worth running once — you'll see)

Website & full docs: getdvt.net · Quickstart · PyPI


Architecture

DVT is a wrapper around stock dbt-core, not a fork. dbt compiles the project and runs every standard model; DVT picks up the models dbt can't express — cross-engine federation models and locally-executed Python models — and runs them through its own pipeline:

flowchart TB
    CLI([dvt run / build]) --> COMPILE["dbt compile → manifest.json"]
    COMPILE --> STD["standard models<br/>table · view · incremental"]
    COMPILE --> FED["federated models<br/>f_table · f_incremental · .py"]
    STD --> DBT["stock dbt executes them<br/>on the default target"]
    FED --> HOMO{"all sources on<br/>ONE connection?"}
    HOMO -- "yes — Sling direct" --> DIRECT["whole query transpiled to the
    source engine, streamed straight
    to the target — no staging"]
    HOMO -- "no — federation" --> DECOMP["1 · DECOMPOSE — SQLGlot maps every
    table to a profiles.yml connection"]
    DECOMP --> PUSH["2 · TRANSPILE + PUSH DOWN — minimal
    per-source queries, each engine's dialect"]
    PUSH --> EXTRACT["3 · EXTRACT — Sling → Parquet, parallel"]
    EXTRACT --> COMPUTE["4 · COMPUTE — DuckDB joins locally"]
    COMPUTE --> LOAD["5 · LOAD — Sling delivers to any
    database, warehouse, or bucket"]
    DIRECT --> TARGET[("target")]
    LOAD --> TARGET
    DBT --> TARGET

The Sling direct path (0.2.5)

When every source in an f_table model lives on one connection, steps 3–5 collapse into a single hop: the whole query — joins and aggregations included — is transpiled to that engine's dialect and submitted to Sling as a custom-SQL stream loaded straight to the target. No Parquet staging, no DuckDB. The check is automatic and per-run; any failure falls back to the standard pipeline. Add a source on another engine tomorrow and the same model silently switches to the federation pipeline — the model never changes.

Execution paths

Your model Who runs it How
table / view / incremental stock dbt natively on the default target — whole-query pushdown by definition
f_table / f_incremental, sources on one connection DVT Sling direct: full query pushed to the source engine, result streamed to the target
f_table / f_incremental, sources span connections DVT decompose → per-source pushdown extraction → DuckDB compute → load
.py models DVT locally executed; result loads to any target

Standard models that set a non-default target= are refused with guidance (dbt silently ignores the kwarg — your model would land on the wrong engine).

Design principles

  • Wrapper, not fork. DVT shells out to the dbt CLI and reads manifest.json — it never imports dbt internals. Your dbt version can move; DVT keeps working. Every dbt feature (Jinja, macros, packages, tests, snapshots, docs) works because it is dbt running it.
  • Dual-dialect by design. Federation models are written in DuckDB dialect and are transpiled per source — they never depend on any engine's syntax. Standard models compile in the default target's dialect via the official dbt adapter. Result: switching your default target (e.g. PostgreSQL → Databricks) is a profile change, not a rewrite.
  • Pushdown first. The fastest row to move is the one you never extract. Filters and column pruning push down to source engines; when every table in a model lives on one engine, the entire query pushes down and nothing moves at all.
  • Your hardware does the joining. Cross-engine compute happens in DuckDB on the machine running DVT — source databases only ever serve simple filtered SELECTs, and no warehouse bills you for the join.

What DVT adds on top of dbt

Capability How
Cross-engine models materialized='f_table' / 'f_incremental' (long aliases: federated_*, federation_*)
Incremental federation watermark-based delta extraction; persistent local cache between runs
Per-model targets config(target='other_output') on federated models — results land on any profile output (standard models are refused a foreign target)
Bucket targets materialize models to S3 / GCS / Azure as Parquet or CSV
Python models, locally dbt-style .py models run on your machine (APIs, pandas, ML) — results land on any target
Bulk seeds dvt seed loads csv / tsv / parquet / json / jsonl / avro / arrow / xlsx via Sling to any engine, with enforced snake_case columns
One-command setup dvt sync installs every adapter and driver your profiles.yml requires
Connection diagnostics dvt debug --all checks every connection on both layers (dbt adapter + Sling)
Cross-engine catalog dvt docs builds one catalog + lineage UI across all engines in your project

Everything else — profiles.yml in ~/.dbt/, sources.yml, ref/source/config, tests, snapshots — is standard dbt. Existing dbt projects work unchanged; sources gain an optional meta.connection to bind them to other engines.

Supported engines

Every engine supported by both dbt and Sling is a DVT adapter — as federation source, federation target, seed target, and default target:

PostgreSQL · MySQL · MariaDB · SQL Server · Oracle · SQLite · DuckDB · Snowflake · BigQuery · Redshift · Databricks · ClickHouse · Trino · Athena · Microsoft Fabric — plus S3, GCS, and Azure Blob as bucket storages.

Anything else (MongoDB, Elasticsearch, REST APIs, …) is reachable through locally-executed Python models. Details per adapter: getdvt.net/docs/adapters.

Learn more

License

DVT is distributed as compiled wheels on PyPI. Free tier covers local files, transactional databases, and Python models; Pro adds cloud buckets (S3/GCS/Azure); Max adds cloud warehouses. See getdvt.net/pricing.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

dvt_core-0.2.39-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (26.2 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.39-cp314-cp314-macosx_11_0_arm64.whl (4.2 MB view details)

Uploaded CPython 3.14macOS 11.0+ ARM64

dvt_core-0.2.39-cp314-cp314-macosx_10_15_x86_64.whl (4.3 MB view details)

Uploaded CPython 3.14macOS 10.15+ x86-64

dvt_core-0.2.39-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (26.6 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.39-cp313-cp313-macosx_11_0_arm64.whl (4.2 MB view details)

Uploaded CPython 3.13macOS 11.0+ ARM64

dvt_core-0.2.39-cp313-cp313-macosx_10_13_x86_64.whl (4.3 MB view details)

Uploaded CPython 3.13macOS 10.13+ x86-64

dvt_core-0.2.39-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (26.7 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.39-cp312-cp312-macosx_11_0_arm64.whl (4.2 MB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

dvt_core-0.2.39-cp312-cp312-macosx_10_13_x86_64.whl (4.3 MB view details)

Uploaded CPython 3.12macOS 10.13+ x86-64

dvt_core-0.2.39-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (27.6 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.39-cp311-cp311-macosx_11_0_arm64.whl (4.2 MB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

dvt_core-0.2.39-cp311-cp311-macosx_10_9_x86_64.whl (4.3 MB view details)

Uploaded CPython 3.11macOS 10.9+ x86-64

dvt_core-0.2.39-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (26.1 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.39-cp310-cp310-macosx_11_0_arm64.whl (4.2 MB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

dvt_core-0.2.39-cp310-cp310-macosx_10_9_x86_64.whl (4.4 MB view details)

Uploaded CPython 3.10macOS 10.9+ x86-64

File details

Details for the file dvt_core-0.2.39-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.39-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 46e56d659e845c99a334cf46a43797800dee6b0ffe7566a2ed5c17730b362d69
MD5 0311cb35d788cab04fd86aa4156c7629
BLAKE2b-256 6a8c9013e45384521ea726db06124699f64e59f845709b427b594ba38285490a

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.39-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.39-cp314-cp314-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.39-cp314-cp314-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 2f3933324825971d49f5c0d5d9dd33cd2f9690bdc857f4291e3ecffa9a652989
MD5 9613cc0ccdf136a50349fc2fe116a74c
BLAKE2b-256 c8f87d8a248b5c465e6ebb6ace688eeb61e2d7e87acb7a02a4a2a6e366757adf

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.39-cp314-cp314-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.39-cp314-cp314-macosx_10_15_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.39-cp314-cp314-macosx_10_15_x86_64.whl
Algorithm Hash digest
SHA256 0d2e42eb14d671e8a993976fbb624301ad8ffd53914a3b487a743774b05bad3a
MD5 f329b4bd6967d1af4d28d85ecb58ba7b
BLAKE2b-256 d14121177825c70ec90a793e74ca3136fca8319ce2cf94d390a8050f88175163

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.39-cp314-cp314-macosx_10_15_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.39-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.39-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 24d591f4a3e2cbdf9f941b269f32d100ccaf5253018b46260cc6fbc384398344
MD5 ca42c224625d29c35aa2a10c55435e90
BLAKE2b-256 bddfd4adbf630120c1e1b31ce160edd54ca3df19b9fdb0ba69b00cee4386cfee

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.39-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.39-cp313-cp313-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.39-cp313-cp313-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 2f43a226dd91c6b6d0727865190c0b7a4acfb1ed84e9e90142c651e6f37818e0
MD5 242ab1244c43607ab2566e5945f3704b
BLAKE2b-256 21169cea9e3b7f2ccd40dcb6dd59ea13d9ea096ce65ac04b7f5b8f070a2006d9

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.39-cp313-cp313-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.39-cp313-cp313-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.39-cp313-cp313-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 9ed8386ee77cf30a4c48ccd28687de35f2a9eee2c9925e512ac5a6245463b990
MD5 0ba19d00045ebd7d27631dab5b743d8f
BLAKE2b-256 1d685ec8e3242187adcc893e51d5822080fbdaa3b94385883d8d05ccab740f9b

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.39-cp313-cp313-macosx_10_13_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.39-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.39-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 e828c8ee01c9386ee746dcafa3650aa81a6c99ccd035d5a7fd547ed3dd19d6e6
MD5 2fa3529cd55294bbe3a2cc9bea6bd11a
BLAKE2b-256 d6f938a3360a190d82bf46e65cdd925dbd77412d68dbb11a4abc1d4bc90af96f

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.39-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.39-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.39-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 4a0bafc8dd383315b1f81e7ee6c25ed64e389de03c30d7097d697afd8d737338
MD5 414b363cce43fbcdfa1296486b581890
BLAKE2b-256 baff55b4159112eb7a15a4dbc97570fd7cdc1f7b485cf9e5cbd885a69561d1ec

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.39-cp312-cp312-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.39-cp312-cp312-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.39-cp312-cp312-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 3fbfacc851dc84fa0189f8a769cf013386f44636773650b1eec3fb461c7459c1
MD5 b5caee09f0bdf17aaa098d78713acf38
BLAKE2b-256 be78f362e347db6fc20a98eb3439eae40155b45ea48f97b58709d909599d3ca5

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.39-cp312-cp312-macosx_10_13_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.39-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.39-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 17c3ba5222cea3bdb298836bea1b705047513e0a8558d257958279a6e5fa5470
MD5 4811a5b500bcc8ce6d30405e619e0931
BLAKE2b-256 8cc02885436c93345edc16f59cdd2eeb24be896bc8b8ebbe658377b93848ee26

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.39-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.39-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.39-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 f574615921fc1fcea4d3af6d58523b3c939386d8edf811a375d2bed628c9232e
MD5 28150d75687950d9aeef6f4dc808af26
BLAKE2b-256 b4a31caaaa9d0a2c6da06303d4358c1a9dfa3712abc71b03eb2cf36c9acf3b11

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.39-cp311-cp311-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.39-cp311-cp311-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.39-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 dc44c28f2d99fe836a8ccb8802aa8077715fd8d9f703f825e67f7cf6ae03003d
MD5 67baa46f0aeef7afaea67ad357609bea
BLAKE2b-256 f3ce6479a44fae503c2e5eb4cc2d9b4b3253092f08c5b85afa7fccfaf54e77bf

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.39-cp311-cp311-macosx_10_9_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.39-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.39-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 073c07e978e0e1598eb4b1ee1078c16da9cacbff208b0ea59cd5a873bf554d83
MD5 39df51c8e2991aa1fc8ad342ab4c1063
BLAKE2b-256 2c7f71564febf47bec01db787e3c6183bf49b6e654c0ac74d65a77590102b7df

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.39-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.39-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.39-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 ff9b751cc2315ca504988ff4a0a39ffabe4cb07a303417f194d5fb23cd9fd23e
MD5 af715466869c26e9710e4eb39deee999
BLAKE2b-256 9089452503821fc7bc2055e789c05ac282a87f26994595c65c416d150170fc4f

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.39-cp310-cp310-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.39-cp310-cp310-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.39-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 f9accdbf4c14a9ebb9d437894654a6d398bfcaf73965f0486c13bba648060b43
MD5 dc1d0445fa3652bcf1e15a79b94ea0f4
BLAKE2b-256 0ec5bc494f8c824d080112b1edd24865c377ec41b6323547d5d469b72c83ebfa

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.39-cp310-cp310-macosx_10_9_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.2.52

15 files

0.2.51

15 files

0.2.50

15 files

0.2.40

15 files

This release

0.2.39 This release

15 files

0.2.38

15 files

0.2.37

15 files

0.2.36

15 files

0.2.35

15 files

0.2.34

15 files

0.2.33

15 files

0.2.32

15 files

0.2.31

15 files

0.2.30

15 files

0.2.29

15 files

0.2.28

15 files

0.2.27

15 files

0.2.26

15 files

0.2.25

15 files

0.2.24

15 files

0.2.23

15 files

0.2.22

15 files

0.2.21

15 files

0.2.20

15 files

0.2.19

15 files

0.2.18

15 files

0.2.17

15 files

0.2.16

15 files

0.2.15

15 files

0.2.14

15 files

0.2.13

15 files

0.2.12

15 files

0.2.11

15 files

0.2.10

15 files

0.2.9

19 files

0.2.8

9 files

0.2.7

9 files

0.2.5

9 files

0.2.4

9 files

0.2.3

9 files

0.2.2

9 files

0.2.1

9 files

0.2.0

9 files

0.1.54

10 files

0.1.53

10 files

0.1.52

10 files

0.1.51

10 files

0.1.50

10 files

0.0.1

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page