Skip to main content

DVT logo

DVT — Data Virtualization Tool

Any source. Any target. One SQL project.

DVT is a cross-engine data transformation CLI built on dbt. Write SQL models that JOIN tables living on different database engines — MySQL with Snowflake, Oracle with PostgreSQL, anything with anything — and DVT handles extraction, federation, and loading automatically.

{{ config(materialized='f_table') }}   -- "f" = federated

select
    c.customer_name,
    o.order_date,
    o.total_amount
from {{ source('crm', 'customers') }} c      -- lives on MySQL
join {{ source('erp', 'orders') }} o          -- lives on Snowflake
  on c.customer_id = o.customer_id
where o.order_date >= '2024-01-01'
pip install dvt-core
dvt sync       # installs every adapter + driver your profiles.yml needs
dvt run        # dbt runs what dbt can; DVT federates the rest
dvt version    # (worth running once — you'll see)

Website & full docs: getdvt.net · Quickstart · PyPI


Architecture

DVT is a wrapper around stock dbt-core, not a fork. dbt compiles the project and runs every standard model; DVT picks up the models dbt can't express — cross-engine federation models and locally-executed Python models — and runs them through its own pipeline:

flowchart TB
    CLI([dvt run / build]) --> COMPILE["dbt compile → manifest.json"]
    COMPILE --> STD["standard models<br/>table · view · incremental"]
    COMPILE --> FED["federated models<br/>f_table · f_incremental · .py"]
    STD --> DBT["stock dbt executes them<br/>on the default target"]
    FED --> HOMO{"all sources on<br/>ONE connection?"}
    HOMO -- "yes — Sling direct" --> DIRECT["whole query transpiled to the
    source engine, streamed straight
    to the target — no staging"]
    HOMO -- "no — federation" --> DECOMP["1 · DECOMPOSE — SQLGlot maps every
    table to a profiles.yml connection"]
    DECOMP --> PUSH["2 · TRANSPILE + PUSH DOWN — minimal
    per-source queries, each engine's dialect"]
    PUSH --> EXTRACT["3 · EXTRACT — Sling → Parquet, parallel"]
    EXTRACT --> COMPUTE["4 · COMPUTE — DuckDB joins locally"]
    COMPUTE --> LOAD["5 · LOAD — Sling delivers to any
    database, warehouse, or bucket"]
    DIRECT --> TARGET[("target")]
    LOAD --> TARGET
    DBT --> TARGET

The Sling direct path (0.2.5)

When every source in an f_table model lives on one connection, steps 3–5 collapse into a single hop: the whole query — joins and aggregations included — is transpiled to that engine's dialect and submitted to Sling as a custom-SQL stream loaded straight to the target. No Parquet staging, no DuckDB. The check is automatic and per-run; any failure falls back to the standard pipeline. Add a source on another engine tomorrow and the same model silently switches to the federation pipeline — the model never changes.

Execution paths

Your model Who runs it How
table / view / incremental stock dbt natively on the default target — whole-query pushdown by definition
f_table / f_incremental, sources on one connection DVT Sling direct: full query pushed to the source engine, result streamed to the target
f_table / f_incremental, sources span connections DVT decompose → per-source pushdown extraction → DuckDB compute → load
.py models DVT locally executed; result loads to any target

Standard models that set a non-default target= are refused with guidance (dbt silently ignores the kwarg — your model would land on the wrong engine).

Design principles

  • Wrapper, not fork. DVT shells out to the dbt CLI and reads manifest.json — it never imports dbt internals. Your dbt version can move; DVT keeps working. Every dbt feature (Jinja, macros, packages, tests, snapshots, docs) works because it is dbt running it.
  • Dual-dialect by design. Federation models are written in DuckDB dialect and are transpiled per source — they never depend on any engine's syntax. Standard models compile in the default target's dialect via the official dbt adapter. Result: switching your default target (e.g. PostgreSQL → Databricks) is a profile change, not a rewrite.
  • Pushdown first. The fastest row to move is the one you never extract. Filters and column pruning push down to source engines; when every table in a model lives on one engine, the entire query pushes down and nothing moves at all.
  • Your hardware does the joining. Cross-engine compute happens in DuckDB on the machine running DVT — source databases only ever serve simple filtered SELECTs, and no warehouse bills you for the join.

What DVT adds on top of dbt

Capability How
Cross-engine models materialized='f_table' / 'f_incremental' (long aliases: federated_*, federation_*)
Incremental federation watermark-based delta extraction; persistent local cache between runs
Per-model targets config(target='other_output') on federated models — results land on any profile output (standard models are refused a foreign target)
Bucket targets materialize models to S3 / GCS / Azure as Parquet or CSV
Python models, locally dbt-style .py models run on your machine (APIs, pandas, ML) — results land on any target
Bulk seeds dvt seed loads DVT's four file formats — csv / parquet / json / jsonl — via Sling to any engine, with enforced snake_case columns
One-command setup dvt sync installs every adapter and driver your profiles.yml requires
Connection diagnostics dvt debug --all checks every connection on both layers (dbt adapter + Sling)
Cross-engine catalog dvt docs builds one catalog + lineage UI across all engines in your project

Everything else — profiles.yml in ~/.dbt/, sources.yml, ref/source/config, tests, snapshots — is standard dbt. Existing dbt projects work unchanged; sources gain an optional meta.connection to bind them to other engines.

Supported engines

Every engine supported by both dbt and Sling is a DVT adapter — as federation source, federation target, seed target, and default target:

PostgreSQL · MySQL · MariaDB · SQL Server · Oracle · SQLite · DuckDB · Snowflake · BigQuery · Redshift · Databricks · ClickHouse · Trino · Athena · Microsoft Fabric — plus S3, GCS, and Azure Blob as bucket storages.

Anything else (MongoDB, Elasticsearch, REST APIs, …) is reachable through locally-executed Python models. Details per adapter: getdvt.net/docs/adapters.

Learn more

License

DVT is distributed as compiled wheels on PyPI. Free tier covers local files, transactional databases, and Python models; Pro adds cloud buckets (S3/GCS/Azure); Max adds cloud warehouses. See getdvt.net/pricing.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

dvt_core-0.2.50-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (53.8 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.50-cp314-cp314-macosx_11_0_arm64.whl (8.2 MB view details)

Uploaded CPython 3.14macOS 11.0+ ARM64

dvt_core-0.2.50-cp314-cp314-macosx_10_15_x86_64.whl (8.3 MB view details)

Uploaded CPython 3.14macOS 10.15+ x86-64

dvt_core-0.2.50-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (54.5 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.50-cp313-cp313-macosx_11_0_arm64.whl (8.1 MB view details)

Uploaded CPython 3.13macOS 11.0+ ARM64

dvt_core-0.2.50-cp313-cp313-macosx_10_13_x86_64.whl (8.3 MB view details)

Uploaded CPython 3.13macOS 10.13+ x86-64

dvt_core-0.2.50-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (54.9 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.50-cp312-cp312-macosx_11_0_arm64.whl (8.1 MB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

dvt_core-0.2.50-cp312-cp312-macosx_10_13_x86_64.whl (8.3 MB view details)

Uploaded CPython 3.12macOS 10.13+ x86-64

dvt_core-0.2.50-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (56.3 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.50-cp311-cp311-macosx_11_0_arm64.whl (8.2 MB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

dvt_core-0.2.50-cp311-cp311-macosx_10_9_x86_64.whl (8.4 MB view details)

Uploaded CPython 3.11macOS 10.9+ x86-64

dvt_core-0.2.50-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (53.3 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.50-cp310-cp310-macosx_11_0_arm64.whl (8.2 MB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

dvt_core-0.2.50-cp310-cp310-macosx_10_9_x86_64.whl (8.5 MB view details)

Uploaded CPython 3.10macOS 10.9+ x86-64

File details

Details for the file dvt_core-0.2.50-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.50-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 cb987896032d5302752624e83165f41933c7d52c9c4fc75243e668d30dd34df0
MD5 99d78a5cac3866463b98aa84833c1d71
BLAKE2b-256 56889797a9183a8efbe8bc4f55a01fe623493e713c3e0cbfbeec2a55c97f54e6

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.50-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.50-cp314-cp314-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.50-cp314-cp314-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 329bda5041e09d9512a51fed092fa6b029e7c4abee37dc003c28c2ff56c9dbbd
MD5 0d2a1124e2591b43f4f00948aca00f07
BLAKE2b-256 10854628122aad1e68e436648e7fdf21fe67812296a84295ae515f579bbe92c5

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.50-cp314-cp314-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.50-cp314-cp314-macosx_10_15_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.50-cp314-cp314-macosx_10_15_x86_64.whl
Algorithm Hash digest
SHA256 d636df3a8a294661a89bf6ad798be8dd7c1bbadccb8d190104f81b2fcba367a3
MD5 0eaa93d5f76201c39a65e4e3c67bf58c
BLAKE2b-256 34e78974d34fb0c99728658f39c9c00a7313a1eb35f252525c0083ea2606314b

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.50-cp314-cp314-macosx_10_15_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.50-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.50-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 b96eefc0ac3f92aa4935e520be651d75775fdd11d2f07890ddab5a16b4e5f11a
MD5 d39653af266aa7ae6a8623bf3e75ccf7
BLAKE2b-256 69290eaff830b54d8f4304cef51a4846f753a1973f3690ab2cc52bb4ddb678ed

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.50-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.50-cp313-cp313-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.50-cp313-cp313-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 f974aa63e45bff20b8072618fd66b5279984d16db4ef9e7a56c8b62e4aa37f75
MD5 3fd426e69fda2d64ef50074c083cf820
BLAKE2b-256 72606cf6e4304001af51bd85117dd960f2bc8b5d8e9ad869541a806f78dbe0f5

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.50-cp313-cp313-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.50-cp313-cp313-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.50-cp313-cp313-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 d8c270264ca4507e1a8810449863103c8c7193b54dd3780333061b8b2c7145d1
MD5 f382bcf359d3b39b97f06a7d75e6b544
BLAKE2b-256 f9d0c1cb084d7b2778cfa77c418bb5a95a5dcf52b1adc480d5ed7a486146e18b

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.50-cp313-cp313-macosx_10_13_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.50-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.50-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 945f77720af5f9fce0c9490dd837a4d7aeb4e9d4f1dc4b003d7612e441577f4b
MD5 a31b1546dc53eb8834f539a1a1e65934
BLAKE2b-256 6b81e31fa1843d2d0734ddd37142ad541e05c3653b83327d8f2f1312d91a0ba2

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.50-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.50-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.50-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 b41ab32cb14173fea224518734bb52f85ac3b59915a3022cf46fcafe2df7b0bd
MD5 d09cf7efbfd121e9eb5e46a54c44fef6
BLAKE2b-256 bcbc8699e606b4fef3f5857273b4b9cec19ae67e01b0868c0d58973387fe13ff

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.50-cp312-cp312-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.50-cp312-cp312-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.50-cp312-cp312-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 27c9fc64b6b1ca48ea06288b12aa25788ded0dcbb4fc5f4b373bb3467c37edca
MD5 7906730d19441bb4b699501271094b38
BLAKE2b-256 e5b2b57d644cd48de7ba8b279b94ead85e53e675f7872ef54cb650321d6bb415

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.50-cp312-cp312-macosx_10_13_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.50-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.50-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 3333c8199665ac1ccd0399d24e9d029677f6fa5d0e46d5a4610a7f1cdcf8f76f
MD5 1445ce87d3a1da1e2e3de81907dcea36
BLAKE2b-256 7f73a59720a6b757ca693166beafb6aeb10cd6d39b1ab546a1afab1b63de562b

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.50-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.50-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.50-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 55e07eb559e22c468a34dc08426d8f7ce14a0739a8ceb056bf380b2a9d798f60
MD5 3c62c5b35406d0a42ae3ae9ed475ec46
BLAKE2b-256 7dbe6d49ff2a77753f28f10f58ed66cb6e69e286ce5e286258d3e55812be868f

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.50-cp311-cp311-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.50-cp311-cp311-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.50-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 f0c6788e04bfd0ce5caf6bf2ddd416de2ac6cea2b8eee13fe806cb0ae8ffb31c
MD5 225d16d66c202552f6c604f83ff2e960
BLAKE2b-256 8436ddfbc8bd640ea3e724a5866edbf06086b44ac0ea4c8f74ddb359f8951b73

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.50-cp311-cp311-macosx_10_9_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.50-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.50-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 66d9eae39f9d933d1c59560b758ecf4875591e98c754023053b3c0d9bde60fb4
MD5 f18e4cd312ff6cf6bf45c51dc6f7b6a6
BLAKE2b-256 02dfb6cb5053f7f51ef42d1cef18e62957560205d86ef8dd14faffe3c5d5c65a

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.50-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.50-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.50-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 4b1ff8e5394548cee3e76ecb52a71ccc9c1d7b7115961283eedb488b0c533993
MD5 e4ea4faffebd51f0bb997be344048b97
BLAKE2b-256 01dc35b39ad03d38181391fd6581b6f56b095ea98cfcdd0ac12c0f7c854e99bd

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.50-cp310-cp310-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.50-cp310-cp310-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.50-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 d5c5848650b152130bfcd822ba162ebb6189817b6136dfec5bf487e968be5ac6
MD5 bbb821e1343629efc26bd99b248939bb
BLAKE2b-256 910dd71bbc703e3d2a3a769f2aba8b4cbf93d0d256ccc5dec885ea8e96391fdc

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.50-cp310-cp310-macosx_10_9_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.2.52

15 files

0.2.51

15 files

This release

0.2.50 This release

15 files

0.2.40

15 files

0.2.39

15 files

0.2.38

15 files

0.2.37

15 files

0.2.36

15 files

0.2.35

15 files

0.2.34

15 files

0.2.33

15 files

0.2.32

15 files

0.2.31

15 files

0.2.30

15 files

0.2.29

15 files

0.2.28

15 files

0.2.27

15 files

0.2.26

15 files

0.2.25

15 files

0.2.24

15 files

0.2.23

15 files

0.2.22

15 files

0.2.21

15 files

0.2.20

15 files

0.2.19

15 files

0.2.18

15 files

0.2.17

15 files

0.2.16

15 files

0.2.15

15 files

0.2.14

15 files

0.2.13

15 files

0.2.12

15 files

0.2.11

15 files

0.2.10

15 files

0.2.9

19 files

0.2.8

9 files

0.2.7

9 files

0.2.5

9 files

0.2.4

9 files

0.2.3

9 files

0.2.2

9 files

0.2.1

9 files

0.2.0

9 files

0.1.54

10 files

0.1.53

10 files

0.1.52

10 files

0.1.51

10 files

0.1.50

10 files

0.0.1

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page