Skip to main content

DVT logo

DVT — Data Virtualization Tool

Any source. Any target. One SQL project.

DVT is a cross-engine data transformation CLI built on dbt. Write SQL models that JOIN tables living on different database engines — MySQL with Snowflake, Oracle with PostgreSQL, anything with anything — and DVT handles extraction, federation, and loading automatically.

{{ config(materialized='f_table') }}   -- "f" = federated

select
    c.customer_name,
    o.order_date,
    o.total_amount
from {{ source('crm', 'customers') }} c      -- lives on MySQL
join {{ source('erp', 'orders') }} o          -- lives on Snowflake
  on c.customer_id = o.customer_id
where o.order_date >= '2024-01-01'
pip install dvt-core
dvt sync       # installs every adapter + driver your profiles.yml needs
dvt run        # dbt runs what dbt can; DVT federates the rest
dvt version    # (worth running once — you'll see)

Website & full docs: getdvt.net · Quickstart · PyPI


Architecture

DVT is a wrapper around stock dbt-core, not a fork. dbt compiles the project and runs every standard model; DVT picks up the models dbt can't express — cross-engine federation models and locally-executed Python models — and runs them through its own pipeline:

flowchart TB
    CLI([dvt run / build]) --> COMPILE["dbt compile → manifest.json"]
    COMPILE --> STD["standard models<br/>table · view · incremental"]
    COMPILE --> FED["federated models<br/>f_table · f_incremental · .py"]
    STD --> DBT["stock dbt executes them<br/>on the default target"]
    FED --> HOMO{"all sources on<br/>ONE connection?"}
    HOMO -- "yes — Sling direct" --> DIRECT["whole query transpiled to the
    source engine, streamed straight
    to the target — no staging"]
    HOMO -- "no — federation" --> DECOMP["1 · DECOMPOSE — SQLGlot maps every
    table to a profiles.yml connection"]
    DECOMP --> PUSH["2 · TRANSPILE + PUSH DOWN — minimal
    per-source queries, each engine's dialect"]
    PUSH --> EXTRACT["3 · EXTRACT — Sling → Parquet, parallel"]
    EXTRACT --> COMPUTE["4 · COMPUTE — DuckDB joins locally"]
    COMPUTE --> LOAD["5 · LOAD — Sling delivers to any
    database, warehouse, or bucket"]
    DIRECT --> TARGET[("target")]
    LOAD --> TARGET
    DBT --> TARGET

The Sling direct path (0.2.5)

When every source in an f_table model lives on one connection, steps 3–5 collapse into a single hop: the whole query — joins and aggregations included — is transpiled to that engine's dialect and submitted to Sling as a custom-SQL stream loaded straight to the target. No Parquet staging, no DuckDB. The check is automatic and per-run; any failure falls back to the standard pipeline. Add a source on another engine tomorrow and the same model silently switches to the federation pipeline — the model never changes.

Execution paths

Your model Who runs it How
table / view / incremental stock dbt natively on the default target — whole-query pushdown by definition
f_table / f_incremental, sources on one connection DVT Sling direct: full query pushed to the source engine, result streamed to the target
f_table / f_incremental, sources span connections DVT decompose → per-source pushdown extraction → DuckDB compute → load
.py models DVT locally executed; result loads to any target

Standard models that set a non-default target= are refused with guidance (dbt silently ignores the kwarg — your model would land on the wrong engine).

Design principles

  • Wrapper, not fork. DVT shells out to the dbt CLI and reads manifest.json — it never imports dbt internals. Your dbt version can move; DVT keeps working. Every dbt feature (Jinja, macros, packages, tests, snapshots, docs) works because it is dbt running it.
  • Dual-dialect by design. Federation models are written in DuckDB dialect and are transpiled per source — they never depend on any engine's syntax. Standard models compile in the default target's dialect via the official dbt adapter. Result: switching your default target (e.g. PostgreSQL → Databricks) is a profile change, not a rewrite.
  • Pushdown first. The fastest row to move is the one you never extract. Filters and column pruning push down to source engines; when every table in a model lives on one engine, the entire query pushes down and nothing moves at all.
  • Your hardware does the joining. Cross-engine compute happens in DuckDB on the machine running DVT — source databases only ever serve simple filtered SELECTs, and no warehouse bills you for the join.

What DVT adds on top of dbt

Capability How
Cross-engine models materialized='f_table' / 'f_incremental' (long aliases: federated_*, federation_*)
Incremental federation watermark-based delta extraction; persistent local cache between runs
Per-model targets config(target='other_output') on federated models — results land on any profile output (standard models are refused a foreign target)
Bucket targets materialize models to S3 / GCS / Azure as Parquet or CSV
Python models, locally dbt-style .py models run on your machine (APIs, pandas, ML) — results land on any target
Bulk seeds dvt seed loads csv / tsv / parquet / json / jsonl / avro / arrow / xlsx via Sling to any engine, with enforced snake_case columns
One-command setup dvt sync installs every adapter and driver your profiles.yml requires
Connection diagnostics dvt debug --all checks every connection on both layers (dbt adapter + Sling)
Cross-engine catalog dvt docs builds one catalog + lineage UI across all engines in your project

Everything else — profiles.yml in ~/.dbt/, sources.yml, ref/source/config, tests, snapshots — is standard dbt. Existing dbt projects work unchanged; sources gain an optional meta.connection to bind them to other engines.

Supported engines

Every engine supported by both dbt and Sling is a DVT adapter — as federation source, federation target, seed target, and default target:

PostgreSQL · MySQL · MariaDB · SQL Server · Oracle · SQLite · DuckDB · Snowflake · BigQuery · Redshift · Databricks · ClickHouse · Trino · Athena · Microsoft Fabric — plus S3, GCS, and Azure Blob as bucket storages.

Anything else (MongoDB, Elasticsearch, REST APIs, …) is reachable through locally-executed Python models. Details per adapter: getdvt.net/docs/adapters.

Learn more

License

DVT is distributed as compiled wheels on PyPI. Free tier covers local files, transactional databases, and Python models; Pro adds cloud buckets (S3/GCS/Azure); Max adds cloud warehouses. See getdvt.net/pricing.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

dvt_core-0.2.40-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (28.5 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.40-cp314-cp314-macosx_11_0_arm64.whl (4.5 MB view details)

Uploaded CPython 3.14macOS 11.0+ ARM64

dvt_core-0.2.40-cp314-cp314-macosx_10_15_x86_64.whl (4.6 MB view details)

Uploaded CPython 3.14macOS 10.15+ x86-64

dvt_core-0.2.40-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (28.9 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.40-cp313-cp313-macosx_11_0_arm64.whl (4.5 MB view details)

Uploaded CPython 3.13macOS 11.0+ ARM64

dvt_core-0.2.40-cp313-cp313-macosx_10_13_x86_64.whl (4.6 MB view details)

Uploaded CPython 3.13macOS 10.13+ x86-64

dvt_core-0.2.40-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (29.1 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.40-cp312-cp312-macosx_11_0_arm64.whl (4.5 MB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

dvt_core-0.2.40-cp312-cp312-macosx_10_13_x86_64.whl (4.6 MB view details)

Uploaded CPython 3.12macOS 10.13+ x86-64

dvt_core-0.2.40-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (30.0 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.40-cp311-cp311-macosx_11_0_arm64.whl (4.5 MB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

dvt_core-0.2.40-cp311-cp311-macosx_10_9_x86_64.whl (4.7 MB view details)

Uploaded CPython 3.11macOS 10.9+ x86-64

dvt_core-0.2.40-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (28.3 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.40-cp310-cp310-macosx_11_0_arm64.whl (4.6 MB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

dvt_core-0.2.40-cp310-cp310-macosx_10_9_x86_64.whl (4.7 MB view details)

Uploaded CPython 3.10macOS 10.9+ x86-64

File details

Details for the file dvt_core-0.2.40-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.40-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 de8b52bc0b14bc8692cc650bbfacfe4fa034818a079a5aeef00a2bb57030f43e
MD5 798c3e68182398b9bfe44df7e775b269
BLAKE2b-256 e98b92f1e4e09790a909c2e4bdbe35694195f8eed8e5da3cded6c41e3ba75a3c

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.40-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.40-cp314-cp314-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.40-cp314-cp314-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 3d2f7cc362ef5bcf1a04bbceae0fb782c9649c52122e488e20effcdd96aef73c
MD5 980d67b6fe3cef8292445b73ee688bf1
BLAKE2b-256 cfc0380c3280a24e142905656be275c23278a17d9ab7aae4c7705aa6b74cbfdb

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.40-cp314-cp314-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.40-cp314-cp314-macosx_10_15_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.40-cp314-cp314-macosx_10_15_x86_64.whl
Algorithm Hash digest
SHA256 c59079c6d6b509ca06122db5e1be88206f27f8dbd88f215d736807d05d7b9d57
MD5 57d35c3804b1a8c640cddd7d90bbb06d
BLAKE2b-256 fe9e97f7bbb7b9aa4e848e0314324ed89b99828121a0ae4c1276de400d2d3fad

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.40-cp314-cp314-macosx_10_15_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.40-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.40-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 d4076be404e4a30de5f47a6364385af069f086f2fa92f53a7a40fe3845cdc49b
MD5 05046ebb0608cfc9fdd4be6209b8a789
BLAKE2b-256 ffcd475af08c3256eea142ad84934978878dae61e58ad3713eb488f21ef98485

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.40-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.40-cp313-cp313-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.40-cp313-cp313-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 0eeb0567e4bd0742b62e88540d6279fbc2c9da38270c773d252838d2ea2e9b6e
MD5 90f322dc2dac45e9f6be2db390bec6be
BLAKE2b-256 64d45edd2e8134c92864caf4bea61404f37993d9629d9adcb9ee4285e8ba18e4

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.40-cp313-cp313-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.40-cp313-cp313-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.40-cp313-cp313-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 5e639b3feb4659371386d289ee043cdfc0e7561d7834bfd815cd31f51672f266
MD5 85450e7b0af2f5f01afb217de87d4a56
BLAKE2b-256 784ba156f9faaa607aeac7f9a8675e8803a11463445b1c8b150894862a015037

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.40-cp313-cp313-macosx_10_13_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.40-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.40-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 294a27b2f9b822892b2e42cd5fd051615e30f66ded692c11cdf397e0addc73ec
MD5 d6edb70e6e6a839323571b6b19e039ba
BLAKE2b-256 688eb4b53d5847f190c199058087f9cea75fa44758796ccb0c4d2ac03d9e8ba3

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.40-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.40-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.40-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 2dfea0c01e3e87dcf6e876699e3dca13c3ea7585273be4223e29ac360a3a0d00
MD5 7f9ee704cb10ba9d597457f95ee1e2f4
BLAKE2b-256 e76e2b3e1fcb438d37b42fb20b237b9f00ebf62d98878d16af3ed9ccc118da9b

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.40-cp312-cp312-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.40-cp312-cp312-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.40-cp312-cp312-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 f6d1033c598ed6d69686e949ca6e838c4f5a831bbc7fb65130154b6b1a1f0b3a
MD5 fec8eb2a5d6679f6ce5d5342bdd80c36
BLAKE2b-256 f41680a4e600dbf78c9d51548052b61a6fd33e3dd2594fb3d1936409aa1a9f82

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.40-cp312-cp312-macosx_10_13_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.40-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.40-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 6b87e4216934f550f31480e5be233e4d8fc6d9c32a37f38084d2f406f60d6e06
MD5 4ba7f76d239f508e5889fae79657dc92
BLAKE2b-256 8c15635d31b0a9a91d877fbe9cfec055811ef408dce6410f0a162c5a30f1a94a

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.40-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.40-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.40-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 a2d7b772781a62248ed05b6693220b2342630ba87fe72800ef21056e61f93e18
MD5 695a12cb61f27fb4abb69fd6634e9557
BLAKE2b-256 c87697586e13b894e04007da78d7ebc1dc4de08c7c86fed1e5738c36f358acf6

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.40-cp311-cp311-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.40-cp311-cp311-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.40-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 6606585bb8d883e225542bc0894bf1255645ea0b5d4204b70988748835bde17c
MD5 62499ab929415a88d62e38c1cf853fb7
BLAKE2b-256 b625355ad4c77f8b18f10dd310c3d72c331f7f8adc41d8fef79c16a8add3c2fd

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.40-cp311-cp311-macosx_10_9_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.40-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.40-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 6885cb25d0439b2743ecd84a3dd738c5a95c6ac39a53d4fb9b562f5ab374e77c
MD5 619e718b82f20ff860541e3b649c4f4b
BLAKE2b-256 5882eff0acf52e9b641fd2481dbb084b5097eca1c6b217aedbb901fc549b8f61

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.40-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.40-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.40-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 8f6daba436cce93390cd9ff2df019d6b5b879c228ff84a0650606d07b9ab45a8
MD5 6bb0cc3b6e402ff6caf4e549951b25eb
BLAKE2b-256 70ccb1e2609e080b9c27726653e0d429e1f5f0d6dd6fc38f4e8791c66ec0f09b

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.40-cp310-cp310-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.40-cp310-cp310-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.40-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 33e3587bcb98cc69006884c924b1e13ce101a15823e45c490c137ef3817fd6b8
MD5 245a47e6315f20c6621d80a9c30635b8
BLAKE2b-256 86ce1e2f908fa6155f4d3a81dc3d862c918a6a773db5cdc5d10e27fc6d7b17e7

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.40-cp310-cp310-macosx_10_9_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.2.52

15 files

0.2.51

15 files

0.2.50

15 files

This release

0.2.40 This release

15 files

0.2.39

15 files

0.2.38

15 files

0.2.37

15 files

0.2.36

15 files

0.2.35

15 files

0.2.34

15 files

0.2.33

15 files

0.2.32

15 files

0.2.31

15 files

0.2.30

15 files

0.2.29

15 files

0.2.28

15 files

0.2.27

15 files

0.2.26

15 files

0.2.25

15 files

0.2.24

15 files

0.2.23

15 files

0.2.22

15 files

0.2.21

15 files

0.2.20

15 files

0.2.19

15 files

0.2.18

15 files

0.2.17

15 files

0.2.16

15 files

0.2.15

15 files

0.2.14

15 files

0.2.13

15 files

0.2.12

15 files

0.2.11

15 files

0.2.10

15 files

0.2.9

19 files

0.2.8

9 files

0.2.7

9 files

0.2.5

9 files

0.2.4

9 files

0.2.3

9 files

0.2.2

9 files

0.2.1

9 files

0.2.0

9 files

0.1.54

10 files

0.1.53

10 files

0.1.52

10 files

0.1.51

10 files

0.1.50

10 files

0.0.1

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page