Skip to main content

DVT logo

DVT — Data Virtualization Tool

Any source. Any target. One SQL project.

DVT is a cross-engine data transformation CLI built on dbt. Write SQL models that JOIN tables living on different database engines — MySQL with Snowflake, Oracle with PostgreSQL, anything with anything — and DVT handles extraction, federation, and loading automatically.

{{ config(materialized='f_table') }}   -- "f" = federated

select
    c.customer_name,
    o.order_date,
    o.total_amount
from {{ source('crm', 'customers') }} c      -- lives on MySQL
join {{ source('erp', 'orders') }} o          -- lives on Snowflake
  on c.customer_id = o.customer_id
where o.order_date >= '2024-01-01'
pip install dvt-core
dvt sync       # installs every adapter + driver your profiles.yml needs
dvt run        # dbt runs what dbt can; DVT federates the rest
dvt version    # (worth running once — you'll see)

Website & full docs: getdvt.net · Quickstart · PyPI


Architecture

DVT is a wrapper around stock dbt-core, not a fork. dbt compiles the project and runs every standard model; DVT picks up the models dbt can't express — cross-engine federation models and locally-executed Python models — and runs them through its own pipeline:

flowchart TB
    CLI([dvt run / build]) --> COMPILE["dbt compile → manifest.json"]
    COMPILE --> STD["standard models<br/>table · view · incremental"]
    COMPILE --> FED["federated models<br/>f_table · f_incremental · .py"]
    STD --> DBT["stock dbt executes them<br/>on the default target"]
    FED --> HOMO{"all sources on<br/>ONE connection?"}
    HOMO -- "yes — Sling direct" --> DIRECT["whole query transpiled to the
    source engine, streamed straight
    to the target — no staging"]
    HOMO -- "no — federation" --> DECOMP["1 · DECOMPOSE — SQLGlot maps every
    table to a profiles.yml connection"]
    DECOMP --> PUSH["2 · TRANSPILE + PUSH DOWN — minimal
    per-source queries, each engine's dialect"]
    PUSH --> EXTRACT["3 · EXTRACT — Sling → Parquet, parallel"]
    EXTRACT --> COMPUTE["4 · COMPUTE — DuckDB joins locally"]
    COMPUTE --> LOAD["5 · LOAD — Sling delivers to any
    database, warehouse, or bucket"]
    DIRECT --> TARGET[("target")]
    LOAD --> TARGET
    DBT --> TARGET

The Sling direct path (0.2.5)

When every source in an f_table model lives on one connection, steps 3–5 collapse into a single hop: the whole query — joins and aggregations included — is transpiled to that engine's dialect and submitted to Sling as a custom-SQL stream loaded straight to the target. No Parquet staging, no DuckDB. The check is automatic and per-run; any failure falls back to the standard pipeline. Add a source on another engine tomorrow and the same model silently switches to the federation pipeline — the model never changes.

Execution paths

Your model Who runs it How
table / view / incremental stock dbt natively on the default target — whole-query pushdown by definition
f_table / f_incremental, sources on one connection DVT Sling direct: full query pushed to the source engine, result streamed to the target
f_table / f_incremental, sources span connections DVT decompose → per-source pushdown extraction → DuckDB compute → load
.py models DVT locally executed; result loads to any target

Standard models that set a non-default target= are refused with guidance (dbt silently ignores the kwarg — your model would land on the wrong engine).

Design principles

  • Wrapper, not fork. DVT shells out to the dbt CLI and reads manifest.json — it never imports dbt internals. Your dbt version can move; DVT keeps working. Every dbt feature (Jinja, macros, packages, tests, snapshots, docs) works because it is dbt running it.
  • Dual-dialect by design. Federation models are written in DuckDB dialect and are transpiled per source — they never depend on any engine's syntax. Standard models compile in the default target's dialect via the official dbt adapter. Result: switching your default target (e.g. PostgreSQL → Databricks) is a profile change, not a rewrite.
  • Pushdown first. The fastest row to move is the one you never extract. Filters and column pruning push down to source engines; when every table in a model lives on one engine, the entire query pushes down and nothing moves at all.
  • Your hardware does the joining. Cross-engine compute happens in DuckDB on the machine running DVT — source databases only ever serve simple filtered SELECTs, and no warehouse bills you for the join.

What DVT adds on top of dbt

Capability How
Cross-engine models materialized='f_table' / 'f_incremental' (long aliases: federated_*, federation_*)
Incremental federation watermark-based delta extraction; persistent local cache between runs
Per-model targets config(target='other_output') on federated models — results land on any profile output (standard models are refused a foreign target)
Bucket targets materialize models to S3 / GCS / Azure as Parquet or CSV
Python models, locally dbt-style .py models run on your machine (APIs, pandas, ML) — results land on any target
Bulk seeds dvt seed loads csv / tsv / parquet / json / jsonl / avro / arrow / xlsx via Sling to any engine, with enforced snake_case columns
One-command setup dvt sync installs every adapter and driver your profiles.yml requires
Connection diagnostics dvt debug --all checks every connection on both layers (dbt adapter + Sling)
Cross-engine catalog dvt docs builds one catalog + lineage UI across all engines in your project

Everything else — profiles.yml in ~/.dbt/, sources.yml, ref/source/config, tests, snapshots — is standard dbt. Existing dbt projects work unchanged; sources gain an optional meta.connection to bind them to other engines.

Supported engines

Every engine supported by both dbt and Sling is a DVT adapter — as federation source, federation target, seed target, and default target:

PostgreSQL · MySQL · MariaDB · SQL Server · Oracle · SQLite · DuckDB · Snowflake · BigQuery · Redshift · Databricks · ClickHouse · Trino · Athena · Microsoft Fabric — plus S3, GCS, and Azure Blob as bucket storages.

Anything else (MongoDB, Elasticsearch, REST APIs, …) is reachable through locally-executed Python models. Details per adapter: getdvt.net/docs/adapters.

Learn more

License

DVT is distributed as compiled wheels on PyPI. Free tier covers local files, transactional databases, and Python models; Pro adds cloud buckets (S3/GCS/Azure); Max adds cloud warehouses. See getdvt.net/pricing.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

dvt_core-0.2.38-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (24.6 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.38-cp314-cp314-macosx_11_0_arm64.whl (4.0 MB view details)

Uploaded CPython 3.14macOS 11.0+ ARM64

dvt_core-0.2.38-cp314-cp314-macosx_10_15_x86_64.whl (4.0 MB view details)

Uploaded CPython 3.14macOS 10.15+ x86-64

dvt_core-0.2.38-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (24.9 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.38-cp313-cp313-macosx_11_0_arm64.whl (4.0 MB view details)

Uploaded CPython 3.13macOS 11.0+ ARM64

dvt_core-0.2.38-cp313-cp313-macosx_10_13_x86_64.whl (4.0 MB view details)

Uploaded CPython 3.13macOS 10.13+ x86-64

dvt_core-0.2.38-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (25.0 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.38-cp312-cp312-macosx_11_0_arm64.whl (4.0 MB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

dvt_core-0.2.38-cp312-cp312-macosx_10_13_x86_64.whl (4.1 MB view details)

Uploaded CPython 3.12macOS 10.13+ x86-64

dvt_core-0.2.38-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (25.9 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.38-cp311-cp311-macosx_11_0_arm64.whl (4.0 MB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

dvt_core-0.2.38-cp311-cp311-macosx_10_9_x86_64.whl (4.1 MB view details)

Uploaded CPython 3.11macOS 10.9+ x86-64

dvt_core-0.2.38-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (24.4 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.38-cp310-cp310-macosx_11_0_arm64.whl (4.0 MB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

dvt_core-0.2.38-cp310-cp310-macosx_10_9_x86_64.whl (4.1 MB view details)

Uploaded CPython 3.10macOS 10.9+ x86-64

File details

Details for the file dvt_core-0.2.38-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.38-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 a47473ef43710fdbaa4f0767f27d46a99d8c65ebda2e3b450f883014f9d63e37
MD5 1c69f047feac173502ad85465824509d
BLAKE2b-256 41d4263effec5c225ef7a127948f4e85af0603048a7d7aa12e5a16e747b23924

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.38-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.38-cp314-cp314-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.38-cp314-cp314-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 696b8ac22786ee44076a2eaaf40483c5f5d40750e20a57a6fe3d46bcc755cdab
MD5 04b90e953d272762edc7763ee0ad97f0
BLAKE2b-256 899d8ea21d6d4b8ba94e74bb899f9d1f706cc15ea2b479daf45c0941821b57e7

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.38-cp314-cp314-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.38-cp314-cp314-macosx_10_15_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.38-cp314-cp314-macosx_10_15_x86_64.whl
Algorithm Hash digest
SHA256 a2139e455d546585249c210034ef89f526b05445c6a304da6d34f451abb0f061
MD5 0b9bf9b481e92d3163c3564653b1875f
BLAKE2b-256 6c6aa84209dd13c0b699e254ef8c644943cf0db35b8868d20314a76ba5e8614b

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.38-cp314-cp314-macosx_10_15_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.38-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.38-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 5f36f0dac69ce05b9a22f83c8f63f60bcee39bc8bf18dc1e0afab9c23f6382b2
MD5 35a045020d35860e4e41b1d0cc7b50b4
BLAKE2b-256 add8685ff6caba18c385edff3febc631a988402e10db2f9821ed813b84e9046e

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.38-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.38-cp313-cp313-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.38-cp313-cp313-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 f774f2423307528ce64b535926e5b03e7c31aed873234b248dc3410c05628f43
MD5 146fe9e06e6132453de2490c6dbfe488
BLAKE2b-256 769870be2194b79b240d5e50f27220d9074202e82791b8ac15bcaa86dd9faddf

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.38-cp313-cp313-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.38-cp313-cp313-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.38-cp313-cp313-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 94ac357efab61def127cf627702bff9a6f3b3c079037032547f431cf774c07aa
MD5 b9b221340766e53f2b9eea61a0a6e88d
BLAKE2b-256 a3d0173a833f03a1a9978e552a29185d0e187fc2832541f3e8484e7a4c9cf9f3

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.38-cp313-cp313-macosx_10_13_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.38-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.38-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 039a21f0f9269f0eaa23067852a6003c215df476b027eb781a35cd6aefe0cfbb
MD5 8e700002065ee61d7ee00c84cef3cade
BLAKE2b-256 c12f0ddb54ddf8d6934db4e9bcf0ed5079c0601a10774d5591dc60d6742b43b0

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.38-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.38-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.38-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 cb450a7747263e9f0b10cb81aef2d2752275c7d8050e12014ba2fc28748ec8bc
MD5 d781241767163e52030e6095a5134477
BLAKE2b-256 8324682b84df420560a37edd9dbc6851f218575fcbde5d3b59e8bce4617e600f

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.38-cp312-cp312-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.38-cp312-cp312-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.38-cp312-cp312-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 41017390f16d1cfa1f56045c7efaf8822c7ce0c8230342c5713dce8916e729f9
MD5 1644dee74744009b3c640569cecc4d8f
BLAKE2b-256 4062a5af07874ee6243d509c67762799c6a48be8762d9198454fccaf386dee8a

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.38-cp312-cp312-macosx_10_13_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.38-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.38-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 72c1bdc75a8ed15fbd85b3d7b3a98b138ca79e12d57f383377824460cf5e472b
MD5 948bf7a73830bb0d6c4e3ffed6ffd02e
BLAKE2b-256 02bffd2151e83ee00c1bbaecb2bd1f32780bf2e7ddfb912365e4f1b76a4a5d19

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.38-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.38-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.38-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 58e167ba54f96af2fd6a97ea246ca3a7f4f18a7a274f56a3a7d05c5148f7200a
MD5 b768bde1938ff6c1fad40e7d5c4c50cb
BLAKE2b-256 7d822de1835a2b5e02c651ff1e8380c9b6640a31b77440442538b82eff6c7455

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.38-cp311-cp311-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.38-cp311-cp311-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.38-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 9a0e69a457840dcbb00488b1507707e7885abd92c4bcf89057d5d55532f7be1e
MD5 10efcfa95b0a9f48e099d89e9b07beb9
BLAKE2b-256 4f4c957a063247b0a5ad924815002153ff1e73edfa269f0ae07226fff9425fcf

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.38-cp311-cp311-macosx_10_9_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.38-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.38-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 9fe4e38ac9c9ff6d5de0ab46f442fd77111f665daa260fdf00bf7a54ed438575
MD5 8105579de6b3875102508c3b45cbafac
BLAKE2b-256 006ea329e29d78c691d422ef684cbb72f3782261b5334a3aa871501ee9972c5d

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.38-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.38-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.38-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 7f48afa08a20e87c229409acefe4e3db364d8066d4540ea17089067652d5054d
MD5 4d90b55320213e177a462fdaf8ad5e16
BLAKE2b-256 c7c5010e472359175e65eefdf847f2253b66bc047e16ae87c5b204ed469847e3

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.38-cp310-cp310-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.38-cp310-cp310-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.38-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 624419ebc255bcb1e271de26b6714620077309ed16f4fdbbb33d75ceab2fd460
MD5 ca9de528f38c4057689712fbb4c5e19b
BLAKE2b-256 8c09db2173fae47b224ae3db00f48c5e87f030ecfc87d77bf26832e8b2006105

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.38-cp310-cp310-macosx_10_9_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.2.52

15 files

0.2.51

15 files

0.2.50

15 files

0.2.40

15 files

0.2.39

15 files

This release

0.2.38 This release

15 files

0.2.37

15 files

0.2.36

15 files

0.2.35

15 files

0.2.34

15 files

0.2.33

15 files

0.2.32

15 files

0.2.31

15 files

0.2.30

15 files

0.2.29

15 files

0.2.28

15 files

0.2.27

15 files

0.2.26

15 files

0.2.25

15 files

0.2.24

15 files

0.2.23

15 files

0.2.22

15 files

0.2.21

15 files

0.2.20

15 files

0.2.19

15 files

0.2.18

15 files

0.2.17

15 files

0.2.16

15 files

0.2.15

15 files

0.2.14

15 files

0.2.13

15 files

0.2.12

15 files

0.2.11

15 files

0.2.10

15 files

0.2.9

19 files

0.2.8

9 files

0.2.7

9 files

0.2.5

9 files

0.2.4

9 files

0.2.3

9 files

0.2.2

9 files

0.2.1

9 files

0.2.0

9 files

0.1.54

10 files

0.1.53

10 files

0.1.52

10 files

0.1.51

10 files

0.1.50

10 files

0.0.1

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page