Skip to main content

DVT logo

DVT — Data Virtualization Tool

Any source. Any target. One SQL project.

DVT is a cross-engine data transformation CLI built on dbt. Write SQL models that JOIN tables living on different database engines — MySQL with Snowflake, Oracle with PostgreSQL, anything with anything — and DVT handles extraction, federation, and loading automatically.

{{ config(materialized='f_table') }}   -- "f" = federated

select
    c.customer_name,
    o.order_date,
    o.total_amount
from {{ source('crm', 'customers') }} c      -- lives on MySQL
join {{ source('erp', 'orders') }} o          -- lives on Snowflake
  on c.customer_id = o.customer_id
where o.order_date >= '2024-01-01'
pip install dvt-core
dvt sync       # installs every adapter + driver your profiles.yml needs
dvt run        # dbt runs what dbt can; DVT federates the rest
dvt version    # (worth running once — you'll see)

Website & full docs: getdvt.net · Quickstart · PyPI


Architecture

DVT is a wrapper around stock dbt-core, not a fork. dbt compiles the project and runs every standard model; DVT picks up the models dbt can't express — cross-engine federation models and locally-executed Python models — and runs them through its own pipeline:

flowchart TB
    CLI([dvt run / build]) --> COMPILE["dbt compile → manifest.json"]
    COMPILE --> STD["standard models<br/>table · view · incremental"]
    COMPILE --> FED["federated models<br/>f_table · f_incremental · .py"]
    STD --> DBT["stock dbt executes them<br/>on the default target"]
    FED --> HOMO{"all sources on<br/>ONE connection?"}
    HOMO -- "yes — Sling direct" --> DIRECT["whole query transpiled to the
    source engine, streamed straight
    to the target — no staging"]
    HOMO -- "no — federation" --> DECOMP["1 · DECOMPOSE — SQLGlot maps every
    table to a profiles.yml connection"]
    DECOMP --> PUSH["2 · TRANSPILE + PUSH DOWN — minimal
    per-source queries, each engine's dialect"]
    PUSH --> EXTRACT["3 · EXTRACT — Sling → Parquet, parallel"]
    EXTRACT --> COMPUTE["4 · COMPUTE — DuckDB joins locally"]
    COMPUTE --> LOAD["5 · LOAD — Sling delivers to any
    database, warehouse, or bucket"]
    DIRECT --> TARGET[("target")]
    LOAD --> TARGET
    DBT --> TARGET

The Sling direct path (0.2.5)

When every source in an f_table model lives on one connection, steps 3–5 collapse into a single hop: the whole query — joins and aggregations included — is transpiled to that engine's dialect and submitted to Sling as a custom-SQL stream loaded straight to the target. No Parquet staging, no DuckDB. The check is automatic and per-run; any failure falls back to the standard pipeline. Add a source on another engine tomorrow and the same model silently switches to the federation pipeline — the model never changes.

Execution paths

Your model Who runs it How
table / view / incremental stock dbt natively on the default target — whole-query pushdown by definition
f_table / f_incremental, sources on one connection DVT Sling direct: full query pushed to the source engine, result streamed to the target
f_table / f_incremental, sources span connections DVT decompose → per-source pushdown extraction → DuckDB compute → load
.py models DVT locally executed; result loads to any target

Standard models that set a non-default target= are refused with guidance (dbt silently ignores the kwarg — your model would land on the wrong engine).

Design principles

  • Wrapper, not fork. DVT shells out to the dbt CLI and reads manifest.json — it never imports dbt internals. Your dbt version can move; DVT keeps working. Every dbt feature (Jinja, macros, packages, tests, snapshots, docs) works because it is dbt running it.
  • Dual-dialect by design. Federation models are written in DuckDB dialect and are transpiled per source — they never depend on any engine's syntax. Standard models compile in the default target's dialect via the official dbt adapter. Result: switching your default target (e.g. PostgreSQL → Databricks) is a profile change, not a rewrite.
  • Pushdown first. The fastest row to move is the one you never extract. Filters and column pruning push down to source engines; when every table in a model lives on one engine, the entire query pushes down and nothing moves at all.
  • Your hardware does the joining. Cross-engine compute happens in DuckDB on the machine running DVT — source databases only ever serve simple filtered SELECTs, and no warehouse bills you for the join.

What DVT adds on top of dbt

Capability How
Cross-engine models materialized='f_table' / 'f_incremental' (long aliases: federated_*, federation_*)
Incremental federation watermark-based delta extraction; persistent local cache between runs
Per-model targets config(target='other_output') on federated models — results land on any profile output (standard models are refused a foreign target)
Bucket targets materialize models to S3 / GCS / Azure as Parquet or CSV
Python models, locally dbt-style .py models run on your machine (APIs, pandas, ML) — results land on any target
Bulk seeds dvt seed loads csv / tsv / parquet / json / jsonl / avro / arrow / xlsx via Sling to any engine, with enforced snake_case columns
One-command setup dvt sync installs every adapter and driver your profiles.yml requires
Connection diagnostics dvt debug --all checks every connection on both layers (dbt adapter + Sling)
Cross-engine catalog dvt docs builds one catalog + lineage UI across all engines in your project

Everything else — profiles.yml in ~/.dbt/, sources.yml, ref/source/config, tests, snapshots — is standard dbt. Existing dbt projects work unchanged; sources gain an optional meta.connection to bind them to other engines.

Supported engines

Every engine supported by both dbt and Sling is a DVT adapter — as federation source, federation target, seed target, and default target:

PostgreSQL · MySQL · MariaDB · SQL Server · Oracle · SQLite · DuckDB · Snowflake · BigQuery · Redshift · Databricks · ClickHouse · Trino · Athena · Microsoft Fabric — plus S3, GCS, and Azure Blob as bucket storages.

Anything else (MongoDB, Elasticsearch, REST APIs, …) is reachable through locally-executed Python models. Details per adapter: getdvt.net/docs/adapters.

Learn more

License

DVT is distributed as compiled wheels on PyPI. Free tier covers local files, transactional databases, and Python models; Pro adds cloud buckets (S3/GCS/Azure); Max adds cloud warehouses. See getdvt.net/pricing.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

dvt_core-0.2.34-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (21.1 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.34-cp314-cp314-macosx_11_0_arm64.whl (3.5 MB view details)

Uploaded CPython 3.14macOS 11.0+ ARM64

dvt_core-0.2.34-cp314-cp314-macosx_10_15_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.14macOS 10.15+ x86-64

dvt_core-0.2.34-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (21.4 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.34-cp313-cp313-macosx_11_0_arm64.whl (3.5 MB view details)

Uploaded CPython 3.13macOS 11.0+ ARM64

dvt_core-0.2.34-cp313-cp313-macosx_10_13_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.13macOS 10.13+ x86-64

dvt_core-0.2.34-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (21.5 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.34-cp312-cp312-macosx_11_0_arm64.whl (3.5 MB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

dvt_core-0.2.34-cp312-cp312-macosx_10_13_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.12macOS 10.13+ x86-64

dvt_core-0.2.34-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (22.3 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.34-cp311-cp311-macosx_11_0_arm64.whl (3.5 MB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

dvt_core-0.2.34-cp311-cp311-macosx_10_9_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.11macOS 10.9+ x86-64

dvt_core-0.2.34-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (21.0 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.34-cp310-cp310-macosx_11_0_arm64.whl (3.5 MB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

dvt_core-0.2.34-cp310-cp310-macosx_10_9_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.10macOS 10.9+ x86-64

File details

Details for the file dvt_core-0.2.34-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.34-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 e40a3a3b476086fcc037c3dd979510b0e78d21d5001545f1cc79a788aa656edf
MD5 8ed061f6d0c37ec88ecc0e729b2cb32f
BLAKE2b-256 4274aa44a0b07d6913c5e05e9b3744df6cdf651aa9316f4c0581d815494b1d90

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.34-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.34-cp314-cp314-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.34-cp314-cp314-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 cb88780c9d353cec7fa485d061e574f1e87cec9f7ad6fdb7daa2910f32319b0a
MD5 c4f8bec4c0fd8f446f38f296355edcf5
BLAKE2b-256 7f9a90dea296c6dcfd09fe60406736f9722636d5be2fac1b79c37ef8990fb346

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.34-cp314-cp314-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.34-cp314-cp314-macosx_10_15_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.34-cp314-cp314-macosx_10_15_x86_64.whl
Algorithm Hash digest
SHA256 0fa0e9fc795dc7757e120ef8e2a006d4165eae28c28b26624ed75cc57fde26ea
MD5 9c0814aaa3645047f377835d83468bac
BLAKE2b-256 e93ff28a1f5b6c7f6d4675ec2fec463c826e3c27897d62b46f5671baab27ae33

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.34-cp314-cp314-macosx_10_15_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.34-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.34-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 12828dffcee24dc11fb8393e0b632f1637503a38a94cf76c8edad7c12637670a
MD5 836189eafd7da2e1b1e5fd2f63929b8c
BLAKE2b-256 5939172c92b274fa34f671648639bbb8fbde54e08c5de080c37aa254fcd59510

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.34-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.34-cp313-cp313-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.34-cp313-cp313-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 07c4e1e4047cf58997e3126baee7e8c06cf505c7e02911c3e591637f723c82e0
MD5 8e7da21053b79d6203f99e4767c840a8
BLAKE2b-256 e150f7aef58b2fe1de244a0fc76ca8fde2aa22863b4e43fbc34b3053c515ae7a

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.34-cp313-cp313-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.34-cp313-cp313-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.34-cp313-cp313-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 a99576dab4f04e6e0bcbf92863ea0984f638f67194f8dbe7de57dfa5b05418f0
MD5 18fcdc1e0051cdf20bf4fe014088eb7d
BLAKE2b-256 7c1c8fdc601aae5072575da51b17088b3f54311e4047644e93adbd7091a0b9f3

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.34-cp313-cp313-macosx_10_13_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.34-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.34-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 4ac1241fb529701a92c883121c995b3729ac8242ad78d3a3155fe3e0f758b162
MD5 8ae121a3a0e922733bc1aa143d09c277
BLAKE2b-256 4cc06923af7628df0608c9bff22aebc609a9c89e663b781ba34a9f4ee86abc9f

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.34-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.34-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.34-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 cfe1b72711273605736ea0c067406706847b02310038527055e715eda704a87e
MD5 b93695678c73383fc3dc0a6d678546b6
BLAKE2b-256 e189c974b2de8fff39dbc523f388159befb6b3004193c6aa94dfcd52f317986a

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.34-cp312-cp312-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.34-cp312-cp312-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.34-cp312-cp312-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 3c3a6e9faea533985241df0490f7f74c4c23dd5279d6f9dba95e498eee896d24
MD5 4249f3221c2c41d42a463a6e93a5a02d
BLAKE2b-256 bfbf9c6091b21f89a6b5499e6ad80b5074558d9bea21b1a55f2a35dfcc97ed21

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.34-cp312-cp312-macosx_10_13_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.34-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.34-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 af8eb537b4dd0c139eaeb961f8f253d77a4dd84fae928e73a59f6868556c0b15
MD5 1240c1bdeb72b5f24eabca74b8649667
BLAKE2b-256 98f7f952d49844a416aca90628054eaea7895f4762b4d9dc832d0a2c9f3d6301

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.34-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.34-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.34-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 c74909c30e66fec0fc89c2325b6422d7f7efe487ff3fc2d24f9bc357d6ee5578
MD5 fc1a8fcd4b016502b9c725ae6363f09f
BLAKE2b-256 239f493ccc4379b0e2be9b98f87269632325891a315e3d295f65246584a3d901

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.34-cp311-cp311-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.34-cp311-cp311-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.34-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 6d7a6670c048a2804d0578220ec7991902218461d496d38067834bc418c0d94b
MD5 9cb6f00d0ddcbdcf24f71cf1f839eea3
BLAKE2b-256 637ba57ca8d7581fca825088d2d50d5a0529034e51e3d883ccb251471fdc373b

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.34-cp311-cp311-macosx_10_9_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.34-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.34-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 52bb886a01685901fc184b23d533e271ae8c0176c3ff2b92de5d2c94a25557ee
MD5 91c5b72c48adbab72a623012bad90bd3
BLAKE2b-256 b6ee62e554b7d0d987e9a0fc0673583efd58d9a41cdf68004d32dbfa3ad3f3ae

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.34-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.34-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.34-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 ad21bb3e0703b3e4a9907ab44bbe2f943a0b89244f1ba0a56dea3966ac4d2245
MD5 389ed67db4377ea16307ca9f5638d05b
BLAKE2b-256 f970a381e9e120236f1dc345ba473ad7b639a3f3c03dca04a199b9ce45a5f890

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.34-cp310-cp310-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.34-cp310-cp310-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.34-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 c9f22f90a88abb247b0d4c9ce7ff873c4c3df23898a624fa3d4e44d0d8a8ea9d
MD5 9f3d94420906c366bbe396ed2522cb00
BLAKE2b-256 769551e7ffe2578ada5f85a7588f658ae04a960e0be796382b2ac2c50eeb1c9e

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.34-cp310-cp310-macosx_10_9_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.2.52

15 files

0.2.51

15 files

0.2.50

15 files

0.2.40

15 files

0.2.39

15 files

0.2.38

15 files

0.2.37

15 files

0.2.36

15 files

0.2.35

15 files

This release

0.2.34 This release

15 files

0.2.33

15 files

0.2.32

15 files

0.2.31

15 files

0.2.30

15 files

0.2.29

15 files

0.2.28

15 files

0.2.27

15 files

0.2.26

15 files

0.2.25

15 files

0.2.24

15 files

0.2.23

15 files

0.2.22

15 files

0.2.21

15 files

0.2.20

15 files

0.2.19

15 files

0.2.18

15 files

0.2.17

15 files

0.2.16

15 files

0.2.15

15 files

0.2.14

15 files

0.2.13

15 files

0.2.12

15 files

0.2.11

15 files

0.2.10

15 files

0.2.9

19 files

0.2.8

9 files

0.2.7

9 files

0.2.5

9 files

0.2.4

9 files

0.2.3

9 files

0.2.2

9 files

0.2.1

9 files

0.2.0

9 files

0.1.54

10 files

0.1.53

10 files

0.1.52

10 files

0.1.51

10 files

0.1.50

10 files

0.0.1

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page