Skip to main content

DVT logo

DVT — Data Virtualization Tool

Any source. Any target. One SQL project.

DVT is a cross-engine data transformation CLI built on dbt. Write SQL models that JOIN tables living on different database engines — MySQL with Snowflake, Oracle with PostgreSQL, anything with anything — and DVT handles extraction, federation, and loading automatically.

{{ config(materialized='f_table') }}   -- "f" = federated

select
    c.customer_name,
    o.order_date,
    o.total_amount
from {{ source('crm', 'customers') }} c      -- lives on MySQL
join {{ source('erp', 'orders') }} o          -- lives on Snowflake
  on c.customer_id = o.customer_id
where o.order_date >= '2024-01-01'
pip install dvt-core
dvt sync       # installs every adapter + driver your profiles.yml needs
dvt run        # dbt runs what dbt can; DVT federates the rest
dvt version    # (worth running once — you'll see)

Website & full docs: getdvt.net · Quickstart · PyPI


Architecture

DVT is a wrapper around stock dbt-core, not a fork. dbt compiles the project and runs every standard model; DVT picks up the models dbt can't express — cross-engine federation models and locally-executed Python models — and runs them through its own pipeline:

flowchart TB
    CLI([dvt run / build]) --> COMPILE["dbt compile → manifest.json"]
    COMPILE --> STD["standard models<br/>table · view · incremental"]
    COMPILE --> FED["federated models<br/>f_table · f_incremental · .py"]
    STD --> DBT["stock dbt executes them<br/>on the default target"]
    FED --> HOMO{"all sources on<br/>ONE connection?"}
    HOMO -- "yes — Sling direct" --> DIRECT["whole query transpiled to the
    source engine, streamed straight
    to the target — no staging"]
    HOMO -- "no — federation" --> DECOMP["1 · DECOMPOSE — SQLGlot maps every
    table to a profiles.yml connection"]
    DECOMP --> PUSH["2 · TRANSPILE + PUSH DOWN — minimal
    per-source queries, each engine's dialect"]
    PUSH --> EXTRACT["3 · EXTRACT — Sling → Parquet, parallel"]
    EXTRACT --> COMPUTE["4 · COMPUTE — DuckDB joins locally"]
    COMPUTE --> LOAD["5 · LOAD — Sling delivers to any
    database, warehouse, or bucket"]
    DIRECT --> TARGET[("target")]
    LOAD --> TARGET
    DBT --> TARGET

The Sling direct path (0.2.5)

When every source in an f_table model lives on one connection, steps 3–5 collapse into a single hop: the whole query — joins and aggregations included — is transpiled to that engine's dialect and submitted to Sling as a custom-SQL stream loaded straight to the target. No Parquet staging, no DuckDB. The check is automatic and per-run; any failure falls back to the standard pipeline. Add a source on another engine tomorrow and the same model silently switches to the federation pipeline — the model never changes.

Execution paths

Your model Who runs it How
table / view / incremental stock dbt natively on the default target — whole-query pushdown by definition
f_table / f_incremental, sources on one connection DVT Sling direct: full query pushed to the source engine, result streamed to the target
f_table / f_incremental, sources span connections DVT decompose → per-source pushdown extraction → DuckDB compute → load
.py models DVT locally executed; result loads to any target

Standard models that set a non-default target= are refused with guidance (dbt silently ignores the kwarg — your model would land on the wrong engine).

Design principles

  • Wrapper, not fork. DVT shells out to the dbt CLI and reads manifest.json — it never imports dbt internals. Your dbt version can move; DVT keeps working. Every dbt feature (Jinja, macros, packages, tests, snapshots, docs) works because it is dbt running it.
  • Dual-dialect by design. Federation models are written in DuckDB dialect and are transpiled per source — they never depend on any engine's syntax. Standard models compile in the default target's dialect via the official dbt adapter. Result: switching your default target (e.g. PostgreSQL → Databricks) is a profile change, not a rewrite.
  • Pushdown first. The fastest row to move is the one you never extract. Filters and column pruning push down to source engines; when every table in a model lives on one engine, the entire query pushes down and nothing moves at all.
  • Your hardware does the joining. Cross-engine compute happens in DuckDB on the machine running DVT — source databases only ever serve simple filtered SELECTs, and no warehouse bills you for the join.

What DVT adds on top of dbt

Capability How
Cross-engine models materialized='f_table' / 'f_incremental' (long aliases: federated_*, federation_*)
Incremental federation watermark-based delta extraction; persistent local cache between runs
Per-model targets config(target='other_output') on federated models — results land on any profile output (standard models are refused a foreign target)
Bucket targets materialize models to S3 / GCS / Azure as Parquet or CSV
Python models, locally dbt-style .py models run on your machine (APIs, pandas, ML) — results land on any target
Bulk seeds dvt seed loads csv / tsv / parquet / json / jsonl / avro / arrow / xlsx via Sling to any engine, with enforced snake_case columns
One-command setup dvt sync installs every adapter and driver your profiles.yml requires
Connection diagnostics dvt debug --all checks every connection on both layers (dbt adapter + Sling)
Cross-engine catalog dvt docs builds one catalog + lineage UI across all engines in your project

Everything else — profiles.yml in ~/.dbt/, sources.yml, ref/source/config, tests, snapshots — is standard dbt. Existing dbt projects work unchanged; sources gain an optional meta.connection to bind them to other engines.

Supported engines

Every engine supported by both dbt and Sling is a DVT adapter — as federation source, federation target, seed target, and default target:

PostgreSQL · MySQL · MariaDB · SQL Server · Oracle · SQLite · DuckDB · Snowflake · BigQuery · Redshift · Databricks · ClickHouse · Trino · Athena · Microsoft Fabric — plus S3, GCS, and Azure Blob as bucket storages.

Anything else (MongoDB, Elasticsearch, REST APIs, …) is reachable through locally-executed Python models. Details per adapter: getdvt.net/docs/adapters.

Learn more

License

DVT is distributed as compiled wheels on PyPI. Free tier covers local files, transactional databases, and Python models; Pro adds cloud buckets (S3/GCS/Azure); Max adds cloud warehouses. See getdvt.net/pricing.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

dvt_core-0.2.33-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (20.4 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.33-cp314-cp314-macosx_11_0_arm64.whl (3.4 MB view details)

Uploaded CPython 3.14macOS 11.0+ ARM64

dvt_core-0.2.33-cp314-cp314-macosx_10_15_x86_64.whl (3.4 MB view details)

Uploaded CPython 3.14macOS 10.15+ x86-64

dvt_core-0.2.33-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (20.6 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.33-cp313-cp313-macosx_11_0_arm64.whl (3.4 MB view details)

Uploaded CPython 3.13macOS 11.0+ ARM64

dvt_core-0.2.33-cp313-cp313-macosx_10_13_x86_64.whl (3.4 MB view details)

Uploaded CPython 3.13macOS 10.13+ x86-64

dvt_core-0.2.33-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (20.8 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.33-cp312-cp312-macosx_11_0_arm64.whl (3.4 MB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

dvt_core-0.2.33-cp312-cp312-macosx_10_13_x86_64.whl (3.5 MB view details)

Uploaded CPython 3.12macOS 10.13+ x86-64

dvt_core-0.2.33-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (21.5 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.33-cp311-cp311-macosx_11_0_arm64.whl (3.4 MB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

dvt_core-0.2.33-cp311-cp311-macosx_10_9_x86_64.whl (3.5 MB view details)

Uploaded CPython 3.11macOS 10.9+ x86-64

dvt_core-0.2.33-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (20.3 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

dvt_core-0.2.33-cp310-cp310-macosx_11_0_arm64.whl (3.4 MB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

dvt_core-0.2.33-cp310-cp310-macosx_10_9_x86_64.whl (3.5 MB view details)

Uploaded CPython 3.10macOS 10.9+ x86-64

File details

Details for the file dvt_core-0.2.33-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.33-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 9a506b3c83718ee8e4df7e351816948d0777dd51527211fcaff4de83d04cbcb8
MD5 b8c9653386f305094b29b495b56c2a94
BLAKE2b-256 b9aa9514d03bf2ca3eac5893ca056630bd634d39980a890aa87064ab47d8e064

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.33-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.33-cp314-cp314-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.33-cp314-cp314-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 c64dabe890a8feceef50f1af4bc8f91ab7fd84df2b2277be720a0e859013f60e
MD5 f7142e091fab0b793aa9b9c465e915da
BLAKE2b-256 d141e1d631a437839f41c431356dbea028fd0ed4bafeb76536a68cf9ee14fe43

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.33-cp314-cp314-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.33-cp314-cp314-macosx_10_15_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.33-cp314-cp314-macosx_10_15_x86_64.whl
Algorithm Hash digest
SHA256 5282884b54ae2c3a6a775ff5dc055e2e65819818ff34667c4cdbe10ce3298caa
MD5 49f7043b1f934ec07b537b9ffdb57459
BLAKE2b-256 276abc6343efcdab5bf3b987e5b62934d27238465c06f8175670e6c2a9a478a1

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.33-cp314-cp314-macosx_10_15_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.33-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.33-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 97de10d584087edb453fbb194a8f00b74e366b3a5fa1493f346606dc6c408b61
MD5 c831219f2a4dd8d10bd5e7b8b6b464ed
BLAKE2b-256 2b69088ead7c85b31b13e4bbbb8f347564c60c334d9cbc07058ed4c352148f7f

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.33-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.33-cp313-cp313-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.33-cp313-cp313-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 1b4af068a0e32d75b836b914bc56843243c95bf62589dcc277700b6328d375ab
MD5 67b541abefcee475b69d76bac5b101e4
BLAKE2b-256 8d882893926cbcd61dbe275b08d7f0e311e9e249437d6c953a2dc1cd7abb2bce

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.33-cp313-cp313-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.33-cp313-cp313-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.33-cp313-cp313-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 7203b0cf3ffd8cf2fcebc797327bba5cec8d0ac0e9644c461166167e66604476
MD5 1aff5e01b7e8dec6fc46bb3108952133
BLAKE2b-256 1a2b60e3e491c3f2555217c63c1ede730aed2c833983d29f3f60b53d9af8a90e

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.33-cp313-cp313-macosx_10_13_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.33-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.33-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 d8b7f9caa76318da557a9550d97a233c93ff40a29877d86e75f78ef7ae7b8939
MD5 21e95ca5d55d430cbd1363233337223a
BLAKE2b-256 e8d319c5a10fbc97ae23f75f4bf5b5d3dbd19800311aad68f196ed499de1b2b7

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.33-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.33-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.33-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 5664df7815ec1ee1f9ca2a6bbbf5fa397651135835af25c7329144c64bed3c84
MD5 8859f5942bff82e7280ebc13b1d83f3e
BLAKE2b-256 6325ff73c1fad77145067eec858eb727f6028be39a26b9c5968653eff3bec801

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.33-cp312-cp312-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.33-cp312-cp312-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.33-cp312-cp312-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 2e856cd9f5288c49b748ff35a28f3f5663887a57a28f8724f9cc41f8a35070bb
MD5 73075476473271c03cabb01638ac709d
BLAKE2b-256 753df1dd4260da98c99993c2e6abb487714bad415ba3e481763b8be36c0428e2

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.33-cp312-cp312-macosx_10_13_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.33-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.33-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 bf5e7591de14126868cf92dd7d9b32b9a351025e2281808b179d0f9fc44dab1f
MD5 0940b10b94510abbe89bc45889d02bff
BLAKE2b-256 ace918656b58bd8d3d6a3279fac56f7259bd0fa397f067bd671ed39fb46f275a

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.33-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.33-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.33-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 85a19c1fb475c29052a8ce22ad1be1bf30c5da61982774267a8536439a694d96
MD5 db148e5d4dc5f30d34c313f2fd514481
BLAKE2b-256 0dcdaa70eea5c8dda458c5d6f232dc23465327f49323d974bb7ef85b691e673c

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.33-cp311-cp311-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.33-cp311-cp311-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.33-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 89da7b8f7a0b63c160849faa9750bd110f34ebf0afa4245add7dde5e3e257be1
MD5 5cd13ccc40676be58ac20eae6c19b47a
BLAKE2b-256 361f7b89905429fd862ffd4fd1cb238b6a6360fc3a125cc74bff79f777c67790

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.33-cp311-cp311-macosx_10_9_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.33-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.33-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 4014e1313118947faa80b0c410021bdfa624d1cbd2a6bcc28a5a2b4f6fb6de14
MD5 367f110ba8dd5f2a510bf616e32b03e0
BLAKE2b-256 d0f820bdc75f1c5b51d4564ff8689bf522aaf8fffcb854a2460da38ec7a57a0b

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.33-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.33-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.33-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 2351046a0c77a25fbbf52f24efc5148d328491ad22b1b1b872274e2b5df66a19
MD5 4f15664e5708878ec0c8750310e70325
BLAKE2b-256 381354c075684c860ea7c3c64c64004901a2e8b06d174e42624ce88deca77e32

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.33-cp310-cp310-macosx_11_0_arm64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dvt_core-0.2.33-cp310-cp310-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for dvt_core-0.2.33-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 bfac5a35e6909d5674b18e086aa08c071011d7b32adc9440216f1d40f121dc7c
MD5 32e84f65e3fa0a28956dc29e561c9918
BLAKE2b-256 f897c0a522f0beb549dfae3795d1fd8ca3b9f01681972155d50f7b16c39a2d81

See more details on using hashes here.

Provenance

The following attestation bundles were made for dvt_core-0.2.33-cp310-cp310-macosx_10_9_x86_64.whl:

Publisher: build-wheels.yml on heshamh96/dvt-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.2.52

15 files

0.2.51

15 files

0.2.50

15 files

0.2.40

15 files

0.2.39

15 files

0.2.38

15 files

0.2.37

15 files

0.2.36

15 files

0.2.35

15 files

0.2.34

15 files

This release

0.2.33 This release

15 files

0.2.32

15 files

0.2.31

15 files

0.2.30

15 files

0.2.29

15 files

0.2.28

15 files

0.2.27

15 files

0.2.26

15 files

0.2.25

15 files

0.2.24

15 files

0.2.23

15 files

0.2.22

15 files

0.2.21

15 files

0.2.20

15 files

0.2.19

15 files

0.2.18

15 files

0.2.17

15 files

0.2.16

15 files

0.2.15

15 files

0.2.14

15 files

0.2.13

15 files

0.2.12

15 files

0.2.11

15 files

0.2.10

15 files

0.2.9

19 files

0.2.8

9 files

0.2.7

9 files

0.2.5

9 files

0.2.4

9 files

0.2.3

9 files

0.2.2

9 files

0.2.1

9 files

0.2.0

9 files

0.1.54

10 files

0.1.53

10 files

0.1.52

10 files

0.1.51

10 files

0.1.50

10 files

0.0.1

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page