Skip to main content
drt logo

drt — data reverse tool

The reverse leg of your data stack.

dlt loads data in, dbt transforms it, and drt activates it back out — reverse ETL from your warehouse to the tools your team works in. Declarative YAML, one drt run.

dlt load  →  dbt transform  →  drt activate

CI Coverage PyPI Python License

drt-core downloads dagster-drt downloads Sponsor Website X

All Contributors

drt quickstart demo

pip install drt-core          # core (DuckDB included)
drt init && drt run

Open in GitHub Codespaces — or try it in your browser, zero local setup.


Why drt?

Same DX as dbt. If you know dbt run, you already know drt run — declarative YAML, versioned in Git, reviewed in PRs.

CI-native. Exit codes and --output json drop straight into GitHub Actions, cron, or Dagster / Airflow / Prefect — no GUI, no clickops.

LLM-native. A built-in MCP server and Claude Code skills let AI tools author and run your syncs.

Free and open source. Every connector, the CLI, the MCP server, and the sync engine — Apache 2.0, self-hosted, no lock-in.

What's always free? All connectors, CLI, MCP server, and sync engine. See OPEN_CORE.md for the open core boundary.


Quickstart

No cloud accounts needed — DuckDB + httpbin.org, three commands.

pip install drt-core
mkdir my-drt-project && cd my-drt-project
drt init --template duckdb_to_rest

That scaffolds a runnable syncs/duckdb_to_rest.yml. Seed a tiny DuckDB table and run:

python -c "
import duckdb
c = duckdb.connect('warehouse.duckdb')
c.execute('''CREATE TABLE IF NOT EXISTS users AS SELECT * FROM (VALUES
  (1, 'Alice', 'alice@example.com'),
  (2, 'Bob',   'bob@example.com'),
  (3, 'Carol', 'carol@example.com')
) t(id, name, email)''')
c.close()
"
drt run --dry-run   # preview, no data sent
drt run             # POST each row to httpbin.org
drt status          # check results

Other starter templates

drt init --template list             # see all available templates
drt init --template postgres_to_slack
drt init --template duckdb_to_hubspot

Each template prints next-steps for the env vars / source data it needs. See examples/ for the full collection (Discord, Google Sheets, GitHub Actions, MySQL, ClickHouse, BigQuery, …) and docs/connectors/ for per-connector reference.

Customizing your sync

For a guided wizard that walks you through profile + project setup:

drt init   # interactive — picks a source, configures profile, scaffolds project

Both flows produce the same project shape (drt_project.yml, syncs/, .drt/). drt sources --detailed and drt destinations --detailed print every connector's required env vars and a sample YAML stanza — useful when hand-authoring beyond the templates.


CLI Reference

drt init                    # initialize project
drt list                    # list sync definitions
drt sources                 # list available source connectors
drt destinations            # list available destination connectors
drt run                     # run all syncs
drt run --select <name>     # run a specific sync (globs work: 'users_*')
drt run --select tag:<tag>  # run syncs matching a tag (repeat --select to union)
drt run --select destination:<type>  # run syncs by destination type
drt run --exclude <name>    # subtract syncs (same grammar as --select)
drt run --failed            # re-run only syncs that failed last time
drt run --limit 10          # sampled run: send only N rows (watermark frozen)
drt run --fail-fast         # stop scheduling after the first failure
drt run --vars 'lookback_days: 1'   # override project vars: for this run
drt run --threads 4         # parallel sync execution
drt run --dry-run           # dry run
drt run --verbose           # show row-level error details
drt run --output json       # structured JSON output for CI/scripting
drt run --log-format json   # structured JSON logging to stderr
drt run --profile prd       # override profile (or DRT_PROFILE env var)
drt run --cursor-value '…'  # override watermark cursor for backfill
drt test                    # run post-sync validation tests
drt test --select <name>    # test a specific sync
drt build                   # run each sync and its tests in one pass
drt validate                # validate sync YAML configs
drt status                  # show recent sync status
drt status --output json    # JSON output for status
drt profile list            # list credential profiles in ~/.drt/profiles.yml
drt profile show <name>     # show a profile (secrets masked)
drt profile test <name>     # verify a profile's source connectivity
drt profile add <name>      # interactively add a profile
drt profile remove <name>   # remove a profile
drt serve                   # start HTTP webhook endpoint
drt docs generate --format mermaid  # print project DAG as Mermaid
drt deploy github-actions   # scaffold a scheduled sync workflow (drt-action + secrets wired)
drt mcp run                 # start MCP server (requires drt-core[mcp])
drt --install-completion    # install shell completion (bash/zsh/fish)
drt --show-completion       # show completion script

Visualize your syncs

Generate a Mermaid DAG from your local drt_project.yml and syncs/*.yml files:

drt docs generate --format mermaid > dag.md
graph LR
    subgraph Sources
        src_bigquery_prod["bigquery_prod<br/><i>bigquery</i>"]
    end
    subgraph Syncs
        sync_users_to_hubspot{{"users_to_hubspot<br/><i>upsert</i>"}}
        sync_accounts_to_hubspot{{"accounts_to_hubspot<br/><i>upsert</i>"}}
    end
    subgraph Destinations
        dst_hubspot_contacts["hubspot (contacts)<br/><i>hubspot</i>"]
    end
    src_bigquery_prod -->|extract| sync_users_to_hubspot
    src_bigquery_prod -->|extract| sync_accounts_to_hubspot
    sync_users_to_hubspot -->|load| dst_hubspot_contacts
    sync_accounts_to_hubspot -->|load| dst_hubspot_contacts
    sync_users_to_hubspot -.lookup.-> sync_accounts_to_hubspot

Shell completion

Shell completion is supported for bash, zsh, and fish:

# Recommended: auto-install for your current shell (idempotent)
drt --install-completion

# Or manually add to your shell config (run once from the target shell)
drt --show-completion >> ~/.bashrc   # bash
drt --show-completion >> ~/.zshrc    # zsh
drt --show-completion > ~/.config/fish/completions/drt.fish  # fish

Note: --show-completion outputs the script for your current shell. Run it from the shell you want to configure. The manual >> append is not idempotent — run it once only.

After installation, restart your shell and tab-complete commands and options.


MCP Server

Connect drt to Claude, Cursor, or any MCP-compatible client so you can run syncs, check status, and validate configs without leaving your AI environment.

pip install drt-core[mcp]
drt mcp run

Claude Desktop (~/Library/Application Support/Claude/claude_desktop_config.json):

{
  "mcpServers": {
    "drt": {
      "command": "drt",
      "args": ["mcp", "run"]
    }
  }
}

Available MCP tools:

Tool What it does
drt_list_syncs List all sync definitions
drt_run_sync Run a sync (supports dry_run + compute_diff for --diff parity)
drt_run_test Run post-sync validation tests (mirrors drt test)
drt_get_status Get last run result(s)
drt_get_history Get recent sync run history
drt_validate Validate sync YAML configs
drt_get_schema Return JSON Schema for config files
drt_list_connectors List available sources and destinations
drt_dlq Inspect a sync's Dead Letter Queue (depth + failed records)
drt_retry Replay a sync's Dead Letter Queue (mirrors drt retry)
drt_get_manifest Machine-readable sync catalog + lineage (drt docs JSON)
drt_list_profiles List credential profiles (name + type, no secrets)
drt_test_profile Connectivity check for a credential profile
drt_doctor Environment diagnostics (mirrors drt doctor)

AI Skills for Claude Code

Install the official Claude Code skills to generate YAML, debug failures, and migrate from other tools — all from the chat interface.

Install via Plugin Marketplace (recommended)

/plugin marketplace add drt-hub/drt
/plugin install drt@drt-hub

Tip: Enable auto-update so you always get the latest skills when drt is updated: /plugin → Marketplaces → drt-hub → Enable auto-update

Manual install (slash commands)

Copy the files from .claude/commands/ into your drt project's .claude/commands/ directory.

Skill Trigger What it does
/drt-create-sync "create a sync" Generates valid sync YAML from your intent
/drt-debug "sync failed" Diagnoses a specific error and suggests fixes
/drt-troubleshoot "drt isn't working" Walks a full top-to-bottom diagnostic checklist
/drt-init "set up drt" Guides through project initialization
/drt-migrate "migrate from Census" Converts existing configs to drt YAML

Connectors

Per-connector reference: docs/connectors/ · Discoverable from the CLI: drt sources --detailed / drt destinations --detailed

Sources

Connector Status Install Auth
BigQuery ✅ v0.1 pip install drt-core[bigquery] Application Default / Service Account Keyfile
DuckDB ✅ v0.1 (core) File path
PostgreSQL ✅ v0.1 pip install drt-core[postgres] Password (env var)
Snowflake ✅ v0.5 pip install drt-core[snowflake] Password (env var)
SQLite ✅ v0.4.2 (core) File path
Redshift ✅ v0.3.4 pip install drt-core[redshift] Password (env var)
ClickHouse ✅ v0.4.3 pip install drt-core[clickhouse] Password (env var)
MySQL ✅ v0.5 pip install drt-core[mysql] Password (env var)
Databricks ✅ v0.6 pip install drt-core[databricks] Access Token (env var)
Delta Lake ✅ v0.8 pip install drt-core[deltalake] Storage options (env var)
Iceberg ✅ v0.8 pip install drt-core[iceberg] Catalog properties (env var)
SQL Server ✅ v0.6 pip install drt-core[sqlserver] Password (env var)
REST API ✅ v0.7 (core) Bearer / API Key / Basic / OAuth2

Destinations

34 destinations — warehouses, SaaS APIs, cloud storage, files & webhooks (click to expand)
Connector Status Install Auth
REST API ✅ v0.1 (core) Bearer / API Key / Basic / OAuth2
Slack Incoming Webhook ✅ v0.1 (core) Webhook URL
Discord Webhook ✅ v0.4.2 (core) Webhook URL
GitHub Actions ✅ v0.1 (core) Token (env var)
HubSpot ✅ v0.1 (core) Token (env var)
Zendesk ✅ v0.7 (core) Basic (email + API token)
Amplitude ✅ v0.7 (core) Project API key (env var)
Klaviyo ✅ v0.8 (core) Private API key (env var)
Mixpanel ✅ v0.8 (core) Project token / service account
Google Ads ✅ v0.6 (core) OAuth2 Client Credentials
Google Sheets ✅ v0.4 pip install drt-core[sheets] Service Account Keyfile
PostgreSQL (upsert) ✅ v0.4 pip install drt-core[postgres] Password (env var)
MySQL (upsert) ✅ v0.4 pip install drt-core[mysql] Password (env var)
ClickHouse ✅ v0.5 pip install drt-core[clickhouse] Password (env var)
Parquet file ✅ v0.5 pip install drt-core[parquet] File path
Amazon S3 ✅ v0.7.9 pip install drt-core[s3] AWS credential chain / env vars
Google Cloud Storage ✅ v0.7.9 pip install drt-core[gcs] Application Default / Service Account Keyfile
Azure Blob Storage ✅ v0.7.9 pip install drt-core[azure] Connection string env / DefaultAzureCredential
Microsoft Teams Webhook ✅ v0.5 (core) Webhook URL
CSV / JSON / JSONL file ✅ v0.5 (core) File path
Jira ✅ v0.5 (core) Basic (email + API token)
Linear ✅ v0.5 (core) API Key (env var)
SendGrid ✅ v0.5 (core) API Key (env var)
Notion ✅ v0.6 (core) Bearer Token (env var)
Airtable ✅ v0.8 (core) Personal Access Token (env var)
Twilio SMS ✅ v0.6 (core) Basic (Account SID + Auth Token)
Intercom ✅ v0.6 (core) Bearer Token (env var)
Email SMTP ✅ v0.6 (core) Username / Password (env var)
Salesforce Bulk API 2.0 ✅ v0.6 (core) OAuth2 (username-password)
Staged Upload ✅ v0.6 (core) Configurable per provider
Elasticsearch / OpenSearch ✅ v0.7.9 (core) API key / HTTP Basic (env var)
Snowflake ✅ v0.7 pip install drt-core[snowflake] Password (env var)
Databricks Delta Lake ✅ v0.7.9 pip install drt-core[databricks] Personal Access Token (env var)
BigQuery ✅ v0.8 pip install drt-core[bigquery] ADC / Service Account keyfile

Integrations

Connector Status Install
Dagster ✅ v0.4 pip install dagster-drt
Prefect ✅ v0.6 (core)
Airflow ✅ v0.6 (core)
dbt manifest reader ✅ v0.4 (core)

Roadmap

Upcoming releases → ROADMAP.md (scope, themes, targets) Issue-level tracking → GitHub Milestones Looking to contribute? → Good First Issues

Shipped: now on v0.8.0 — the drt docs lineage site (safe-by-default labels · static SVG DAG) · project vars: · drt build · --fail-fast / --limit / --failed · dbt-style selection v2 · incremental REST API. Grouped milestones below; the full per-release changelog lives in CHANGELOG.md and GitHub Releases.

Milestone Highlights
v0.1–0.3 Core CLI · BigQuery / DuckDB / Postgres sources · REST / Slack / HubSpot destinations · incremental sync · MCP server + AI Skills for Claude Code
v0.4–0.6 20+ destinations (Sheets, SQL upserts, Notion, Twilio, Salesforce…) · dagster-drt / Airflow / Prefect · drt test · parallel --threads · JSON Schema validation
v0.7 Production Ready — graceful shutdown · sync history · atomic table swaps · drt doctor · opt-in telemetry · sync.mode: mirror · DLQ + drt profile
v0.7.9–0.7.11 Cloud destinations (S3 / GCS / Azure / Databricks / BigQuery / Elasticsearch) · Delta Lake + Iceberg sources · tracked & scoped mirror · PII masking · VS Code extension
v0.8.0 drt docs lineage site (safe-by-default labels, static SVG DAG, byte-identical) · project vars: · drt build · --fail-fast / --limit / --failed · selection v2 · incremental REST API · watermark.lag · drt deploy github-actions

Next: v0.8.1 Diff Polishv0.9 Enterprise Foundationv1.0 Stable Releasev1.x Rust Engine


Orchestration: dagster-drt

Community-maintained Dagster integration. Expose drt syncs as Dagster assets with full observability.

pip install dagster-drt
from dagster import AssetExecutionContext, Definitions
from dagster_drt import drt_assets, DagsterDrtResource

@drt_assets(project_dir="path/to/drt-project")
def my_syncs(context: AssetExecutionContext, drt: DagsterDrtResource):
    yield from drt.run(context=context)

defs = Definitions(
    assets=[my_syncs],
    resources={"drt": DagsterDrtResource(project_dir="path/to/drt-project")},
)

See dagster-drt README for full API docs (Translator, Pipes support, DrtConfig dry-run, MaterializeResult).


CI/CD: GitHub Action

Run drt syncs straight from CI/CD with the official drt-hub/drt-action — no infrastructure, just a few lines of YAML. Trigger on a schedule, on every push, or right after dbt finishes.

- uses: drt-hub/drt-action@v1
  with:
    select: '*'
    extras: postgres
  env:
    PG_PASSWORD: ${{ secrets.PG_PASSWORD }}

Inputs cover sync selection (select), connector extras, profile, dry-run and threads; outputs expose status, succeeded, failed and duration-seconds (plus a step-summary table). Secrets are passed via env: and resolved by drt's *_env keys. See the action README for the secrets pattern and more examples (run-after-dbt, PR preview).


Ecosystem

drt is designed to work alongside, not against, the modern data stack:

drt ecosystem — dlt load, dbt transform, drt activate


Telemetry

drt collects no telemetry by default. Opting in helps us understand which sources / destinations / sync modes are actually used, so we can prioritise.

drt config set telemetry.enabled true     # opt in
drt config show-telemetry                 # preview the exact payload that would be sent
drt config set telemetry.enabled false    # opt out
DO_NOT_TRACK=1 drt run                    # universal kill switch — overrides everything

When opted in, drt sends one sync_completed event per sync. The only properties we collect are these 9 fields: drt_version, python_version, os, source_type, destination_type, sync_mode, rows_synced, duration_seconds, status. The wire envelope additionally carries event, distinct_id (a per-machine random UUID at ~/.drt/.anonymous_id), timestamp, and api_key. Sync names, model SQL, destination URLs, credentials, and project paths are never transmitted — the allow-list is enforced at the function-signature level in drt/telemetry.py. By default events go to PostHog Cloud (EU region); override with DRT_TELEMETRY_ENDPOINT and DRT_TELEMETRY_API_KEY for self-hosted PostHog or a custom collector.

Note: drt itself never transmits your IP, but the receiving PostHog backend records the TCP source IP as $ip. See docs/telemetry.md for details and how to disable / substitute the backend.

For full details see docs/telemetry.md.

Contributing

We welcome contributions of all sizes — from typo fixes to new connectors. drt has a transparent contributor ladder so your work builds toward greater trust and responsibility over time.

Contributors ✨

Every contribution — code, docs, bug reports, ideas — makes drt better. Thank you 💜

drt contributors

Full credits by contribution type (emoji key)
K.Masuda
K.Masuda

💻 📖 🚧 🤔 🔧 📆 👀 🧑‍🏫
yodakanohoshi
yodakanohoshi

💻 📖 🚧 🤔 👀 📆
Moavia Amir
Moavia Amir

💻 📖 🚇 🚧 🤔
Khush Domadiya
Khush Domadiya

💻
Pawan Singh Kapkoti
Pawan Singh Kapkoti

💻
PFCAaron12
PFCAaron12

💻
Semy Ingle
Semy Ingle

💻 🚧
きわみざむらい
きわみざむらい

💻 🐛
armorbreak001
armorbreak001

💻
pureqin
pureqin

💻
Wahaj Ahmed
Wahaj Ahmed

💻
cian-ps
cian-ps

💻 🐛 🚇
Erik Estrella
Erik Estrella

⚠️
Ai (藍)
Ai (藍)

📖
GokulKashyap
GokulKashyap

💻 ⚠️
Add your contributions

Disclaimer

drt is an independent open-source project and is not affiliated with, endorsed by, or sponsored by dbt Labs, dlt-hub, or any other company.

"dbt" is a registered trademark of dbt Labs, Inc. "dlt" is a project maintained by dlt-hub.

drt is designed to complement these tools as part of the modern data stack, but is a separate project with its own codebase and maintainers.

License

Apache 2.0 — see LICENSE.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

drt_core-0.8.0.tar.gz (1.7 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

drt_core-0.8.0-py3-none-any.whl (357.0 kB view details)

Uploaded Python 3

File details

Details for the file drt_core-0.8.0.tar.gz.

File metadata

  • Download URL: drt_core-0.8.0.tar.gz
  • Upload date:
  • Size: 1.7 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for drt_core-0.8.0.tar.gz
Algorithm Hash digest
SHA256 a2868706b1a608d3f501b178ac86efd85f3c18c45c38bbfc864b2099d407deb9
MD5 b29ddc1ca398f0fc4e05eb73c63ceb44
BLAKE2b-256 0e514dca8af3e4477a860dc0c7b117292e7b2c8ae0989452325586c2eb1f462d

See more details on using hashes here.

Provenance

The following attestation bundles were made for drt_core-0.8.0.tar.gz:

Publisher: publish-drt-core.yml on drt-hub/drt

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file drt_core-0.8.0-py3-none-any.whl.

File metadata

  • Download URL: drt_core-0.8.0-py3-none-any.whl
  • Upload date:
  • Size: 357.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for drt_core-0.8.0-py3-none-any.whl
Algorithm Hash digest
SHA256 9237b81834eea1b9b39bc5d4ec03f05726d3529f3cb2f5e8c2d494ef898dc184
MD5 49c45c4c63dfeac5c8428dba8b058f09
BLAKE2b-256 e676b66ab0dc7bfdcde1d8d97828d97588ea96828a15e0357d3cee14d4a2943e

See more details on using hashes here.

Provenance

The following attestation bundles were made for drt_core-0.8.0-py3-none-any.whl:

Publisher: publish-drt-core.yml on drt-hub/drt

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page