Skip to main content
drt logo

drt — data reverse tool

The reverse leg of your data stack.

dlt loads data in, dbt transforms it, and drt activates it back out — reverse ETL from your warehouse to the tools your team works in. Declarative YAML, one drt run.

dlt load  →  dbt transform  →  drt activate

CI Coverage PyPI Python License

drt-core downloads dagster-drt downloads Sponsor Website X

All Contributors

drt quickstart demo

pip install drt-core          # core (DuckDB included)
drt init && drt run

Open in GitHub Codespaces — or try it in your browser, zero local setup.


Why drt?

Same DX as dbt. If you know dbt run, you already know drt run — declarative YAML, versioned in Git, reviewed in PRs.

CI-native. Exit codes and --output json drop straight into GitHub Actions, cron, or Dagster / Airflow / Prefect — no GUI, no clickops.

LLM-native. A built-in MCP server and Claude Code skills let AI tools author and run your syncs.

Free and open source. Every connector, the CLI, the MCP server, and the sync engine — Apache 2.0, self-hosted, no lock-in.

What's always free? All connectors, CLI, MCP server, and sync engine. See OPEN_CORE.md for the open core boundary.


Quickstart

No cloud accounts needed — DuckDB + httpbin.org, three commands.

pip install drt-core
mkdir my-drt-project && cd my-drt-project
drt init --template duckdb_to_rest

That scaffolds a runnable syncs/duckdb_to_rest.yml. Seed a tiny DuckDB table and run:

python -c "
import duckdb
c = duckdb.connect('warehouse.duckdb')
c.execute('''CREATE TABLE IF NOT EXISTS users AS SELECT * FROM (VALUES
  (1, 'Alice', 'alice@example.com'),
  (2, 'Bob',   'bob@example.com'),
  (3, 'Carol', 'carol@example.com')
) t(id, name, email)''')
c.close()
"
drt run --dry-run   # preview, no data sent
drt run             # POST each row to httpbin.org
drt status          # check results

Other starter templates

drt init --template list             # see all available templates
drt init --template postgres_to_slack
drt init --template duckdb_to_hubspot

Each template prints next-steps for the env vars / source data it needs. See examples/ for the full collection (Discord, Google Sheets, GitHub Actions, MySQL, ClickHouse, BigQuery, …) and docs/connectors/ for per-connector reference.

Customizing your sync

For a guided wizard that walks you through profile + project setup:

drt init   # interactive — picks a source, configures profile, scaffolds project

Both flows produce the same project shape (drt_project.yml, syncs/, .drt/). drt sources --detailed and drt destinations --detailed print every connector's required env vars and a sample YAML stanza — useful when hand-authoring beyond the templates.


CLI Reference

drt init                    # initialize project
drt list                    # list sync definitions
drt sources                 # list available source connectors
drt destinations            # list available destination connectors
drt run                     # run all syncs
drt run --select <name>     # run a specific sync (globs work: 'users_*')
drt run --select tag:<tag>  # run syncs matching a tag (repeat --select to union)
drt run --select destination:<type>  # run syncs by destination type
drt run --exclude <name>    # subtract syncs (same grammar as --select)
drt run --failed            # re-run only syncs that failed last time
drt run --limit 10          # sampled run: send only N rows (watermark frozen)
drt run --fail-fast         # stop scheduling after the first failure
drt run --vars 'lookback_days: 1'   # override project vars: for this run
drt run --threads 4         # parallel sync execution
drt run --dry-run           # dry run
drt run --verbose           # show row-level error details
drt run --output json       # structured JSON output for CI/scripting
drt run --log-format json   # structured JSON logging to stderr
drt run --profile prd       # override profile (or DRT_PROFILE env var)
drt run --cursor-value '…'  # override watermark cursor for backfill
drt test                    # run post-sync validation tests
drt test --select <name>    # test a specific sync
drt build                   # run each sync and its tests in one pass
drt validate                # validate sync YAML configs
drt status                  # show recent sync status
drt status --output json    # JSON output for status
drt profile list            # list credential profiles in ~/.drt/profiles.yml
drt profile show <name>     # show a profile (secrets masked)
drt profile test <name>     # verify a profile's source connectivity
drt profile add <name>      # interactively add a profile
drt profile remove <name>   # remove a profile
drt serve                   # start HTTP webhook endpoint
drt docs generate --format mermaid  # print project DAG as Mermaid
drt deploy github-actions   # scaffold a scheduled sync workflow (drt-action + secrets wired)
drt mcp run                 # start MCP server (requires drt-core[mcp])
drt --install-completion    # install shell completion (bash/zsh/fish)
drt --show-completion       # show completion script

Visualize your syncs

Generate a Mermaid DAG from your local drt_project.yml and syncs/*.yml files:

drt docs generate --format mermaid > dag.md
graph LR
    subgraph Sources
        src_bigquery_prod["bigquery_prod<br/><i>bigquery</i>"]
    end
    subgraph Syncs
        sync_users_to_hubspot{{"users_to_hubspot<br/><i>upsert</i>"}}
        sync_accounts_to_hubspot{{"accounts_to_hubspot<br/><i>upsert</i>"}}
    end
    subgraph Destinations
        dst_hubspot_contacts["hubspot (contacts)<br/><i>hubspot</i>"]
    end
    src_bigquery_prod -->|extract| sync_users_to_hubspot
    src_bigquery_prod -->|extract| sync_accounts_to_hubspot
    sync_users_to_hubspot -->|load| dst_hubspot_contacts
    sync_accounts_to_hubspot -->|load| dst_hubspot_contacts
    sync_users_to_hubspot -.lookup.-> sync_accounts_to_hubspot

Shell completion

Shell completion is supported for bash, zsh, and fish:

# Recommended: auto-install for your current shell (idempotent)
drt --install-completion

# Or manually add to your shell config (run once from the target shell)
drt --show-completion >> ~/.bashrc   # bash
drt --show-completion >> ~/.zshrc    # zsh
drt --show-completion > ~/.config/fish/completions/drt.fish  # fish

Note: --show-completion outputs the script for your current shell. Run it from the shell you want to configure. The manual >> append is not idempotent — run it once only.

After installation, restart your shell and tab-complete commands and options.


MCP Server

Connect drt to Claude, Cursor, or any MCP-compatible client so you can run syncs, check status, and validate configs without leaving your AI environment.

pip install drt-core[mcp]
drt mcp run

Claude Desktop (~/Library/Application Support/Claude/claude_desktop_config.json):

{
  "mcpServers": {
    "drt": {
      "command": "drt",
      "args": ["mcp", "run"]
    }
  }
}

Available MCP tools:

Tool What it does
drt_list_syncs List all sync definitions
drt_run_sync Run a sync (supports dry_run + compute_diff for --diff parity)
drt_run_test Run post-sync validation tests (mirrors drt test)
drt_get_status Get last run result(s)
drt_get_history Get recent sync run history
drt_validate Validate sync YAML configs
drt_get_schema Return JSON Schema for config files
drt_list_connectors List available sources and destinations
drt_dlq Inspect a sync's Dead Letter Queue (depth + failed records)
drt_retry Replay a sync's Dead Letter Queue (mirrors drt retry)
drt_get_manifest Machine-readable sync catalog + lineage (drt docs JSON)
drt_list_profiles List credential profiles (name + type, no secrets)
drt_test_profile Connectivity check for a credential profile
drt_doctor Environment diagnostics (mirrors drt doctor)

AI Skills for Claude Code

Install the official Claude Code skills to generate YAML, debug failures, and migrate from other tools — all from the chat interface.

Install via Plugin Marketplace (recommended)

/plugin marketplace add drt-hub/drt
/plugin install drt@drt-hub

Tip: Enable auto-update so you always get the latest skills when drt is updated: /plugin → Marketplaces → drt-hub → Enable auto-update

Manual install (slash commands)

Copy the files from .claude/commands/ into your drt project's .claude/commands/ directory.

Skill Trigger What it does
/drt-create-sync "create a sync" Generates valid sync YAML from your intent
/drt-debug "sync failed" Diagnoses a specific error and suggests fixes
/drt-troubleshoot "drt isn't working" Walks a full top-to-bottom diagnostic checklist
/drt-init "set up drt" Guides through project initialization
/drt-migrate "migrate from Census" Converts existing configs to drt YAML

Connectors

Per-connector reference: docs/connectors/ · Discoverable from the CLI: drt sources --detailed / drt destinations --detailed

Sources

Connector Status Install Auth
BigQuery ✅ v0.1 pip install drt-core[bigquery] Application Default / Service Account Keyfile
DuckDB ✅ v0.1 (core) File path
PostgreSQL ✅ v0.1 pip install drt-core[postgres] Password (env var)
Snowflake ✅ v0.5 pip install drt-core[snowflake] Password (env var)
SQLite ✅ v0.4.2 (core) File path
Redshift ✅ v0.3.4 pip install drt-core[redshift] Password (env var)
ClickHouse ✅ v0.4.3 pip install drt-core[clickhouse] Password (env var)
MySQL ✅ v0.5 pip install drt-core[mysql] Password (env var)
Databricks ✅ v0.6 pip install drt-core[databricks] Access Token (env var)
Delta Lake ✅ v0.8 pip install drt-core[deltalake] Storage options (env var)
Iceberg ✅ v0.8 pip install drt-core[iceberg] Catalog properties (env var)
SQL Server ✅ v0.6 pip install drt-core[sqlserver] Password (env var)
REST API ✅ v0.7 (core) Bearer / API Key / Basic / OAuth2

Destinations

34 destinations — warehouses, SaaS APIs, cloud storage, files & webhooks (click to expand)
Connector Status Install Auth
REST API ✅ v0.1 (core) Bearer / API Key / Basic / OAuth2
Slack Incoming Webhook ✅ v0.1 (core) Webhook URL
Discord Webhook ✅ v0.4.2 (core) Webhook URL
GitHub Actions ✅ v0.1 (core) Token (env var)
HubSpot ✅ v0.1 (core) Token (env var)
Zendesk ✅ v0.7 (core) Basic (email + API token)
Amplitude ✅ v0.7 (core) Project API key (env var)
Klaviyo ✅ v0.8 (core) Private API key (env var)
Mixpanel ✅ v0.8 (core) Project token / service account
Google Ads ✅ v0.6 (core) OAuth2 Client Credentials
Google Sheets ✅ v0.4 pip install drt-core[sheets] Service Account Keyfile
PostgreSQL (upsert) ✅ v0.4 pip install drt-core[postgres] Password (env var)
MySQL (upsert) ✅ v0.4 pip install drt-core[mysql] Password (env var)
ClickHouse ✅ v0.5 pip install drt-core[clickhouse] Password (env var)
Parquet file ✅ v0.5 pip install drt-core[parquet] File path
Amazon S3 ✅ v0.7.9 pip install drt-core[s3] AWS credential chain / env vars
Google Cloud Storage ✅ v0.7.9 pip install drt-core[gcs] Application Default / Service Account Keyfile
Azure Blob Storage ✅ v0.7.9 pip install drt-core[azure] Connection string env / DefaultAzureCredential
Microsoft Teams Webhook ✅ v0.5 (core) Webhook URL
CSV / JSON / JSONL file ✅ v0.5 (core) File path
Jira ✅ v0.5 (core) Basic (email + API token)
Linear ✅ v0.5 (core) API Key (env var)
SendGrid ✅ v0.5 (core) API Key (env var)
Notion ✅ v0.6 (core) Bearer Token (env var)
Airtable ✅ v0.8 (core) Personal Access Token (env var)
Twilio SMS ✅ v0.6 (core) Basic (Account SID + Auth Token)
Intercom ✅ v0.6 (core) Bearer Token (env var)
Email SMTP ✅ v0.6 (core) Username / Password (env var)
Salesforce Bulk API 2.0 ✅ v0.6 (core) OAuth2 (username-password)
Staged Upload ✅ v0.6 (core) Configurable per provider
Elasticsearch / OpenSearch ✅ v0.7.9 (core) API key / HTTP Basic (env var)
Snowflake ✅ v0.7 pip install drt-core[snowflake] Password (env var)
Databricks Delta Lake ✅ v0.7.9 pip install drt-core[databricks] Personal Access Token (env var)
BigQuery ✅ v0.8 pip install drt-core[bigquery] ADC / Service Account keyfile

Integrations

Connector Status Install
Dagster ✅ v0.4 pip install dagster-drt
Prefect ✅ v0.6 (core)
Airflow ✅ v0.6 (core)
dbt manifest reader ✅ v0.4 (core)

Roadmap

Upcoming releases → ROADMAP.md (scope, themes, targets) Issue-level tracking → GitHub Milestones Looking to contribute? → Good First Issues

Shipped: now on v0.8.2 — single-object drt docs --inline (navigable hosted catalog on GCS/S3), on top of v0.8.1's sync.match_policy (update_only / create_only on Postgres + HubSpot) · self-contained drt docs --inline for object-store hosting · manifest schema v2, on top of the v0.8.0 drt docs lineage site · project vars: · drt build · dbt-style selection v2. Grouped milestones below; the full per-release changelog lives in CHANGELOG.md and GitHub Releases.

Milestone Highlights
v0.1–0.3 Core CLI · BigQuery / DuckDB / Postgres sources · REST / Slack / HubSpot destinations · incremental sync · MCP server + AI Skills for Claude Code
v0.4–0.6 20+ destinations (Sheets, SQL upserts, Notion, Twilio, Salesforce…) · dagster-drt / Airflow / Prefect · drt test · parallel --threads · JSON Schema validation
v0.7 Production Ready — graceful shutdown · sync history · atomic table swaps · drt doctor · opt-in telemetry · sync.mode: mirror · DLQ + drt profile
v0.7.9–0.7.11 Cloud destinations (S3 / GCS / Azure / Databricks / BigQuery / Elasticsearch) · Delta Lake + Iceberg sources · tracked & scoped mirror · PII masking · VS Code extension
v0.8.0 drt docs lineage site (safe-by-default labels, static SVG DAG, byte-identical) · project vars: · drt build · --fail-fast / --limit / --failed · selection v2 · incremental REST API · watermark.lag · drt deploy github-actions
v0.8.1 sync.match_policy update_only / create_only (Postgres + HubSpot) · drt docs generate --inline (self-contained HTML for object-store hosting) · manifest schema v2 (run history · column facts · DLQ depth) · tracked-mirror no-CREATE-grant · Retry-After
v0.8.2 drt docs --inline → single navigable HTML object — the whole catalog in one file with in-page nav, so it renders and navigates on an authenticated GCS / S3 object URL (completes v0.8.1's --inline)

Next: v0.8.3 Diff Polishv0.9 Enterprise Foundationv1.0 Stable Releasev1.x Rust Engine


Orchestration: dagster-drt

Community-maintained Dagster integration. Expose drt syncs as Dagster assets with full observability.

pip install dagster-drt
from dagster import AssetExecutionContext, Definitions
from dagster_drt import drt_assets, DagsterDrtResource

@drt_assets(project_dir="path/to/drt-project")
def my_syncs(context: AssetExecutionContext, drt: DagsterDrtResource):
    yield from drt.run(context=context)

defs = Definitions(
    assets=[my_syncs],
    resources={"drt": DagsterDrtResource(project_dir="path/to/drt-project")},
)

See dagster-drt README for full API docs (Translator, Pipes support, DrtConfig dry-run, MaterializeResult).


CI/CD: GitHub Action

Run drt syncs straight from CI/CD with the official drt-hub/drt-action — no infrastructure, just a few lines of YAML. Trigger on a schedule, on every push, or right after dbt finishes.

- uses: drt-hub/drt-action@v1
  with:
    select: '*'
    extras: postgres
  env:
    PG_PASSWORD: ${{ secrets.PG_PASSWORD }}

Inputs cover sync selection (select), connector extras, profile, dry-run and threads; outputs expose status, succeeded, failed and duration-seconds (plus a step-summary table). Secrets are passed via env: and resolved by drt's *_env keys. See the action README for the secrets pattern and more examples (run-after-dbt, PR preview).


Ecosystem

drt is designed to work alongside, not against, the modern data stack:

drt ecosystem — dlt load, dbt transform, drt activate


Telemetry

drt collects no telemetry by default. Opting in helps us understand which sources / destinations / sync modes are actually used, so we can prioritise.

drt config set telemetry.enabled true     # opt in
drt config show-telemetry                 # preview the exact payload that would be sent
drt config set telemetry.enabled false    # opt out
DO_NOT_TRACK=1 drt run                    # universal kill switch — overrides everything

When opted in, drt sends one sync_completed event per sync. The only properties we collect are these 9 fields: drt_version, python_version, os, source_type, destination_type, sync_mode, rows_synced, duration_seconds, status. The wire envelope additionally carries event, distinct_id (a per-machine random UUID at ~/.drt/.anonymous_id), timestamp, and api_key. Sync names, model SQL, destination URLs, credentials, and project paths are never transmitted — the allow-list is enforced at the function-signature level in drt/telemetry.py. By default events go to PostHog Cloud (EU region); override with DRT_TELEMETRY_ENDPOINT and DRT_TELEMETRY_API_KEY for self-hosted PostHog or a custom collector.

Note: drt itself never transmits your IP, but the receiving PostHog backend records the TCP source IP as $ip. See docs/telemetry.md for details and how to disable / substitute the backend.

For full details see docs/telemetry.md.

Contributing

We welcome contributions of all sizes — from typo fixes to new connectors. drt has a transparent contributor ladder so your work builds toward greater trust and responsibility over time.

Contributors ✨

Every contribution — code, docs, bug reports, ideas — makes drt better. Thank you 💜

drt contributors

Full credits by contribution type (emoji key)
K.Masuda
K.Masuda

💻 📖 🚧 🤔 🔧 📆 👀 🧑‍🏫
yodakanohoshi
yodakanohoshi

💻 📖 🚧 🤔 👀 📆
Moavia Amir
Moavia Amir

💻 📖 🚇 🚧 🤔
Khush Domadiya
Khush Domadiya

💻
Pawan Singh Kapkoti
Pawan Singh Kapkoti

💻
PFCAaron12
PFCAaron12

💻
Semy Ingle
Semy Ingle

💻 🚧
きわみざむらい
きわみざむらい

💻 🐛
armorbreak001
armorbreak001

💻
pureqin
pureqin

💻
Wahaj Ahmed
Wahaj Ahmed

💻
cian-ps
cian-ps

💻 🐛 🚇
Erik Estrella
Erik Estrella

⚠️
Ai (藍)
Ai (藍)

📖
GokulKashyap
GokulKashyap

💻 ⚠️
Add your contributions

Disclaimer

drt is an independent open-source project and is not affiliated with, endorsed by, or sponsored by dbt Labs, dlt-hub, or any other company.

"dbt" is a registered trademark of dbt Labs, Inc. "dlt" is a project maintained by dlt-hub.

drt is designed to complement these tools as part of the modern data stack, but is a separate project with its own codebase and maintainers.

License

Apache 2.0 — see LICENSE.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

drt_core-0.8.2.tar.gz (1.8 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

drt_core-0.8.2-py3-none-any.whl (367.8 kB view details)

Uploaded Python 3

File details

Details for the file drt_core-0.8.2.tar.gz.

File metadata

  • Download URL: drt_core-0.8.2.tar.gz
  • Upload date:
  • Size: 1.8 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for drt_core-0.8.2.tar.gz
Algorithm Hash digest
SHA256 03eef88c314fbec20f74522f5adbfafa742c4802d5b94a3db325926334f10ae4
MD5 867f96e9917568c7044735497562dfe5
BLAKE2b-256 f6735771905d5a0c07c59836a03265503f9c47d89748e1602c0c3880db9a7133

See more details on using hashes here.

Provenance

The following attestation bundles were made for drt_core-0.8.2.tar.gz:

Publisher: publish-drt-core.yml on drt-hub/drt

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file drt_core-0.8.2-py3-none-any.whl.

File metadata

  • Download URL: drt_core-0.8.2-py3-none-any.whl
  • Upload date:
  • Size: 367.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for drt_core-0.8.2-py3-none-any.whl
Algorithm Hash digest
SHA256 6928d9c7ee3ca5be9dfc3d1634bfd4492a343a8bb7a03803835d2a68530e80c4
MD5 e07d7fa3f68d6d0524a13f4a76d24e31
BLAKE2b-256 5e09d98d631bad2d2706a1eedfd100b4666a744125c4961e9641296d4f351a8f

See more details on using hashes here.

Provenance

The following attestation bundles were made for drt_core-0.8.2-py3-none-any.whl:

Publisher: publish-drt-core.yml on drt-hub/drt

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page