exmergo-dex-core 1.0.0

Analytics engineering for Claude Code and any agent: data warehouse exploration, dbt transformation and semantic modeling, and schema-drift maintenance on dbt.

exmergo-dex-core

The portable, Apache-2.0 analytics-engineering engine behind dex. All non-trivial logic lives here; the Claude Code skills and the cross-agent AGENTS.md are thin wrappers that drive it through one stable command contract.

dex is the agent-native analytics engineering toolkit: explore an unfamiliar warehouse, transform raw data into clean dbt models and a semantic layer on top, and maintain all of it as the data underneath changes. Read-only against your data; every change is a reviewable diff.

Install

pip install "exmergo-dex-core"

Connector client libraries live behind extras, so the zero-credential DuckDB on-ramp installs only duckdb and sqlglot:

exmergo-dex-core[duckdb]       # the on-ramp and the eval/benchmark engine
exmergo-dex-core[snowflake]
exmergo-dex-core[bigquery]
exmergo-dex-core[databricks]
exmergo-dex-core[postgres]
exmergo-dex-core[all]          # every connector at once

The command contract

Every subcommand prints exactly one sanitized JSON envelope to stdout and nothing else; nothing reaches agent context except through that envelope. Credentials never cross it, and data values cross only from profiled, PII-cleared columns, bounded and capped by the query firewall. State persists in .dex/, so subcommands are stateless and the agent orchestrates multi-step flows.

dex connect test --path data.duckdb

See references/command-contract.md for the full surface and the envelope spec.

Status

Early and under active development; expect pre-release versions. Today the engine runs Explore and Transform end to end on DuckDB and on BigQuery.

Explore: ranks what matters in an unfamiliar warehouse, profiles columns selectively, flags PII, surfaces grain and data-quality warnings, infers joins and verifies them with overlap probes (--verify), and executes agent-authored ad-hoc SELECTs behind a PII-aware query firewall (explore query), all read-only.

Transform: bootstraps a dbt project where none exists (transform init, with an explicit connector, never a default), turns agent-authored edits and deterministic staging scaffolds into reviewable, conflict-checked diffs (transform plan / apply, with human edits authoritative on conflict), runs gated dev-target-only builds with cost surfaced before any spend (transform build), and authors the semantic layer as MetricFlow-validated dbt semantic models (semantic define|update|plan, applied with transform apply).

BigQuery: connects through Application Default Credentials (gcloud auth application-default login; dex discovers credentials, it never asks for keys). Metadata is free; every scan is dry-run first, returned as a needs_confirmation estimate, and runs only with --confirm --budget <bytes>, capped server-side by maximum_bytes_billed and recorded in a local .dex/spend.jsonl ledger. dbt builds go to a dedicated dev dataset via dbt-bigquery, which the [bigquery] extra carries. See references/bigquery.md.

Snowflake: connects through discovered credentials (connections.toml, SNOWFLAKE_* env, or a dbt profile; dex never asks for or persists a password). The cost inversion from BigQuery: metadata is free (SHOW commands, no warehouse), while scans bill warehouse time, so budgets are warehouse-seconds with credits shown alongside. Estimates are an honestly labeled heuristic (Snowflake has no dry-run), floored by the 60-second resume minimum on a cold warehouse; the budget is hard-enforced anyway by a per-statement server-side STATEMENT_TIMEOUT_IN_SECONDS, and actual seconds land in the same .dex/spend.jsonl ledger. Billed work runs only on the warehouse the config pins. dbt builds go to a dedicated dev database.schema via dbt-snowflake, which the [snowflake] extra carries. See references/snowflake.md.

PostgreSQL: the operational-database connector. Connects through discovered credentials (pg_service.conf, DATABASE_URL, the PG* environment, or a dbt profile; dex never asks for or persists a password). Nothing is billed in dollars; the guarded quantity is load on what is often a production primary, so budgets are database-seconds through the same confirm handshake. Query estimates come from the genuinely free planner preflight (EXPLAIN), profile estimates from relation sizes, both labeled heuristic; the budget is hard-enforced anyway by a per-statement server-side statement_timeout, and actual seconds land in the same ledger. The session is read-only at the server (default_transaction_read_only = on), profiling leans on the planner's own statistics instead of scanning distincts, and dbt builds go to a dedicated dev schema via dbt-postgres, which the [postgres] extra carries, with the ceiling injected as a statement timeout through PGOPTIONS. See references/postgres.md.

Maintain detects drift against the .dex/ snapshot on four axes and proposes the fix: schema (structure), volume (freshness), grain (uniqueness and fanout), and semantic (definitions, dangling references, and dimension cardinality). maintain check sweeps all of them, ranked by blast radius; reconcile proposes reviewable diffs tagged mechanical or advisory, applied through transform apply. Detection is read-only on every connector; on billed connectors the metadata axes (schema, volume, references) stay free while the scanning axes (grain, dimension cardinality) take the --confirm --budget handshake, so check is two-phase.

The remaining cloud connector (Databricks) and the Viz preview report not_implemented until they land.

License

Apache-2.0.