datalex-cli 1.0.0

Git-native data modeling for dbt users

DataLex

Git-native data modeling for dbt users.

Point us at your dbt project and warehouse — we produce versioned, reviewable YAML with contracts, lineage, ERDs, and clean round-trip back to dbt.

Quickstart — two commands

pip install 'datalex-cli[serve]'       # CLI + bundled Node — one command, no prereqs
datalex serve                          # opens http://localhost:3030

That's it. No Node install, no Docker, no database. [serve] pulls a portable Node runtime so Python alone is enough. If you already have Node 20+ on PATH, plain pip install datalex-cli works too.

Point it at your dbt repo:

cd ~/my-dbt-project                    # folder containing dbt_project.yml
datalex serve --project-dir .

The folder auto-registers as your active project; the browser opens straight into your real file tree. Every UI edit writes back to the original .yml files — git status shows real diffs.

Build your first ER diagram:

1. Click Import dbt repo → Local folder → pick your project root
2. In the Explorer, right-click any folder → New diagram here… (or use the Explorer toolbar's New Diagram button for the default datalex/diagrams/ location)
3. Open the new .diagram.yaml and click Add Entities on the canvas toolbar — multi-select with search + domain filter, then confirm. Entities auto-layout via ELK on add. You can also drag any schema.yml / .model.yaml from the Explorer onto the canvas as an alternative — FK edges from tests: - relationships: render automatically.
4. Drag to reposition → Save All → positions persist in the diagram file; git commit picks them up. Save All is merge-safe: multiple in-memory docs targeting the same schema.yml are merged through the core-engine merge_models_preserving_docs helper instead of clobbering siblings.

See docs/getting-started.md for the full path matrix (demo → local dbt → git URL → live warehouse).

Want your warehouse drivers too?

pip install 'datalex-cli[serve,postgres]'        # or snowflake, bigquery, databricks…
pip install 'datalex-cli[serve,all]'             # every driver + Node

Pick a tutorial

Once datalex serve is running, follow the path that matches what you have in hand:

You have...	Tutorial	Time
Nothing — just want the demo	Jaffle-shop one-click walkthrough	3 min
An existing dbt project (folder or git)	Import an existing dbt project	5 min
A live warehouse (Snowflake/Postgres/…)	Pull a warehouse schema	7 min
CLI-only, no UI	CLI dbt-sync tutorial	5 min

New here? Start with docs/getting-started.md — it's the map across all four paths plus the mental model.

60-second demo (offline, no warehouse)

pip install 'datalex-cli[duckdb]'
git clone https://github.com/duckcode-ai/DataLex.git
cd DataLex

# 1. Build a local DuckDB warehouse (no external credentials)
python examples/jaffle_shop_demo/setup.py

# 2. Sync the dbt project into DataLex YAML
datalex datalex dbt sync examples/jaffle_shop_demo \
    --out-root examples/jaffle_shop_demo/datalex-out

# 3. Emit dbt-parseable YAML back, with contracts enforced
datalex datalex dbt emit examples/jaffle_shop_demo/datalex-out \
    --out-dir examples/jaffle_shop_demo/dbt-out

Open examples/jaffle_shop_demo/datalex-out/sources/jaffle_shop_raw.yaml — every column has its warehouse type, descriptions from the manifest, and a meta.datalex.dbt.unique_id stamp so re-running the sync never clobbers anything you've hand-authored.

What it does

DataLex treats your data models as code. On top of a stricter YAML substrate (the DataLex layout — one file per entity, kind:-dispatched, streaming-safe for 10K+ entities), it gives you:

• datalex datalex dbt sync <project> — reads target/manifest.json + your profiles.yml, introspects live column types, and merges them into DataLex YAML. Idempotent: user-authored description:, tags:, sensitivity:, and tests: survive re-sync.
• datalex datalex dbt emit — writes sources.yml and schema.yml with contract.enforced: true and data_type: on every column. dbt parse succeeds out of the box.
• datalex datalex emit ddl --dialect ... — Postgres, Snowflake, BigQuery, Databricks, MySQL, SQL Server, Redshift. Same source, all dialects.
• datalex datalex diff — semantic diff with explicit rename tracking (previous_name:), breaking-change gate for CI.
• Cross-repo package imports — pin acme/warehouse-core@1.4.0 in imports:, lockfile + content hash drift detection, Git-or-path resolution, on-disk parse cache for large projects.
• Visual studio — React Flow UI for editing entities, relationships, and metadata; same YAML files as the CLI.

Supported warehouses

Warehouse	dbt sync introspection	Forward DDL	Reverse engineering
DuckDB	✓	—	—
PostgreSQL	✓	✓	✓
Snowflake	(fallback)	✓	✓
BigQuery	(fallback)	✓	✓
Databricks	(fallback)	✓	✓
MySQL	(fallback)	✓	✓
SQL Server / Azure SQL	(fallback)	✓	✓
Redshift	(fallback)	✓	✓

"Fallback" = uses the existing full-schema connector (slower than the per-table path but already works today; a narrow introspection path ships per-dialect over time).

Install

For users — from PyPI:

pip install 'datalex-cli[serve]'                 # CLI + UI (recommended)
pip install 'datalex-cli[serve,postgres]'        # add a warehouse driver
pip install 'datalex-cli[serve,all]'             # every driver + UI
pip install datalex-cli                          # CLI-only, no UI

Available extras: serve, duckdb, postgres, mysql, snowflake, bigquery, databricks, sqlserver, redshift, all.

Prereqs: Python 3.9+ and Git. That's it — [serve] bundles Node.

For contributors — from source:

git clone https://github.com/duckcode-ai/DataLex.git
cd DataLex
python3 -m venv .venv && source .venv/bin/activate
pip install -e '.[serve,duckdb]'
datalex serve                                    # auto-builds the UI on first run

Project layout

DataLex/
  packages/
    core_engine/           # Python: loader, dialects, dbt integration, packages
      src/datalex_core/
        _schemas/datalex/  # JSON Schema per `kind:` — bundled with the package
    cli/                   # `datalex` entry point
    api-server/            # Node.js API (UI backend)
    web-app/               # React Flow studio
  examples/
    jaffle_shop_demo/      # zero-setup dbt-sync demo (DuckDB)
  model-examples/          # sample projects and scenario walkthroughs
  docs/                    # architecture, specs, runbooks
  tests/                   # unittest suite (core engine + datalex)

Visual Studio

datalex serve ships the full UI — no extra setup. If you're hacking on the web app itself and want hot-reload, run the two dev servers from a source checkout:

# Terminal 1 — api (port 3030)
npm --prefix packages/api-server run dev
# Terminal 2 — web (port 5173)
npm --prefix packages/web-app run dev

The UI reads and writes the same YAML files the CLI does — no database, no hosted service.

CI / GitOps

DataLex is designed to live in your repo next to your dbt project. A typical CI step:

./datalex datalex validate datalex/
./datalex datalex diff datalex-main/ datalex/ --exit-on-breaking
./datalex datalex dbt emit datalex/ --out-dir dbt/
dbt parse

Documentation

Onboarding

• Getting started — the one-page map covering install, the three GUI paths, and the mental model.
• Jaffle-shop walkthrough — 3-minute offline demo of every UI feature.
• Import an existing dbt project — 5-minute bring-your-own-repo flow (local folder or git URL).
• Pull a warehouse schema — 7-minute live-connection flow with inferred PKs/FKs and streaming progress.
• CLI dbt-sync tutorial — original CLI-only jaffle_shop walkthrough.

Reference

• DataLex layout reference — what each kind: file looks like and how the loader discovers them.
• CLI cheat sheet — every datalex datalex … subcommand on one page.
• API contracts — HTTP API reference for integrators.
• Architecture — core engine modules and end-to-end data flow.
• Pre-DataLex specs have moved to docs/archive/.

Community

• Discord:
• Issues:
• Contributing: CONTRIBUTING.md
• License: