datalex-cli 0.2.2

Git-native data modeling for dbt users

DataLex

Git-native data modeling for dbt users.

Point us at your dbt project and warehouse — we produce versioned, reviewable YAML with contracts, lineage, ERDs, and clean round-trip back to dbt.

Quickstart (recommended)

pip install 'datalex-cli[serve]'       # CLI + bundled Node — one command, no prereqs
datalex serve                          # opens http://localhost:3030

That's it. [serve] pulls a portable Node so you don't need to install Node separately — and yes, that's intentional. If you already have Node 20+ on your PATH, plain pip install datalex-cli works too.

Working with your own dbt project?

cd ~/my-dbt-project                    # folder containing dbt_project.yml
datalex serve --project-dir .

DataLex auto-registers the folder as a project on first launch, so the browser opens directly into your real tree — no "set up a workspace" click-through. Every UI edit writes back to the original .yml files on disk. See docs/getting-started.md for the full path matrix (demo → local dbt → git URL → live warehouse).

Want your warehouse drivers too?

pip install 'datalex-cli[serve,postgres]'        # or snowflake, bigquery, databricks…
pip install 'datalex-cli[serve,all]'             # every driver + Node

Pick a tutorial

Once datalex serve is running, follow the path that matches what you have in hand:

You have...	Tutorial	Time
Nothing — just want the demo	Jaffle-shop one-click walkthrough	3 min
An existing dbt project (folder or git)	Import an existing dbt project	5 min
A live warehouse (Snowflake/Postgres/…)	Pull a warehouse schema	7 min
CLI-only, no UI	CLI dbt-sync tutorial	5 min

New here? Start with docs/getting-started.md — it's the map across all four paths plus the mental model.

60-second demo (offline, no warehouse)

pip install 'datalex-cli[duckdb]'
git clone https://github.com/duckcode-ai/DataLex.git
cd DataLex

# 1. Build a local DuckDB warehouse (no external credentials)
python examples/jaffle_shop_demo/setup.py

# 2. Sync the dbt project into DataLex YAML
datalex datalex dbt sync examples/jaffle_shop_demo \
    --out-root examples/jaffle_shop_demo/datalex-out

# 3. Emit dbt-parseable YAML back, with contracts enforced
datalex datalex dbt emit examples/jaffle_shop_demo/datalex-out \
    --out-dir examples/jaffle_shop_demo/dbt-out

Open examples/jaffle_shop_demo/datalex-out/sources/jaffle_shop_raw.yaml — every column has its warehouse type, descriptions from the manifest, and a meta.datalex.dbt.unique_id stamp so re-running the sync never clobbers anything you've hand-authored.

What it does

DataLex treats your data models as code. On top of a stricter YAML substrate (the DataLex layout — one file per entity, kind:-dispatched, streaming-safe for 10K+ entities), it gives you:

• datalex datalex dbt sync <project> — reads target/manifest.json + your profiles.yml, introspects live column types, and merges them into DataLex YAML. Idempotent: user-authored description:, tags:, sensitivity:, and tests: survive re-sync.
• datalex datalex dbt emit — writes sources.yml and schema.yml with contract.enforced: true and data_type: on every column. dbt parse succeeds out of the box.
• datalex datalex emit ddl --dialect ... — Postgres, Snowflake, BigQuery, Databricks, MySQL, SQL Server, Redshift. Same source, all dialects.
• datalex datalex diff — semantic diff with explicit rename tracking (previous_name:), breaking-change gate for CI.
• Cross-repo package imports — pin acme/warehouse-core@1.4.0 in imports:, lockfile + content hash drift detection, Git-or-path resolution, on-disk parse cache for large projects.
• Visual studio — React Flow UI for editing entities, relationships, and metadata; same YAML files as the CLI.

Supported warehouses

Warehouse	dbt sync introspection	Forward DDL	Reverse engineering
DuckDB	✓	—	—
PostgreSQL	✓	✓	✓
Snowflake	(fallback)	✓	✓
BigQuery	(fallback)	✓	✓
Databricks	(fallback)	✓	✓
MySQL	(fallback)	✓	✓
SQL Server / Azure SQL	(fallback)	✓	✓
Redshift	(fallback)	✓	✓

"Fallback" = uses the existing full-schema connector (slower than the per-table path but already works today; a narrow introspection path ships per-dialect over time).

Install

From PyPI:

pip install datalex-cli               # puts `datalex` on PATH
pip install 'datalex-cli[duckdb]'     # add warehouse drivers you need

From source (for contributors or editable installs):

git clone https://github.com/duckcode-ai/DataLex.git
cd DataLex
python3 -m venv .venv
source .venv/bin/activate
pip install -e '.[duckdb]'

# optional — only needed for the Visual Studio
npm --prefix packages/api-server install
npm --prefix packages/web-app install

Available extras: duckdb, postgres, mysql, snowflake, bigquery, databricks, sqlserver, redshift, or all.

Prereqs: Python 3.9+, Git. Node.js 18+ if you want the UI.

Project layout

DataLex/
  packages/
    core_engine/           # Python: loader, dialects, dbt integration, packages
      src/datalex_core/
        _schemas/datalex/  # JSON Schema per `kind:` — bundled with the package
    cli/                   # `datalex` entry point
    api-server/            # Node.js API (UI backend)
    web-app/               # React Flow studio
  examples/
    jaffle_shop_demo/      # zero-setup dbt-sync demo (DuckDB)
  model-examples/          # sample projects and scenario walkthroughs
  docs/                    # architecture, specs, runbooks
  tests/                   # unittest suite (core engine + datalex)

Visual Studio (optional)

If you want the UI on top of your DataLex project, run the two dev servers:

# Terminal 1
npm --prefix packages/api-server run dev
# Terminal 2
npm --prefix packages/web-app run dev

Then open http://localhost:5173. The UI reads and writes the same YAML files the CLI does — no database, no hosted service.

CI / GitOps

DataLex is designed to live in your repo next to your dbt project. A typical CI step:

./datalex datalex validate datalex/
./datalex datalex diff datalex-main/ datalex/ --exit-on-breaking
./datalex datalex dbt emit datalex/ --out-dir dbt/
dbt parse

Documentation

Onboarding

• Getting started — the one-page map covering install, the three GUI paths, and the mental model.
• Jaffle-shop walkthrough — 3-minute offline demo of every UI feature.
• Import an existing dbt project — 5-minute bring-your-own-repo flow (local folder or git URL).
• Pull a warehouse schema — 7-minute live-connection flow with inferred PKs/FKs and streaming progress.
• CLI dbt-sync tutorial — original CLI-only jaffle_shop walkthrough.

Reference

• DataLex layout reference — what each kind: file looks like and how the loader discovers them.
• CLI cheat sheet — every datalex datalex … subcommand on one page.
• API contracts — HTTP API reference for integrators.
• Architecture — core engine modules and end-to-end data flow.
• Pre-DataLex specs have moved to docs/archive/.

Community

• Discord:
• Issues:
• Contributing: CONTRIBUTING.md
• License: