dbt-conceptual 0.2.0

Bridge the gap between conceptual models and your lakehouse

dbt-conceptual

Bridge the gap between conceptual models and your lakehouse.

---

⚠️ Opinionated by Design

This tool is built for a specific architecture. It expects:

• dbt — Your transformation layer
• Medallion architecture — Silver (cleaned/conformed) → Gold (business/integrated)
• Dimensional modeling — dims, facts, bridges in gold

What we're opinionated about:

• Naming conventions (dim_, fact_, bridge_ prefixes)
• Layer structure (silver/gold concepts)
• Conceptual model in a single conceptual.yml file

What we're flexible about:

• Whether you use one schema.yml per model or shared schema files
• Your specific silver/gold path names (configurable)
• Whether you use dbt groups, tags, or other organizational features

If that's not your stack, this tool is not for you. No judgment — just not the target audience.

---

The Problem

Every data team has a conceptual model — "we have Customers, Orders, Products" — but it lives in PowerPoint, Visio, ERD tools, or people's heads. Disconnected from code. Diverging from reality. Rotting.

CONCEPTUAL MODEL                          LAKEHOUSE
(slides, whiteboards, heads)              (dbt, Databricks, Snowflake)

  Customer ── Order ── Product    →→→     dim_customer_shopify, dim_customer_crm,
                                          dim_product, fact_order_lines,
        🤷 GAP 🤷                          bridge_customer_segment...

When someone asks "Do we have Customer data?" — you grep through 200 models hoping to find the answer.

The Solution

dbt-conceptual tracks which business concepts your dbt models implement:

# models/conceptual/conceptual.yml
concepts:
  customer:
    domain: party
    owner: customer_team
    definition: "A person or company that purchases products"

  order:
    domain: transaction
    owner: orders_team
    definition: "A confirmed purchase by a customer"

relationships:
  - name: places
    from: customer
    to: order

  - name: contains
    from: order
    to: product

# models/gold/dim_customer/dim_customer.yml
models:
  - name: dim_customer
    meta:
      concept: customer

# models/gold/fact_order_lines/fact_order_lines.yml
models:
  - name: fact_order_lines
    meta:
      realizes:
        - customer:places:order
        - order:contains:product

Then visualize, validate, and export:

┌──────────────────────┐         ┌──────────────────────┐         ┌──────────────────────┐
│      Customer        │         │        Order         │         │       Product        │
│                      │         │                      │         │                      │
│   [S:●●] [G:●]       │━places━▶│   [S:●]  [G:●]       │━contains▶│   [S:●]  [G:●]      │
└──────────────────────┘         └──────────────────────┘         └──────────────────────┘

---

Installation

# No signup required. No telemetry. No "please star this repo" popups.
pip install dbt-conceptual

Quick Start

Try it with the jaffle-shop demo project:

# Clone jaffle-shop
git clone https://github.com/dbt-labs/jaffle-shop.git
cd jaffle-shop

# Install dbt-conceptual
pip install dbt-conceptual[serve]

# Initialize conceptual model
dbt-conceptual init
# This creates models/conceptual/conceptual.yml

# Define your business concepts (see example below)
# Edit models/conceptual/conceptual.yml

# View coverage
dbt-conceptual status

# Launch interactive UI
dbt-conceptual serve

# Validate in CI
dbt-conceptual validate

# Export diagrams
dbt-conceptual export --format excalidraw -o diagram.excalidraw

Example conceptual.yml for jaffle-shop:

version: 1

domains:
  party:
    name: Party
    color: "#E3F2FD"
  transaction:
    name: Transaction
    color: "#FFF3E0"

concepts:
  customer:
    name: Customer
    domain: party
    definition: "A person or entity that places orders"
    status: complete

  order:
    name: Order
    domain: transaction
    definition: "A purchase transaction made by a customer"
    status: complete

relationships:
  - name: places
    from: customer
    to: order
    cardinality: "1:N"

Then tag your dbt models:

# models/customers.yml
models:
  - name: customers
    meta:
      concept: customer

# models/orders.yml
models:
  - name: orders
    meta:
      realizes:
        - customer:places:order

---

Features

📊 Coverage Tracking

See which concepts are implemented at each layer:

$ dbt-conceptual status

Concepts by Domain
==================

party (1 concept)
  ✓ customer [complete]     [S:●●] [G:●]

transaction (2 concepts)
  ✓ order [complete]        [S:●]  [G:●]
  ⚠ shipment [stub]         [S:○]  [G:○]

catalog (1 concept)
  ✓ product [complete]      [S:●]  [G:●]

🎨 Interactive Web UI

Launch a visual editor for your conceptual model with real-time editing:

$ dbt-conceptual serve
Starting dbt-conceptual UI server...
Open your browser to: http://127.0.0.1:5000

Features:

• Graph Editor — Interactive force-directed graph visualization with D3.js
  • Drag and position concepts
  • Click concepts/relationships to edit
  • Real-time visual updates
  • Domain-based coloring
  • Zoom and pan canvas
• Direct Editing — Changes save directly to conceptual.yml
  • Edit concept name, description, definition, domain, owner, status
  • Edit relationship name, from/to concepts, cardinality, description
  • No sync needed - what you see is what gets saved
• Integrated Reports — View coverage and bus matrix in tabs
  • Coverage Report tab shows concept completion and implementations
  • Bus Matrix tab shows which fact tables realize which relationships
  • Switch between Editor, Coverage, and Bus Matrix views

Installation:

# UI requires Flask
pip install dbt-conceptual[serve]

# Or install Flask separately
pip install flask

✅ CI Validation

Catch drift before it ships:

$ dbt-conceptual validate

ERROR: dim_customer_legacy references concept 'client' which does not exist
       Did you mean: 'customer'?

ERROR: fact_returns realizes 'customer:returns:order' 
       but relationship 'returns' does not exist

📤 Export Formats

# Excalidraw — editable diagrams
dbt-conceptual export --format excalidraw

# Mermaid — for docs and GitHub
dbt-conceptual export --format mermaid

# Coverage report — HTML dashboard
dbt-conceptual export --format coverage

# Bus matrix — dimensions vs facts
dbt-conceptual export --format bus-matrix

---

How It Works

1. Define Concepts

Create models/conceptual/conceptual.yml:

version: 1

domains:
  party:
    name: "Party"
    color: "#E3F2FD"
  transaction:
    name: "Transaction"
    color: "#FFF3E0"
  catalog:
    name: "Catalog"
    color: "#F1F8E9"

concepts:
  customer:
    name: "Customer"
    domain: party
    owner: customer_team
    definition: "A person or company that purchases products"
    status: complete

  order:
    name: "Order"
    domain: transaction
    owner: orders_team
    definition: "A confirmed purchase by a customer"
    status: complete

  product:
    name: "Product"
    domain: catalog
    owner: catalog_team
    definition: "An item available for purchase"
    status: complete

relationships:
  - name: places
    from: customer
    to: order
    cardinality: "1:N"

  - name: contains
    from: order
    to: product
    cardinality: "N:M"

2. Tag dbt Models

Add meta.concept to dimensions:

# models/gold/dim_customer/dim_customer.yml
version: 2
models:
  - name: dim_customer
    description: "Customer dimension"
    meta:
      concept: customer

Add meta.realizes to facts and bridges:

# models/gold/fact_order_lines/fact_order_lines.yml
version: 2
models:
  - name: fact_order_lines
    description: "Order line items fact"
    meta:
      realizes:
        - customer:places:order
        - order:contains:product

3. Validate & Visualize

# Check everything links up
dbt-conceptual validate

# See coverage
dbt-conceptual status

# Open visual editor
dbt-conceptual serve

---

Bottom-Up Adoption

Already have a dbt project? Start tagging models, then generate the conceptual model:

# Add tags to your existing schema.yml files
# Then run:
dbt-conceptual sync --create-stubs

# Output:
# Created 12 concept stubs
# Created 8 relationship stubs
# 
# Run 'dbt-conceptual status' to see what needs enrichment

The tags ARE the adoption. The tool just makes them visible.

---

Configuration

Works out of the box. Override in dbt_project.yml if needed:

vars:
  dbt_conceptual:
    conceptual_path: models/conceptual    # default
    silver_paths:
      - models/silver                     # default
      - models/staging                    # add custom paths
    gold_paths:
      - models/gold                       # default
      - models/marts                      # add custom paths

Or via CLI:

dbt-conceptual validate --gold-paths models/marts

---

CI/CD Integration

# .github/workflows/ci.yml
jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      
      - name: Install dependencies
        run: pip install dbt-core dbt-conceptual
      
      - name: Validate conceptual model
        run: dbt-conceptual validate

---

Documentation

• Full Documentation
• Configuration Reference
• CLI Reference
• Export Formats

---

Contributing

Contributions welcome! See CONTRIBUTING.md for guidelines.

PRs that work > PRs with extensive documentation about why they might work.

# Development setup
git clone https://github.com/feriksen-personal/dbt-conceptual.git
cd dbt-conceptual
pip install -e ".[dev]"

# Run tests
pytest

# Run linting
ruff check .
black --check .

---

License

MIT License. See LICENSE for details.

---

Acknowledgments

Built with frustration from years of conceptual models rotting in PowerPoint, Visio, and expensive ERD software.

No mass emails were sent to coordinate this project. No penguins were harmed in its making.

---

_{Works on my machine. Might work on yours.}