dbt-conceptual 0.1.0

Bridge the gap between conceptual models and your lakehouse

dbt-conceptual

Bridge the gap between conceptual models and your lakehouse.

---

⚠️ Opinionated by Design

This tool is built for a specific architecture. It expects:

• dbt — Your transformation layer
• Medallion architecture — Silver (cleaned/conformed) → Gold (business/integrated)
• Dimensional modeling — dims, facts, bridges in gold

What we're opinionated about:

• Naming conventions (dim_, fact_, bridge_ prefixes)
• Layer structure (silver/gold concepts)
• Conceptual model in a single conceptual.yml file

What we're flexible about:

• Whether you use one schema.yml per model or shared schema files
• Your specific silver/gold path names (configurable)
• Whether you use dbt groups, tags, or other organizational features

If that's not your stack, this tool is not for you. No judgment — just not the target audience.

---

The Problem

Every data team has a conceptual model — "we have Customers, Orders, Products" — but it lives in PowerPoint, Visio, or people's heads. Disconnected from code. Drifting. Stale.

CONCEPTUAL MODEL                          LAKEHOUSE
(slides, whiteboards, heads)              (dbt, Databricks, Snowflake)

  Customer ── Order ── Product    →→→     dim_customer_shopify, dim_customer_crm,
                                          dim_product, fact_order_lines,
        🤷 GAP 🤷                          bridge_customer_segment...

When someone asks "Do we have Customer data?" — you grep through 200 models hoping to find the answer.

The Solution

dbt-conceptual tracks which business concepts your dbt models implement:

# models/conceptual/conceptual.yml
concepts:
  customer:
    domain: party
    owner: customer_team
    definition: "A person or company that purchases products"

  order:
    domain: transaction
    owner: orders_team
    definition: "A confirmed purchase by a customer"

relationships:
  - name: places
    from: customer
    to: order

  - name: contains
    from: order
    to: product

# models/gold/dim_customer/dim_customer.yml
models:
  - name: dim_customer
    meta:
      concept: customer

# models/gold/fact_order_lines/fact_order_lines.yml
models:
  - name: fact_order_lines
    meta:
      realizes:
        - customer:places:order
        - order:contains:product

Then visualize, validate, and export:

┌──────────────────────┐         ┌──────────────────────┐         ┌──────────────────────┐
│      Customer        │         │        Order         │         │       Product        │
│                      │         │                      │         │                      │
│   [S:●●] [G:●]       │━places━▶│   [S:●]  [G:●]       │━contains▶│   [S:●]  [G:●]      │
└──────────────────────┘         └──────────────────────┘         └──────────────────────┘

---

Installation

# No signup required. No telemetry. No "please star this repo" popups.
pip install dbt-conceptual

Quick Start

# Initialize in your dbt project
cd my-dbt-project
dbt-conceptual init

# If you already have meta.concept tags, generate the conceptual model
dbt-conceptual sync --create-stubs

# View coverage
dbt-conceptual status

# Launch visual editor
dbt-conceptual serve

# Validate in CI
dbt-conceptual validate

# Export diagram
dbt-conceptual export --format excalidraw

---

Features

📊 Coverage Tracking

See which concepts are implemented at each layer:

$ dbt-conceptual status

Concepts by Domain
==================

party (1 concept)
  ✓ customer [complete]     [S:●●] [G:●]

transaction (2 concepts)
  ✓ order [complete]        [S:●]  [G:●]
  ⚠ shipment [stub]         [S:○]  [G:○]

catalog (1 concept)
  ✓ product [complete]      [S:●]  [G:●]

🎨 Visual Editor

Interactive viewer with hand-drawn Excalidraw aesthetic:

$ dbt-conceptual serve
Serving at http://localhost:8080

• Drag concepts to arrange
• Click to edit properties
• Draw new relationships
• Saves directly to YAML

✅ CI Validation

Catch drift before it ships:

$ dbt-conceptual validate

ERROR: dim_customer_legacy references concept 'client' which does not exist
       Did you mean: 'customer'?

ERROR: fact_returns realizes 'customer:returns:order' 
       but relationship 'returns' does not exist

📤 Export Formats

# Excalidraw — editable diagrams
dbt-conceptual export --format excalidraw

# Mermaid — for docs and GitHub
dbt-conceptual export --format mermaid

# Coverage report — HTML dashboard
dbt-conceptual export --format coverage

# Bus matrix — dimensions vs facts
dbt-conceptual export --format bus-matrix

---

How It Works

1. Define Concepts

Create models/conceptual/conceptual.yml:

version: 1

domains:
  party:
    name: "Party"
    color: "#E3F2FD"
  transaction:
    name: "Transaction"
    color: "#FFF3E0"
  catalog:
    name: "Catalog"
    color: "#F1F8E9"

concepts:
  customer:
    name: "Customer"
    domain: party
    owner: customer_team
    definition: "A person or company that purchases products"
    status: complete

  order:
    name: "Order"
    domain: transaction
    owner: orders_team
    definition: "A confirmed purchase by a customer"
    status: complete

  product:
    name: "Product"
    domain: catalog
    owner: catalog_team
    definition: "An item available for purchase"
    status: complete

relationships:
  - name: places
    from: customer
    to: order
    cardinality: "1:N"

  - name: contains
    from: order
    to: product
    cardinality: "N:M"

2. Tag dbt Models

Add meta.concept to dimensions:

# models/gold/dim_customer/dim_customer.yml
version: 2
models:
  - name: dim_customer
    description: "Customer dimension"
    meta:
      concept: customer

Add meta.realizes to facts and bridges:

# models/gold/fact_order_lines/fact_order_lines.yml
version: 2
models:
  - name: fact_order_lines
    description: "Order line items fact"
    meta:
      realizes:
        - customer:places:order
        - order:contains:product

3. Validate & Visualize

# Check everything links up
dbt-conceptual validate

# See coverage
dbt-conceptual status

# Open visual editor
dbt-conceptual serve

---

Bottom-Up Adoption

Already have a dbt project? Start tagging models, then generate the conceptual model:

# Add tags to your existing schema.yml files
# Then run:
dbt-conceptual sync --create-stubs

# Output:
# Created 12 concept stubs
# Created 8 relationship stubs
# 
# Run 'dbt-conceptual status' to see what needs enrichment

The tags ARE the adoption. The tool just makes them visible.

---

Configuration

Works out of the box. Override in dbt_project.yml if needed:

vars:
  dbt_conceptual:
    conceptual_path: models/conceptual    # default
    silver_paths:
      - models/silver                     # default
      - models/staging                    # add custom paths
    gold_paths:
      - models/gold                       # default
      - models/marts                      # add custom paths

Or via CLI:

dbt-conceptual validate --gold-paths models/marts

---

CI/CD Integration

# .github/workflows/ci.yml
jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      
      - name: Install dependencies
        run: pip install dbt-core dbt-conceptual
      
      - name: Validate conceptual model
        run: dbt-conceptual validate

---

Documentation

• Full Documentation
• Configuration Reference
• CLI Reference
• Export Formats

---

Contributing

Contributions welcome! See CONTRIBUTING.md for guidelines.

PRs that work > PRs with extensive documentation about why they might work.

# Development setup
git clone https://github.com/feriksen-personal/dbt-conceptual.git
cd dbt-conceptual
pip install -e ".[dev]"

# Run tests
pytest

# Run linting
ruff check .
black --check .

---

License

MIT License. See LICENSE for details.

---

Acknowledgments

Built with frustration from years of conceptual models rotting in PowerPoint.

No mass emails were sent to coordinate this project.

---

_{Works on my machine. Might work on yours.}