Skip to main content

AI-first CLI for dbt metadata extraction - Fast, modern Python tool

Project description

dbt-meta

โšก AI-first CLI for dbt metadata extraction

dbt-meta is a lightning-fast command-line tool that extracts metadata from dbt's artifacts for DEs and AI agents, eliminating the need to parse .sql files or query your data warehouse for schema information. This is especially useful for fast and accurate agent operation, for example, Claude Code.

โœจ Features

  • ๐ŸŽฏ Works out-of-box โ€” Simple Mode: just run dbt compile and start using
  • โš™๏ธ TOML configuration โ€” Modern config files with XDG compliance (optional)
  • โšก Lightning fast โ€” Optimized Python with LRU caching and orjson parser
  • ๐Ÿ”„ Production Mode โ€” Full defer workflow support for multi-project setups and development environment
  • ๐Ÿ“Š AI-friendly JSON โ€” Machine-readable structured output (-j flag) on every command
  • ๐Ÿ” Rich metadata โ€” Schema, columns, dependencies, config, compiled SQL, column docs
  • ๐ŸŒณ Dependency navigation โ€” Trace upstream/downstream models (flat or tree view)
  • ๐Ÿ”Ž Advanced filtering โ€” Filter models by tags, config, path, package with OR/AND logic
  • ๐Ÿ”€ Git-aware โ€” Find modified models and dependencies needing --full-refresh
  • ๐Ÿ“‹ Smart search โ€” Find models by name or description
  • โœ… SQL validation โ€” Validate SQL syntax using BigQuery dry run
  • ๐Ÿ“ Scan estimation โ€” Estimate query scan size before running (MB / GB)
  • ๐Ÿ“Š Optimization analysis โ€” Find hotspots, analyze single models, branch-level alignment
  • ๐Ÿงฌ Column-level lineage โ€” meta lineage build/column/downstream/stats with rustworkx graph (sub-10ms queries)
  • ๐ŸŽฏ Column-usage-aware advisors โ€” meta optimize cluster/partition/refresh recommends BigQuery clustering/partition keys and minimal --full-refresh plan based on real downstream WHERE/JOIN usage
  • ๐Ÿ”— Power BI integration โ€” Extract BigQuery tables (+ DAX measures & column schemas) from dashboards
  • ๐Ÿ” Artifact sync โ€” meta refresh pulls prod artifacts or parses local project
  • ๐ŸŽจ Beautiful UI โ€” Rich terminal formatting with categorized help panels
  • โšก Combined flags โ€” Use -dj, -adj, -mf for faster typing (order-independent)

๐Ÿค– Built for AI Workflows

dbt-meta was specifically designed to eliminate AI agent hallucinations when working with dbt projects.

The Problem

AI agents (like Claude Code, GitHub Copilot, ChatGPT) often hallucinate when working with dbt:

  • โŒ Wrong table names - Confusing alias vs filename (customers vs dim_customers)
  • โŒ Wrong schema names - Confusing prod and dev schemas
  • โŒ Unknown dependencies - Missing refs/sources in lineage
  • โŒ Incorrect column types - Using wrong data types in WHERE clauses
  • โŒ Non-existent fields - Querying columns that don't exist

The Solution

Following Anthropic's recommendation to use CLI tools over MCP for AI agents, dbt-meta provides:

  • โœ… Fast - Optimized Python with caching, no repeated manifest parsing
  • โœ… Machine-readable JSON - Every command has -j flag for structured output, no text parsing needed
  • โœ… Schema validation - Prevents hallucinations by providing accurate metadata
  • โœ… Type-safe - Mypy strict mode, comprehensive test coverage (91%+)
  • โœ… 3-level fallback - Production manifest โ†’ Dev manifest โ†’ your database (always finds metadata)
  • โœ… Git-aware - Auto-detects model state (modified, new, deleted) with helpful warnings

Integration

dbt-meta integrates seamlessly with:

  • Claude Code (Anthropic) - Add to allowed commands in .claude/settings.local.json
  • GitHub Copilot - Use in terminal and inline suggestions
  • ChatGPT / Custom GPTs - Execute commands and parse JSON output
  • Other AI agents - Standard CLI interface with JSON output

Why CLI over MCP?

  • Have deterministic, structured output
  • Are faster and more reliable
  • Work in any environment
  • Don't require additional infrastructure

Performance

dbt-meta uses several optimization techniques:

  • LRU Caching: ManifestParser cached with @lru_cache(maxsize=1)
  • orjson: Fast JSON parsing (2-3x faster than standard json)
  • Lazy loading: Manifest parsed only when needed
  • Catalog fallback: Use catalog.json instead of BigQuery queries

Measured performance (~900 models manifest):

Command Time Notes
meta schema ~250ms Manifest only
meta info ~335ms Manifest only
meta parents --all ~300ms Traversed 295 ancestors
meta columns (catalog) ~50ms With fresh catalog.json
meta columns (BigQuery via bq CLI) ~2-3s Fallback when catalog stale

Tip: Keep catalog.json fresh (prod state) for fastest columns performance. But this only works for unmodified columns. For models built using defer in dev schema, column metadata is only in DWH.

๐Ÿ“ฆ Installation

PyPI Installation (Recommended)

# Install from PyPI (when published)
pip install dbt-meta

# Verify installation
dbt-meta --version
# or use shorter alias
meta --version

Development Installation

# Clone repository
git clone https://github.com/Filianin/dbt-meta.git
cd dbt-meta

# Install in development mode
pip install -e .

# Or install with dev dependencies
pip install -e ".[dev]"

# Verify installation
meta --version

Requirements

  • Python 3.9+ (3.12+ recommended for best performance)
  • dbt project with manifest.json
  • Optional: jq for advanced JSON processing

๐Ÿš€ Quick Start

Simple Mode (no configuration)

# Step 1: Compile your dbt project
dbt compile

# Step 2: Use dbt-meta immediately!
meta schema customers           # โ†’ your_project.analytics.customers
meta columns -j orders          # โ†’ JSON array of columns
meta deps customers             # โ†’ Dependencies list
meta list tag:daily             # โ†’ Filter models by tag
meta search "customer"          # โ†’ Find models by description

# Get comprehensive help with examples
meta --help
meta

Production Mode (with defer workflow)

To organize a dev environment, you need to have the current version of the prod manifest.json, which will be regularly updated to the latest state. For example, you can regularly compile your manifest and upload it to some cloud storage. From there, you can download this file to your machine in any way you like. If you generate documentation and upload it as a static website, then catalog.json is generated as part of this process, which is recommended to be uploaded along with the manifest. This file contains data about columns and data types that are missing from the manifest.

# One-time setup: Create config file
meta settings init

# Edit ~/.config/dbt-meta/config.toml:
prod_manifest_path = "~/dbt-state/manifest.json"
dev_schema = "personal_myname"

# Now works from any directory!
cd /tmp && meta schema customers  # โ†’ Uses production manifest

# For dev models (after defer run):
defer run --select customers
meta schema --dev customers      # โ†’ personal_myname.customers
meta columns -dj customers       # โ†’ Dev columns with JSON output

Combined Flags (faster typing)

meta schema -dj customers                    # โ†’ Dev + JSON
meta parents -ajd model                      # โ†’ All ancestors + JSON + Dev
meta columns -j --manifest ~/path.json m     # โ†’ JSON + Custom manifest

๐Ÿ“š Commands Reference

All commands accept -h/--help for detailed per-command help.

Core Metadata

Command Description Key flags Example
info <model> Model summary (name, schema, table, materialization, tags) -j, -d meta info -j customers
schema <model> Full table name (database.schema.table) -j, -d meta schema customers
path <model> Relative file path to .sql file -j, -d meta path customers
columns <model> Column names and types -j, -d meta columns -dj customers
columns --all <pattern> Repo-wide column search โ€” every model with a column matching <pattern> -j, --case-sensitive meta columns --all user_id
config <model> Full dbt config (partition_by, cluster_by, incremental, etc.) -j, -d meta config -j customers
sql <model> Compiled SQL (default) or raw with --jinja -j, -d, --jinja meta sql --jinja customers
docs <model> Column names, types, and descriptions -j, -d meta docs customers
find <fqn> Reverse FQN lookup โ€” which dbt model materialises <table> / <schema.table> / <database.schema.table> -j, -d meta find analytics.customers

0.3.1 breaking: meta deps was removed. Use meta parents for upstream refs/sources and meta children for downstream consumers (meta children --source <schema.table> for source-rooted lookup).

Lineage (model-level)

Command Description Key flags Example
parents <model> Upstream dependencies (direct or all ancestors) -j, -d, -a meta parents -aj customers
children <model> Downstream dependencies (direct or all descendants) -j, -d, -a meta children -a customers
children --source <ref> Downstream models for a dbt source (schema.table, source_name.table, or bare table) -j, -a meta children --source raw_events.orders
  • Without -a: direct parents/children only (classic format).
  • With -a -j and โ‰ค20 nodes: nested JSON with children key; otherwise flat array.

Column-level lineage (meta lineage)

Backed by SQLGlot 30.7+ (sqlglot.lineage, all-columns mode) and rustworkx for native graph traversal. The graph is built once into a JSON artifact (~/dbt-state/lineage.json), then queried in sub-10 ms.

Command Description Key flags Example
lineage build Build the lineage artifact from manifest + catalog -d, -j, -v, --timeout N, -o PATH, --manifest, --catalog meta lineage build -v
lineage column <model>.<col> Direct + transitive upstream lineage for a column -d, -j, --artifact meta lineage column core_clients.client_id
lineage downstream <model>.<col> Direct + transitive downstream impact for a column -d, -j, --artifact meta lineage downstream raw.events.user_id
lineage stats Print artifact summary (nodes, edges, generated_at, warnings) -d, -j, --artifact meta lineage stats -j

build flags:

  • -v, --verbose โ€” print per-model progress ([123/934] model_name (0.42s) ok)
  • --timeout N โ€” per-model SIGALRM budget in seconds (default 30, 0 disables)
  • -o PATH โ€” explicit output path (default: ~/dbt-state/lineage.json for prod, ./target/lineage.json for --dev)

For best performance install the mypyc-compiled SQLGlot: pip install 'dbt-meta[lineage]' 'sqlglot[c]' โ€” gives 2-4ร— speedup on large projects (~470 s โ†’ ~150 s for 934 models).

Optimization advisors (meta optimize)

Column-usage-aware advisors. They read each downstream model's compiled SQL via SQLGlot and apply explainable heuristics. No INFORMATION_SCHEMA.JOBS_BY_PROJECT, no LLM, no extra build step โ€” pure AST analysis on existing artifacts. On-demand only (no extra CI artifact).

Command Description Key flags Example
optimize cluster <model> Recommend BigQuery clustering keys (โ‰ค4) from downstream WHERE/JOIN/GROUP BY usage -d, -j, --top N meta optimize cluster core_sessions
optimize partition <model> Recommend a single partition column (DATE/DATETIME/TIMESTAMP/INT64) + granularity -d, -j meta optimize partition core_clients
optimize refresh [<models>...] Column-aware --full-refresh planner with chain-aware propagation -m, --base BRANCH, --cols MODEL:c1,c2, -d, -j, --no-compile meta optimize refresh -m

optimize cluster heuristic: per-column score = WHERE eqร—3.0 + WHERE inร—2.5 + WHERE between/rangeร—2.0 + JOINร—2.0 + GROUP BYร—1.0 + WHERE fn-wrappedร—0.5, multiplied by log2(downstream_count_using_column + 1). Excludes the model's own partition_by column and types unfit for clustering (STRUCT, ARRAY, GEOGRAPHY, JSON).

optimize partition heuristic: range/equality filter weights ร— type bonus (TIMESTAMP/DATE ร— 1.5, DATETIME ร— 1.3, INT64 ร— 1.0). Granularity heuristic: TIMESTAMP/DATE โ†’ DAY, INT64 โ†’ RANGE_BUCKET. Returns one primary recommendation + up to 4 alternatives + pruning_impact_pct (% of downstream queries that would benefit).

optimize refresh algorithm: topological BFS over transitive downstream. Affectedness propagates through the chain โ€” a 3-level descendant C that never directly mentions changed A is still classified correctly when A โ†’ B โ†’ C and B consumes A's changed columns. SELECT * propagates whole-row affectedness through the chain via SQLGlot AST detection (not regex). Per-model bucket: full_refresh (incremental key collision, SELECT *, or non-incremental materialization) / incremental (incremental + only safe columns affected) / skip.

optimize refresh flags:

  • -m, --modified โ€” auto-detect changed models from git (committed-vs-base + uncommitted + untracked)
  • --base BRANCH โ€” explicit base for git diff (auto: origin/main โ†’ origin/master โ†’ main โ†’ master)
  • --cols MODEL:c1,c2 โ€” column-level diff for precise propagation; without it the planner conservatively treats the entire changed model as affected
  • --no-compile โ€” skip the on-demand dbt compile fallback (level 3 of compiled-SQL chain)

Compiled-SQL fallback chain (used by optimize refresh): manifest compiled_code โ†’ <project>/target/compiled/<pkg>/<path>.sql on disk โ†’ one bulk dbt compile --select <downstream models> per run. Only triggers when project root is found (dbt_project.yml), dbt is in PATH, and >50 % of a sampled downstream slice lacks SQL.

Output (text mode) ends with a ready-to-paste shell command:

Suggested commands:
  dbt run -fs core_sessions stg_clients core_orders ...
  dbt run -s some_incremental_model

The line is emitted via plain print() (not Rich) so terminal copy-paste returns one uninterrupted command.

Discovery & Filtering

Command Description Key flags Example
list [selectorsโ€ฆ] Filter with selectors (tag:, config., path:, package:, name:) -j, -d, -m, -f, -a, --and, --group meta list tag:daily --and tag:core
models [pattern] Simple substring search over model names -j meta models staging
search <query> Search models by name or description -j meta search "customer" -j
resolve <fuzzy> Fuzzy "did you mean?" lookup via difflib (after not found) -j, -n LIMIT, --cutoff meta resolve client_profile_evnts
sources [name] List dbt sources from manifest (with --freshness) -j, --freshness meta sources stripe --freshness

list selectors:

  • tag:name โ€” filter by tag (OR logic by default)
  • config.key:value โ€” filter by config value (e.g. config.materialized:incremental)
  • path:dir/ โ€” filter by file path prefix
  • package:name โ€” filter by package
  • name:substr โ€” substring match on model name (case-insensitive)

list flags:

  • --and โ€” require ALL selectors to match (default: OR)
  • --group โ€” group output by tag combinations (headers)
  • -m, --modified โ€” git-aware: only changed/new models

For refresh planning of changed models, use meta optimize refresh (column-aware chain analysis).

SQL Validation & Cost

Uses BigQuery dry run (no rows processed, no charge for dry run itself). Both commands fall back through manifest โ†’ target/compiled/ โ†’ auto dbt compile (with --dev) โ€” see details below.

Command Description Key flags Example
validate <model> [<more>โ€ฆ] Validate compiled SQL syntax (batch-friendly; exits 1 if any fails) -j, -d, --no-compile meta validate orders customers payments
scan <model> Estimate query scan size (๐ŸŸข <1 GB, ๐ŸŸก 1โ€“10 GB, ๐Ÿ”ด โ‰ฅ10 GB) -j, -d meta scan --dev -j customers

Optimization (requires dbt_bigquery_monitoring)

Command Description Key flags Example
hotspots Rank all tables by optimization score (cost, partitioning, clustering, cache) -j, -n/--limit, --min-gb meta hotspots -n 20 --min-gb 10
analyze <model> Deep analysis of a single model โ€” storage, usage, recommendations -j meta analyze -j customers
branch <model> Branch-level analysis โ€” upstream/downstream partitioning alignment -j meta branch customers

hotspots flags:

  • -n, --limit N โ€” number of hotspots to display (default: 10)
  • --min-gb GB โ€” minimum table size in GB (default: 1.0)

Integration โ€” Power BI

Requires Azure AD Service Principal with Power BI Admin API access.

Command Description Key flags Example
powerbi [workspace_id] Power BI โ†’ BigQuery โ†’ dbt model mapping -j, --by-table, --measures, --columns, --full meta powerbi --by-table -j

powerbi flags:

  • --measures โ€” include DAX measure expressions
  • --columns โ€” include column schemas with data types
  • --full โ€” include all metadata (measures + columns)
  • --by-table โ€” aggregated view grouped by BigQuery table

Artifacts & Settings

Command Description Key flags Example
refresh Sync production artifacts from remote storage -d/--dev (parse local via dbt parse) meta refresh
settings init Create config file from template -f/--force (overwrite existing) meta settings init
settings show Display merged configuration (TOML + env) -j meta settings show -j
settings validate Validate active config file โ€” meta settings validate
settings path Show active config file path โ€” meta settings path

Global Flags

Flag Description Applies to
-h, --help Show help All commands
-v, --version Show version and exit Main app
--manifest PATH Explicit path to manifest.json (takes precedence over --dev) All metadata commands
-d, --dev Use dev manifest/schema (./target/manifest.json, personal_USERNAME) Most metadata commands
-j, --json Output as JSON (AI-friendly structured data) Most commands

Combined short flags work in any order: -dj, -adj, -mf, -fa, etc.

Note: --manifest has no short form. If --manifest and --dev are both provided, --dev is ignored with a warning.

Manifest discovery precedence (highest โ†’ lowest):

  1. --manifest PATH โ€” explicit
  2. DBT_DEV_MANIFEST_PATH โ€” only with --dev (default ./target/manifest.json)
  3. DBT_PROD_MANIFEST_PATH โ€” default (~/dbt-state/manifest.json)

BigQuery location: bq calls (columns, validate, scan) default to EU. Override via DBT_META_BQ_LOCATION (e.g. US, asia-east1).

๐Ÿ’ก Common Use Cases

Querying BigQuery with Correct Table Names

# Get production table name (eliminates AI hallucinations)
TABLE=$(meta schema customers)
bq query "SELECT * FROM $TABLE LIMIT 10"
# โ†’ SELECT * FROM your_project.analytics.dim_customers LIMIT 10

# Or with JSON output
TABLE=$(meta schema -j customers | jq -r '.full_name')
bq query "SELECT * FROM $TABLE LIMIT 10"

Finding All Columns for a Model

# Get column list for WHERE clauses
meta columns -j orders | jq -r '.[] | .name'
# โ†’ order_id, customer_id, order_date, status, amount

# Get column types for schema validation
meta columns -j orders | jq -r '.[] | "\(.name): \(.data_type)"'
# โ†’ order_id: INTEGER, customer_id: INTEGER, order_date: DATE, ...

Analyzing Dependencies

# Get all upstream models (for CI/CD impact analysis)
meta parents -aj customers | jq -r '.[] | .path'
# โ†’ staging/customers.sql, staging/orders.sql, staging/payments.sql

# Find downstream impact of model changes
meta children -a customers
# โ†’ Shows all models that depend on customers

Working with Dev Models

# Build dev model
defer run --select customers

# Query dev table (not production)
TABLE=$(meta schema --dev customers)
bq query "SELECT * FROM $TABLE LIMIT 10"
# โ†’ SELECT * FROM personal_USERNAME.customers LIMIT 10

Validating SQL and Estimating Scan Size

# Validate SQL syntax before running (uses BigQuery dry run)
meta validate customers
# โ†’ โœ… Valid

# Check for syntax errors
meta validate broken_model
# โ†’ โŒ Error: Unrecognized name: unknown_column at [3:5]

# Estimate query scan size
meta scan customers
# โ†’ Scan size: 3.2 GB

# JSON output for scripting
meta scan -j customers | jq -r '.formatted'
# โ†’ 3.2 GB

# Use in CI/CD to prevent expensive queries
BYTES=$(meta scan -j customers | jq -r '.bytes')
if [ "$BYTES" -gt 10000000000 ]; then  # 10 GB limit
  echo "Query too expensive: $BYTES bytes"
  exit 1
fi

# With local changes: --dev auto-compiles if needed
meta validate --dev my_model
# โ†’ โ„น๏ธ  No compiled SQL for 'my_model'. Running `dbt compile --select my_model --target dev`...
# โ†’ โœ… Valid

Compiled SQL lookup (validate / scan): 3-level fallback strategy:

  1. model['compiled_code'] from manifest (always works after dbt compile/dbt run)
  2. target/compiled/{package}/{original_file_path} on disk (works if you ran dbt compile separately)
  3. With --dev: auto-runs dbt compile --select <model> --target dev and re-reads from disk

The third level only fires when --dev is set, since auto-compilation is only safe in your local project. If dbt isn't on PATH, compilation fails, or the project root can't be found, you get a clear error with a suggested manual command.

Refreshing Artifacts

# Sync production manifest.json + catalog.json from remote storage
meta refresh
# โ†’ Downloads to ~/dbt-state/ (always --force)

# Parse local project to ./target/manifest.json (use after editing models)
meta refresh --dev
# โ†’ Runs: dbt parse --target dev

Optimization Analysis

# Find top optimization opportunities
meta hotspots --limit 10
# โ†’ Shows tables with highest optimization potential (scoring_details included)

# Deep analysis of specific model
meta analyze customers
# โ†’ Storage metrics, query costs, partition info, recommendations

# Analyze branch impact before merging
meta branch
# โ†’ Shows optimization impact of current branch changes

# JSON output for AI agents (includes recommendations)
meta hotspots -j | jq '.hotspots[0].scoring_details'
# โ†’ [{"criterion": "no_partition", "points": 60, "recommendation": "Add partition_by config"}]

Column-Level Lineage

# Build the lineage artifact (one-time, runs SQLGlot on every model's compiled SQL)
meta lineage build --verbose
# โ†’ Wrote ~/dbt-state/lineage.json (8.7 MB, 27,059 nodes, 23,815 edges in 470 s)

# Where does a column come from?
meta lineage column core_internal_tracking__sessions.session_channel
# โ†’ Direct: stg_traffic_channel__mapping.assigned_channel
# โ†’ Ancestors: source.raw_traffic_channel.traffic_channel_mapping.assigned_channel, โ€ฆ

# What breaks if I change this column?
meta lineage downstream stg_traffic_channel__mapping.assigned_channel
# โ†’ 30 direct downstream columns across core_client__*, core_plausible__*, โ€ฆ

# Artifact stats (size, age, warnings)
meta lineage stats
# โ†’ schema 1.0, 27,059 nodes, 23,815 edges, generated 2026-05-08T10:20:03+00:00

Refresh Planner โ€” Minimum --full-refresh Set

# Detect changed models from git, classify their downstream
cd ~/Projects/reports
meta optimize refresh -m
# โ†’ 12 full_refresh, 0 incremental, 99 skip (out of 110 transitive downstream)
# โ†’ Suggested: dbt run -fs core_internal_tracking__events core_plausible__pageviews โ€ฆ

# Column-precision (drastically narrows the set when you know the diff)
meta optimize refresh --cols core_internal_tracking__events:event_type
# โ†’ 1 full_refresh, 1 incremental (sessions uses event_type in WHERE), 103 skip

# Override base branch (auto-detects origin/main โ†’ origin/master โ†’ main โ†’ master)
meta optimize refresh -m --base origin/develop

# Programmatic use
meta optimize refresh -m -j | jq -r '.dbt_commands.full_refresh'
# โ†’ "dbt run -fs core_internal_tracking__events โ€ฆ"

Clustering & Partitioning Recommendations

# What columns should I cluster `core_sessions` on?
meta optimize cluster core_internal_tracking__sessions
# โ†’ match_utm_source (JOIN ร—30 in 6 models, score 168.44)
# โ†’ match_utm_medium (JOIN ร—30, score 168.44)
# โ†’ โ€ฆ (excluded: existing partition column, STRUCT/ARRAY/JSON columns)

# What's the right partition column for this model?
meta optimize partition amas_client_profiles
# โ†’ status (INT64) RANGE_BUCKET, pruning ~4.5%
# โ†’ Alternatives: client_id (INT64) score=1.5

# JSON for tooling
meta optimize cluster core_sessions -j | jq '.recommendations[].column'

Power BI Integration

# Extract table mappings from default workspace (from config)
meta powerbi
# โ†’ Shows datasets, reports, BigQuery tables, and dbt model mappings

# Table usage view: aggregated by BigQuery table with report/dataset counts
meta powerbi --by-table
# โ†’ Table | Reports | Datasets | dbt Model

# Include DAX measure expressions
meta powerbi --measures

# Include column schemas (name, data type, visibility)
meta powerbi --columns

# All metadata: tables + measures + columns
meta powerbi --full

# Specific workspace
meta powerbi 677db568-5923-4c5b-9b45-f14ec16a2b62

# JSON output for automation (always returns full metadata when -j is set)
meta powerbi -j | jq '.datasets[] | {name: .name, tables: .tables | length}'

# Table usage as JSON (includes report/dataset lists per table)
meta powerbi --by-table -j | jq '.tables[] | select(.in_manifest == false) | .bigquery_table'
# โ†’ BigQuery tables used in Power BI but not tracked in dbt

# Configuration (in ~/.config/dbt-meta/config.toml)
[powerbi]
enabled = true
tenant_id = "your-tenant-id"
client_id = "your-client-id"
client_secret = "your-secret"
workspaces = ["workspace-id-1", "workspace-id-2"]

Search and Discovery

# Simple substring search (old command)
meta models staging

# Advanced filtering with selectors
meta list tag:daily                           # Models with 'daily' tag
meta list config.materialized:incremental     # Incremental models
meta list path:models/core/                   # Models in specific folder

# Multiple selectors with OR logic (default)
meta list tag:daily tag:core
# โ†’ Returns models with 'daily' OR 'core' tag

# Multiple selectors with AND logic
meta list tag:daily tag:core --and
# โ†’ Returns models with both 'daily' AND 'core' tags

# Git-aware filtering
meta list -m                                  # Show modified models
# For chain-aware refresh planning: `meta optimize refresh -m`

# Group by tag combinations
meta list tag:daily tag:core --group
# โ†’ Groups results by tag combinations

# JSON output for scripting
meta list tag:daily -j | jq -r '.[].model'

# Search models by description (different from list)
meta search "customer dimension" -j | jq -r '.[] | .name'

# Get file path for editing
meta path customers
# โ†’ models/marts/customers.sql

โš™๏ธ Configuration

Priority: CLI flags > TOML config > Environment variables > Defaults

Simple Mode (Zero Configuration)

No configuration needed! Just run dbt compile and start using:

cd ~/my-dbt-project
dbt compile
meta schema customers  # โœ“ Works immediately with ./target/manifest.json

TOML Configuration (Recommended)

# Create config file with template
meta settings init

Edit ~/.config/dbt-meta/config.toml:

# Manifest paths
prod_manifest_path = "~/dbt-state/manifest.json"
dev_manifest_path = "./target/manifest.json"

# Dev environment
dev_schema = "personal_myname"

# Fallback behavior
fallback_dev_enabled = true      # Try dev manifest if model not in prod
fallback_bigquery_enabled = true # Query BigQuery if model not in manifests

# Production naming (optional)
prod_table_name_strategy = "alias_or_name"  # alias_or_name | name | alias
prod_schema_source = "config_or_model"      # config_or_model | model | config

Config file locations (priority order):

  1. ./.dbt-meta.toml - Project-local config
  2. ~/.config/dbt-meta/config.toml - User config (XDG standard)
  3. ~/.dbt-meta.toml - Fallback location

Settings commands:

meta settings show      # View current configuration
meta settings validate  # Check config file for errors
meta settings path      # Show active config file path

Environment Variables (Alternative)

All TOML settings can be set via environment variables. CLI flags > TOML > env vars > defaults.

Manifest & catalog:

TOML key Environment variable Default
prod_manifest_path DBT_PROD_MANIFEST_PATH ~/dbt-state/manifest.json
dev_manifest_path DBT_DEV_MANIFEST_PATH ./target/manifest.json
prod_catalog_path DBT_PROD_CATALOG_PATH ~/dbt-state/catalog.json
dev_catalog_path DBT_DEV_CATALOG_PATH ./target/catalog.json
dev_schema DBT_DEV_SCHEMA personal_{USER}

Fallback behavior:

TOML key Environment variable Default
fallback_dev_enabled DBT_FALLBACK_TARGET true
fallback_bigquery_enabled DBT_FALLBACK_BIGQUERY true
fallback_catalog_enabled DBT_FALLBACK_CATALOG true

Production naming strategy:

TOML key Environment variable Options
prod_table_name_strategy DBT_PROD_TABLE_NAME alias_or_name (default), name, alias
prod_schema_source DBT_PROD_SCHEMA_SOURCE config_or_model (default), model, config

Power BI integration (optional):

TOML key Environment variable
powerbi.enabled POWERBI_ENABLED
powerbi.tenant_id POWERBI_TENANT_ID
powerbi.client_id POWERBI_CLIENT_ID
powerbi.client_secret POWERBI_CLIENT_SECRET
powerbi.workspaces POWERBI_WORKSPACES (comma-separated)

๐Ÿงช Development

Running Tests

# Install with dev dependencies
pip install -e ".[dev]"

# Run all tests
pytest

# Run with coverage
pytest --cov=dbt_meta --cov-report=html

# Run specific test categories
pytest -m unit              # Unit tests only
pytest -m integration       # Integration tests only
pytest -m performance       # Performance benchmarks

# Run tests in parallel
pytest -n auto

Code Quality

# Type checking
mypy src/dbt_meta

# Linting
ruff check src/dbt_meta

# Formatting
ruff format src/dbt_meta

๐Ÿค Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

  1. Fork the repository
  2. Create your feature branch (git checkout -b feature/amazing-feature)
  3. Commit your changes (git commit -m 'Add amazing feature')
  4. Push to the branch (git push origin feature/amazing-feature)
  5. Open a Pull Request

Development Guidelines

  • Write tests for new features (maintain 90%+ coverage)
  • Follow type hints (mypy strict mode)
  • Use ruff for formatting and linting
  • Add docstrings for public APIs
  • Update README with new features

๐Ÿ“„ License

Copyright ยฉ 2025 Pavel Filianin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.


Built with โค๏ธ for the dbt community

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dbt_meta-0.3.1.tar.gz (305.8 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

dbt_meta-0.3.1-py3-none-any.whl (188.4 kB view details)

Uploaded Python 3

File details

Details for the file dbt_meta-0.3.1.tar.gz.

File metadata

  • Download URL: dbt_meta-0.3.1.tar.gz
  • Upload date:
  • Size: 305.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.8

File hashes

Hashes for dbt_meta-0.3.1.tar.gz
Algorithm Hash digest
SHA256 15b498136eed6207ac94ed4a8122b4cb0785b6a082f2e7d6dd86d99e2034c108
MD5 fc59d01becfd05a433c4b672751e20a8
BLAKE2b-256 59984505aa0279ba1eb6aa9791d714f779d379745d8587d85425358c411fd074

See more details on using hashes here.

File details

Details for the file dbt_meta-0.3.1-py3-none-any.whl.

File metadata

  • Download URL: dbt_meta-0.3.1-py3-none-any.whl
  • Upload date:
  • Size: 188.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.8

File hashes

Hashes for dbt_meta-0.3.1-py3-none-any.whl
Algorithm Hash digest
SHA256 93044cc605695c1235643f242c013a0543291a50aa21f203075a9e063de0622d
MD5 d16c9594659e8f674ab4eb7f7d80e310
BLAKE2b-256 696c866a6959e30e0b8a48445b6b8e7a5cec68aafe4a0d1cb08305fee4fd37bb

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page