dbt-pathfinder 0.2.0

CLI and library for exploring dbt manifests as graphs.

dbt-pathfinder

🔎 Explore, debug, and understand your dbt lineage — fast.

dbt-pathfinder is a Python CLI + library that parses your manifest.json and turns it into a navigable graph of your dbt project.

It helps you answer questions like:

• “What breaks if I change this model?”
• “Which nodes and tests are affected by these changed files?”
• “How are these two models connected?”
• “What joins actually exist along this path?”
• “Is this relationship 1:1 or 1:N?”

---

🚀 Why this exists

dbt gives you lineage… but not exploration.

When projects scale:

• lineage graphs get noisy
• impact analysis becomes manual
• understanding joins requires digging through SQL

dbt-pathfinder bridges that gap: 👉 fast CLI exploration 👉 precise impact analysis 👉 inferred join + cardinality insight

---

✨ Features

🔍 show

Inspect a node and its immediate relationships

dbt-pathfinder show --manifest target/manifest.json --model fct_orders

Outputs:

• node metadata
• upstream dependencies
• downstream children

---

💥 impact

See everything downstream of a model (grouped by depth)

dbt-pathfinder impact --manifest target/manifest.json --model stg_orders

Example output:

Depth 1:
- fct_orders
- dim_customers

Depth 2:
- mart_customer_ltv

Perfect for:

• change impact analysis
• safe refactoring
• CI/CD validation

---

🎯 ci-impact

From a list of changed project files (paths as in manifest original_file_path), see downstream blast radius, impacted tests, and a suggested dbt build --select command. Built for CI/CD (for example, pass files touched in a PR).

dbt-pathfinder ci-impact \
  --manifest target/manifest.json \
  --files models/staging/stg_orders.sql \
  --files models/marts/dim_customers.sql

Example output (text):

Changed files:
- models/staging/stg_orders.sql
- models/marts/dim_customers.sql

Changed nodes:
- stg_orders
- dim_customers

Impacted downstream:
Depth 1:
- fct_orders
- mart_customer_ltv

Depth 2:
- customer_health_dashboard

Impacted tests:
- unique_fct_orders_order_id
- not_null_dim_customers_customer_id

Suggested command:
dbt build --select stg_orders+ dim_customers+

Options:

• --output text (default) or --output json (JSON uses node unique_id values in impacted_by_depth; text uses short labels)
• --ui rich (default) or --ui text when using --output text (same pattern as impact and path-explain; JSON always prints plain text)
• --include-tests (default) or --no-include-tests to omit the impacted-tests section and skip computing it

Repeat --files once per path. File matching is against each node’s original_file_path (.sql and .yml).

---

🧭 path

Find the shortest path between two nodes

dbt-pathfinder path \
  --manifest target/manifest.json \
  --from stg_orders \
  --to mart_customer_ltv

Supports:

• --mode directed (default)
• --mode any (for debugging unexpected relationships)

---

🧠 path-explain (🔥 killer feature)

Explain how models are connected — not just that they are.

dbt-pathfinder path-explain \
  --manifest target/manifest.json \
  --from stg_orders \
  --to mart_customer_ltv

Includes:

• inferred join conditions (JOIN ... ON ...)

• join keys

• inferred cardinality:

  • 1:1
  • 1:N
  • N:1
  • N:N

• confidence level

⚠️ Note: inference is heuristic and depends on SQL structure and dbt tests.

---

📦 Installation

From PyPI (recommended)

pip install dbt-pathfinder

From source (dev)

git clone https://github.com/<your-username>/dbt-pathfinder.git
cd dbt-pathfinder
pip install -e ".[dev]"

---

⚡ Quick Start

1. Generate your dbt manifest:

dbt compile

2. Run:

dbt-pathfinder show --manifest target/manifest.json --model my_model

---

🧰 CLI Usage

dbt-pathfinder --help

Examples:

dbt-pathfinder show --manifest target/manifest.json --model fct_orders --verbose

dbt-pathfinder impact --manifest target/manifest.json --model stg_orders

dbt-pathfinder impact --manifest target/manifest.json --model stg_orders --output json

dbt-pathfinder ci-impact \
  --manifest target/manifest.json \
  --files models/staging/stg_orders.sql \
  --files models/marts/dim_customers.sql

dbt-pathfinder ci-impact --manifest target/manifest.json --files models/staging/stg_orders.sql --output json

dbt-pathfinder path --manifest target/manifest.json --from stg_orders --to mart_customer_ltv

dbt-pathfinder path-explain --manifest target/manifest.json --from stg_orders --to mart_customer_ltv --ui rich

---

🖥 Output Modes

• --ui rich (default): formatted terminal output (show, path, impact, path-explain, ci-impact when using text output)
• --ui text: plain text
• --output json: machine-readable (great for CI/CD; supported on impact, path-explain, and ci-impact)

---

🧱 Library Usage

You can also use dbt-pathfinder programmatically:

from dbt_pathfinder.services.pathfinder_service import PathfinderService
from dbt_pathfinder.services.ci_impact_service import CIImpactService

service = PathfinderService.from_manifest("target/manifest.json")

print(service.show("fct_orders"))
print(service.impact("stg_orders"))
print(service.path("stg_orders", "mart_customer_ltv", directed=True))
print(service.explain_path("stg_orders", "mart_customer_ltv", directed=True))

ci = CIImpactService.from_manifest("target/manifest.json")
result = ci.build_result(
    ["models/staging/stg_orders.sql"],
    include_tests=True,
)
print(result.model_dump())

---

🧠 How it works

• Parses dbt manifest.json

• Builds a directed graph:

  upstream → downstream
  

• Uses:

  • graph traversal for lineage + paths
  • SQL parsing heuristics for join inference
  • dbt tests (unique, not_null) for cardinality hints

---

⚠️ Limitations

• Join inference is best-effort (complex SQL/macros may reduce accuracy)
• Requires a valid dbt manifest.json
• Ambiguous model names may require full unique_id

---

🛠 Roadmap

• CI impact from changed files (ci-impact)
• ci-impact --git-diff (range-based changed files)
• Visualization (graph output / web UI)
• dbt Cloud integration
• Column-level lineage support
• Better SQL parsing for complex joins

---

🤝 Contributing

Contributions welcome!

1. Fork the repo
2. Create a feature branch
3. Add tests
4. Open a PR

---

⭐ Support

If this tool helps you:

• ⭐ Star the repo
• 🐛 Open issues
• 💡 Suggest features

---

📄 License

MIT License