bruin-visualizer 0.1.0

Free, web-based visualizer for Bruin CLI pipelines with impact analysis and run history tracking

Bruin Visualizer

Free, web-based visualizer for Bruin CLI pipelines with impact analysis and run history tracking

What This Does

A standalone web-based visualizer for Bruin data pipelines. Visualize your pipeline DAG, analyze asset criticality, and track run history - all locally without requiring Bruin Cloud.

Features

• Interactive DAG Visualization - D3.js force-directed graph with zoom, pan, and drag
• Impact Analysis - Automated criticality scoring (0-10 scale) with downstream dependency tracking
• Run History Tracking - Track asset performance over time with SQLite storage
• Performance Trends - Sparkline charts showing duration trends, failure patterns, and alerts
• Asset Metadata - View descriptions, owners, tags, column definitions, and materialization info
• Search - Multi-field search with keyboard shortcuts (Ctrl+K)

Installation

pip install bruin-visualizer

For DuckDB row count tracking (optional):

pip install bruin-visualizer[duckdb]

Quick Start

1. Run your pipeline with tracking

bruin-viz run ./pipeline --start-date 2019-01-01 --end-date 2019-01-31

This runs your pipeline and records execution history in bruin_history.db.

2. Generate the pipeline graph

bruin-viz parse ./pipeline

This generates pipeline_graph.json with your pipeline structure and impact analysis.

3. View the visualization

bruin-viz serve

Opens the visualizer at http://localhost:8001

CLI Commands

bruin-viz parse

Parse a Bruin pipeline and generate visualization data:

bruin-viz parse ./pipeline
bruin-viz parse ./pipeline -o custom_output.json

bruin-viz serve

Start the visualization web server:

bruin-viz serve
bruin-viz serve --port 8080
bruin-viz serve --no-browser  # Don't auto-open browser

bruin-viz run

Run a Bruin pipeline with history tracking:

bruin-viz run ./pipeline --start-date 2024-01-01 --end-date 2024-01-31
bruin-viz run ./pipeline --full-refresh --workers 4
bruin-viz run ./pipeline --db custom_history.db

bruin-viz init

Initialize bruin-visualizer in current directory:

bruin-viz init
bruin-viz init -o ./visualizer

Use Cases

Pre-Deployment Impact Analysis

"I need to change stg_yellow_tripdata. What breaks?"

1. Click the asset in the visualizer
2. See: HIGH impact (9.6/10), affects 8 assets, 2 production reports
3. Export impact report
4. Share with team

Performance Monitoring

"Is this asset getting slower over time?"

1. Click the asset
2. View sparkline chart showing duration trends
3. See alerts if 40% slower than baseline
4. Review recent run history

Finding Assets Quickly

"Where's the monthly revenue report?"

1. Press Ctrl+K
2. Type "monthly revenue"
3. Click result
4. See full details + impact

Using as a Library

from bruin_visualizer import BruinParser, BruinRunHistory

# Parse pipeline
parser = BruinParser("./pipeline")
graph = parser.build_full_pipeline_graph()

# Calculate impact
impact = parser.calculate_impact_analysis(graph, "staging.stg_yellow_tripdata")
print(f"Criticality: {impact['criticality_score']}/10")
print(f"Affects {len(impact['total_downstream'])} downstream assets")

# Track runs
history = BruinRunHistory("bruin_history.db")
runs = history.get_recent_runs("my-pipeline", limit=10)

Requirements

• Python 3.7+
• Bruin CLI installed and in PATH
• PyYAML (automatically installed)
• DuckDB (optional, for row count tracking)

Technology Stack

• Frontend: D3.js (force-directed graph), vanilla JavaScript
• Backend: Python http.server (standard library)
• Database: SQLite (via Python sqlite3)
• Dependencies: PyYAML

Project Structure

After installation, you can use bruin-visualizer from anywhere:

your-project/
├── pipeline/              # Your Bruin pipeline
├── pipeline_graph.json    # Generated by 'bruin-viz parse'
└── bruin_history.db       # Generated by 'bruin-viz run'

Contributing

Contributions welcome! See GitHub repository for details.

License

MIT License - feel free to use, modify, and distribute.

Acknowledgments

• Built for Bruin data pipelines
• Visualization powered by D3.js