UI Bridge MCP

MCP (Model Context Protocol) server for UI Bridge - enables AI to inspect and interact with UI elements.

Overview

This MCP server provides tools for:

• Inspecting UI elements - Get element positions, bounds, visibility, and state
• Interacting with elements - Click, type, focus, hover
• Two modes of operation:
  • Control mode: Interact with the qontinui-runner's own Tauri webview
  • External mode: Interact with external browser tabs via Chrome extension

Installation

# Using pip
pip install ui-bridge-mcp

# Using poetry
poetry add ui-bridge-mcp

Prerequisites

The MCP server requires the qontinui-runner to be running on port 9876.

For external browser tab access, you also need the Chrome extension connected.

Configuration

Claude Desktop

Add to your claude_desktop_config.json:

{
  "mcpServers": {
    "ui-bridge": {
      "command": "ui-bridge-mcp"
    }
  }
}

Claude Code

Add to your MCP settings:

{
  "mcpServers": {
    "ui-bridge": {
      "command": "ui-bridge-mcp"
    }
  }
}

Environment Variables

Available Tools

Health Check

Control Mode (Runner's Own UI)

External Mode (Browser Tabs)

Usage Examples

Inspect Runner UI

AI: Let me check what's on the runner's Settings page.

1. First, get a snapshot of the UI:
   ui_snapshot

2. Click the Settings button:
   ui_click element_id="sidebar-nav-item-settings"

3. Get another snapshot to see the new page:
   ui_snapshot

Inspect External Website

AI: Let me check the login form on localhost:3001.

1. Check extension is connected:
   extension_status

2. List browser tabs:
   extension_list_tabs

3. Select the tab with localhost:3001:
   extension_select_tab tab_id=123456

4. Get page elements:
   extension_get_elements

5. Type into the email field:
   extension_type selector="input[name='email']" text="test@example.com"

Element IDs

Elements in the runner's UI have data-ui-id attributes that follow patterns like:

• sidebar-nav-item-{name} - Sidebar navigation items
• button-{action} - Action buttons
• input-{field} - Input fields
• dialog-{name} - Dialog components

Use ui_snapshot to discover all available element IDs.

Architecture

Claude/AI
    ↓ MCP Protocol
ui-bridge-mcp (this server)
    ↓ HTTP
qontinui-runner (port 9876)
    ├── /ui-bridge/control/* → Runner's Tauri webview
    └── /extension/* → Chrome extension → External browser tabs

Development

# Install dependencies
poetry install

# Run linting
poetry run ruff check .
poetry run mypy src

# Format code
poetry run black src
poetry run isort src

License

MIT