PyQt Runtime MCP

Runtime inspection and interaction for PyQt5 and PyQt6 QWidget applications, in the same spirit as browser DevTools / Playwright — but Qt-native.

Cursor talks to an MCP stdio server. That server talks over localhost TCP to a small bridge running inside the target app. All QWidget work happens on the Qt GUI thread.

Installation

pip install pyqt-runtime-mcp

From this folder (development):

pip install -e .

The in-app bridge uses the host's PyQt5 or PyQt6 binding. The MCP process pulls in the mcp SDK and does not import or require Qt. Optional extras: pip install "pyqt-runtime-mcp[qt]" for PyQt5 or pip install "pyqt-runtime-mcp[qt6]" for PyQt6.

The bridge detects an already imported host binding. For standalone examples when both are installed, set PYQT_MCP_QT_API=PyQt5 or PyQt6. Loading both bindings into one process is rejected. Import the host binding before importing PyQtMCPBridge directly; the install_pyqt_mcp function can be imported earlier.

Integrate the in-app bridge as shown below, then configure the pyqt-runtime MCP server in your editor or agent client.

Quick start

1. Run a bridge-enabled PyQt application, or start the included demo with python examples/demo_app.py. The in-app bridge listens on 127.0.0.1:8765.

2. Point Cursor at the MCP server (see Cursor configuration). Reload MCP servers if you just added it.

3. Ask:

Inspect the current PyQt application and give me its widget hierarchy.

Run the included demo from this folder with PyQt5 installed:

python examples/demo_app.py

Integration into an existing PyQt application

from PyQt5.QtWidgets import QApplication
from pyqt_runtime_mcp import install_pyqt_mcp

app = QApplication(sys.argv)
window = MainWindow()

install_pyqt_mcp(app)

window.show()
sys.exit(app.exec_())

Equivalent:

from pyqt_runtime_mcp import PyQtMCPBridge

bridge = PyQtMCPBridge(app)
bridge.start()

The bridge must be started inside the process that owns QApplication. It will not attach to a random already-running PyQt process.

For PyQt6, use from PyQt6.QtWidgets import QApplication and app.exec() in the example. No other integration change is required.

Starting the MCP server

After pip install pyqt-runtime-mcp, either command works:

pyqt-runtime-mcp
python -m pyqt_runtime_mcp

From this source tree without installing, use python run_server.py.

If the app is not running, tools return APPLICATION_NOT_CONNECTED instead of hanging.

Cursor configuration

After a PyPI / editable install, use the console script (same Python that ran pip install):

"pyqt-runtime": {
  "command": "pyqt-runtime-mcp"
}

Equivalent:

"pyqt-runtime": {
  "command": "python",
  "args": ["-m", "pyqt_runtime_mcp"]
}

For source-tree development without installation, configure the MCP command as python run_server.py. Reload MCP servers after saving the configuration, start the PyQt application, then ask the agent to inspect it.

Available MCP tools

Inspection: qt_get_application_info, qt_list_windows, qt_get_widget_tree, qt_find_widgets, qt_locate, qt_get_widget, qt_get_layout, qt_get_geometry, qt_get_ui_snapshot, qt_get_ui_changes

Visual: qt_capture_window, qt_capture_widget, qt_capture_region, qt_capture_screen_region, qt_capture_sections, qt_visual_snapshot

Interaction: qt_click, qt_double_click, qt_move_mouse, qt_set_focus, qt_type_text, qt_key_press

Semantic: qt_set_text, qt_set_value, qt_set_checked, qt_select_combobox, qt_set_current_page

Automation: qt_wait_for, qt_perform, qt_batch

Models: qt_query_model, qt_select_model_item, qt_expand_model_item, qt_scroll_to_model_item, qt_set_model_item

Actions / menus: qt_list_actions, qt_trigger_action, qt_open_menu

Window: qt_resize_window, qt_move_window, qt_activate_window, qt_close_window, qt_close_windows (confirm=true required on both close tools)

Style / meta: qt_get_stylesheet, qt_set_stylesheet, qt_list_properties, qt_get_property, qt_set_property

Diagnostics: qt_analyze_layout, qt_list_signals, qt_watch_signal, qt_get_event_log, qt_get_logs, qt_get_exceptions, qt_show_debug_overlay, qt_hide_debug_overlay

There is no execute_python / eval / shell tool.

Runtime logs

qt_get_logs returns one bounded ring buffer holding both Qt messages (captured through qInstallMessageHandler) and the application's own stdlib logging records (captured with a handler added to the root logger). Entries are tagged source: "qt" | "python"; filter with source, level, or contains. Neither capture changes what the application already prints, and the root logger's level is left untouched — records the app filters out stay filtered out.

Selecting a widget

Every tool that takes widget_id accepts a runtime id (qt://widget/N) or an objectName. Detaching a view usually clones objectNames into a second window, which makes a bare name ambiguous; the error then lists each candidate with the window it lives in. Pass window=<window id or objectName> to scope the lookup to one window's subtree. qt_find_widgets takes the same window argument.

The automation, model, and interaction tools also accept semantic locator objects. Useful fields include object_name, class_name, text, text_contains, accessible_name, path, window, root, visible, enabled, and nth. For example:

{
  "locator": {
    "window": "MainWindow",
    "class_name": "QPushButton",
    "text": "Save",
    "visible": true
  }
}

qt_wait_for waits for exists, destroyed, visible, hidden, enabled, disabled, focused, text, or a Qt property. qt_batch runs up to 50 allowlisted qt_perform steps in one bridge round trip and can return compact per-step results.

Tool profiles

Every registered MCP tool schema consumes context even when unused. Set PYQT_MCP_TOOLSET to choose a smaller surface:

• core — application/window inspection, widget discovery, semantic automation, waits, and batches.
• automation — core plus screenshots, legacy interaction tools, model/view tools, and QAction tools.
• diagnostics or full — all tools, including runtime styles, properties, logs, signals, overlays, and layout diagnostics. This is the default for compatibility.

Example Cursor configuration:

"pyqt-runtime": {
  "command": "pyqt-runtime-mcp",
  "env": {"PYQT_MCP_TOOLSET": "core"}
}

Closing windows

qt_close_window closes one window and verifies the result: closed is false when a closeEvent handler refused, and force=true then hides and deletes the widget. Dialogs are rejected first so an exec_() loop unwinds. qt_close_windows sweeps every visible top-level window — the way to clean up detached views, popups, and leftover dialogs after a test run. It protects QMainWindow instances unless include_main=true, and can be narrowed with windows, class_name, or title_contains. The reply splits results into closed, failed, skipped, and the remaining visible windows.

Example prompts

Inspect the current PyQt application and give me its widget hierarchy.

Take a screenshot of MainWindow.

Find the widget named telemetryPanel and inspect its geometry.

Resize MainWindow to 1024x600 and identify layout problems.

Find all QLabel widgets whose contents are clipped.

Open page 3 of the main QStackedWidget.

Click the Settings button.

Capture the Settings page.

Capture labeled sections for the status bar, emergency rail, and map area so each can be checked separately.

Inspect the layout and tell me why the bottom controls are outside the visible area.

Section screenshots

qt_capture_region grabs a rectangle in widget-local coordinates (default base: active window). Prefer it over qt_capture_screen_region when correlating to layout geometry from qt_get_geometry / qt_visual_snapshot.

qt_capture_sections takes a list of labeled pieces in one call. Each section may be a full widget (widget_id), a crop of that widget (widget_id + x/y/width/height), or a region on a window. Replies include every PNG plus metadata with label, source (widget | widget_region | region), and paths under sections/<label>.png in the screenshot sandbox — so agents can cite specific crops when verifying UI.

Agent workflow (token-aware, normal for this app)

The in-app bridge maintains a live searchable index without sending a widget tree to the model. Building and refreshing that index uses application CPU, not model tokens.

1. Targeted discovery — start with qt_find_widgets using an object name, class, visible text, or accessibility field. No initial full-tree transfer is required.
2. Follow-up inspection — use the returned runtime ID, or a stable unique object name directly, with qt_get_widget / qt_get_geometry / qt_get_layout and interaction tools.
3. Hierarchy fallback — use qt_get_widget_tree only when relationships are genuinely needed. Prefer visible_only=true or a specific root panel to keep the response compact.
4. Visual checks — prefer qt_capture_widget, qt_capture_region, or labeled qt_capture_sections over full-window qt_capture_window / qt_visual_snapshot. Use a full-window shot when you need overall composition, not for every verification.
5. Loop — targeted find/get → edit source → restart app → targeted find/get + crops → verify. A restarted app builds a fresh index and new runtime IDs automatically.

Disable the pyqt-runtime MCP in Cursor when you are not doing live UI work (tool schemas still cost context even if unused).

Cached navigation and popup effects

The cache is application independent. qt_catalog_scan indexes unique interactive destinations, including hidden controls, without clicking them. It reports unnamed and ambiguous controls; it does not discover every route or exercise every feature. Use targeted discovery to teach routes, then reuse them with qt_goto.

qt_find_widgets, qt_locate, and qt_get_widget return compact identity/state by default. Pass compact=false for the previous detailed response. The direct bridge protocol retains its legacy defaults. Runtime IDs are useful within one process but are rejected in persistent recipes.

Save this example with qt_catalog_put(name="settings", entry=...), adapting the names to the application:

{
  "steps": [
    {
      "action": "click",
      "locator": {"object_name": "settingsButton", "class_name": "QPushButton", "window": "MainWindow"},
      "after": {"locator": {"object_name": "settingsPage", "window": "MainWindow"}, "state": "visible", "timeout_ms": 2000}
    }
  ],
  "arrival": {"locator": {"object_name": "settingsPage", "window": "MainWindow"}, "state": "active_page"},
  "fingerprint": {"locator": {"object_name": "settingsPage", "window": "MainWindow"}}
}

Use visible instead of active_page for an ordinary panel. Route steps support click, trigger_action, open_menu, set_current_page, select_model_item, and wait_for, with optional before, after, and skip_if conditions. Page selection uses a stable page locator or title. Model selection uses item_text, resolved uniquely against the live model each time; row indexes are never persisted. Window scopes can be object names or descriptors such as {"class_name":"SettingsDialog","title":"Settings"}. Accessible names and button text provide fallbacks for unnamed controls; ambiguous matches require refinement.

qt_path_check only checks what exists now. Lazy controls may be unresolved until an earlier step creates them. qt_goto resolves each step when needed, checks actionability and modal blockers, and verifies arrival separately. Its first successful execution learns the optional destination fingerprint. Later mismatches return ok:false with progress and mark that route for repair. Use qt_catalog_put(..., repair_from=N) with replacement suffix steps and arrival conditions to preserve a known prefix. No failed action is automatically retried.

The route hash covers the recipe, conditions, and fingerprint scope. The arrival hash covers sorted visible interactive descriptors (names/classes, with accessible or button-text fallbacks). Geometry, parent chains, telemetry labels, edit values, scrollbars, and transient model editors are excluded. These hashes detect structural changes; they do not establish functional correctness. Use explicit assertions and screenshots where the QA task requires them.

qt_learn_effect explicitly executes an opener and records possible popup outcomes:

{
  "name": "settings/help",
  "opener": {"action": "click", "locator": {"object_name": "helpButton", "window": "MainWindow"}},
  "expected_popup": {"class_name": "QMessageBox", "title": "Help", "dismiss": "ok_button", "timeout_ms": 1500}
}

The observer starts before the opener, detects new and reused hidden-to-visible windows, and handles nested QDialog.exec() / exec_() loops. Dismissal is explicit: reject, accept, or ok_button. Omit it to observe only. Use qt_use_effect to reuse the saved opener and its possible class/title/kind outcomes. New variants are reported and appended without first-time dismissal. Unknown/ambiguous popups and failed dismissal stop a batch. Matching a concurrent unrelated popup is possible; use specific class/title matchers and avoid unrelated UI activity during a probe. Native file pickers may suppress Qt observation: kind="system" provides a bounded system_unobserved result, which is not success. An ordinary no-popup observation can be allowed explicitly with allow_none=true.

Navigate and reuse an opener in one qt_batch call:

{"steps":[
  {"action":"follow_route","name":"settings"},
  {"action":"use_effect","name":"settings/help","dismiss":"ok_button"}
]}

Catalogs are bounded transactional SQLite files at <tempfile.gettempdir()>/pyqt-runtime-mcp/catalogs/<app-hash>.sqlite3: normally %TEMP% on Windows and /tmp on Linux, respecting Python's platform temp selection. Discovery files, screenshots, and QA helper output also default to the platform temp directory. OS temp cleanup can remove them; the next session can relearn. Corrupt catalogs report CACHE_ERROR without silently overwriting data.

Namespaces include entrypoint (or explicit application_id), application and organization names, interpreter path, and Qt binding. This separates applications even when their application names are identical. For a stable identity/schema:

install_pyqt_mcp(app, application_id="my-app", schema_version="2")

Alternatively set the application's ui_schema_version property; application version is the fallback. A schema change removes routes/effects and retains destination candidates for fresh resolution. Caches contain compact locators, recipes and observed outcomes, never full widget trees or runtime IDs.

Security

• Binds to 127.0.0.1 only.
• Screenshots write only inside a sandbox directory (default: temp pyqt-runtime-mcp/screenshots/). .. and paths outside the sandbox are rejected.
• Widget IDs are validated. Qt property writes use writable QMetaProperty only.
• No eval, exec, shell, or call-by-name Python methods.

Threading model

MCP stdio process  --TCP JSON-->  bridge acceptor thread
                                      |
                                      | queued Qt signal
                                      v
                                 QApplication thread
                                      |
                                      v
                                 QWidget / QLayout

The IPC thread never touches Qt objects. The GUI thread never waits on the socket.

Architecture

Two processes, JSON only (no pickle):

1. In-app bridge (install_pyqt_mcp) — widget registry (qt://widget/N), inspectors, screenshots, input synthesis.
2. MCP server (pyqt-runtime-mcp / python -m pyqt_runtime_mcp) — official MCP Python SDK (MCPServer / FastMCP fallback), stdio to Cursor.

Wire protocol

TCP on 127.0.0.1, framed as a 4-byte big-endian length followed by UTF-8 JSON (max 16 MiB per frame). A discovery file (%TEMP%/pyqt-runtime-mcp/bridge.json) records the bound port if 8765 is busy.

1. Handshake — client sends {"type": "handshake", "protocol": "pyqt-mcp/1", "token": <optional>}; the bridge replies {"ok": true, "protocol": ..., "port": ...} or {"ok": false, "error": {...}} and closes.
2. Request — {"id": <uuid>, "method": <name>, "params": {...}, "timeout": <seconds>}. timeout is the client's budget; the bridge dispatches to the GUI thread with a slightly shorter deadline so it always answers first.
3. Response — {"id": <same uuid>, "ok": true, "result": ...} or {"id": ..., "ok": false, "error": {"code": ..., "message": ...}}.

Connections stay open and are never closed on idle. Requests are matched by id, so a reply to an abandoned request is skipped rather than mistaken for the current one.

Framing rules the client depends on:

• One exchange at a time per connection. BridgeClient holds a lock, because the MCP SDK runs sync tools on a thread pool and two writers on one socket would corrupt the stream.
• A frame length of 0 or above the limit means the stream is out of sync (PROTOCOL_ERROR); both sides discard the connection instead of trying to resynchronise.
• A reused connection is liveness-checked before sending, and a send failure on it is retried once on a fresh socket. A failure after sending is never retried, since the request may have already run.

Troubleshooting

Symptom	What to check
APPLICATION_NOT_CONNECTED	The PyQt app is not running. Start python main.py (or the demo).
WIDGET_NOT_FOUND	Stale id after destroy. Widgets are rebuilt when a view is detached, so re-query the id.
objectName is not unique	The same name exists in more than one window. Pass window=..., or use a runtime id from the error's candidates.
TIMEOUT	The GUI thread is busy or blocked (modal dialog, long handler). The connection is dropped and the next call reconnects.
PROTOCOL_ERROR (frame too large)	A reply exceeded 16 MiB, or something else is writing to the bridge port. Narrow the request (max_depth, a specific widget_id).
Screenshots empty / tiny	Offscreen platform (QT_QPA_PLATFORM=offscreen) still produces pixmaps but they may look blank. Use a real display for visual QA.
Resize ignored	The reply reports matched: false plus the window's min/max size. Maximized windows are restored first and the resize is re-applied once the window manager settles.
Window will not close	qt_close_window reports closed: false when closeEvent calls ignore(). Retry with force=true.
MCP server import error	pip install pyqt-runtime-mcp in the same Python Cursor uses for command.

Tests

From this folder:

pip install -e ".[dev]"
python -m pytest -q

(The test suite sets QT_QPA_PLATFORM=offscreen itself.)