kwin-mcp

An MCP (Model Context Protocol) server that controls native Wayland windows on KDE Plasma from an AI agent.

It does what cua-driver cannot on Linux/Wayland: see and drive the real desktop. cua-driver (trycua) only enumerates X11/XWayland clients, so on a KDE Wayland session it sees 1 of ~20 windows. kwin-mcp sees all of them.

It is built entirely on KDE-native primitives, so it needs no modifications to trycua's binary and no root daemon. You point your MCP client (Claude Code, Codex, Hermes, etc.) at server.py and get the same capabilities cua offers on X11: window listing, screenshots, clicks, typing, dragging, key presses, and (optionally) AT-SPI element targeting.

Install

kwin-mcp targets KDE Plasma on Wayland. Two paths, pick one:

Path A - Standalone CLI via pip (recommended for manual use)

You get the kwin-mcp command globally. No clone, no venv:

pipx install kwin-mcp-server        # or: uv tool install kwin-mcp-server
kwin-mcp --doctor                    # readiness report
kwin-mcp                             # stdio MCP server

Then wire it into any agent with one command (still no clone):

kwin-mcp setup hermes      # or: claude | codex | cursor | vscode | opencode | openclaw | antigravity | pi | zed | windsurf
kwin-mcp setup list        # show all supported agents
kwin-mcp setup check       # preflight only, no wiring
kwin-mcp setup verify      # preflight + confirm the real server starts and reports ready

Path B - Agent wiring without a pip install (curl / uvx)

If you have not installed the package, the agent can fetch it on first launch via uvx (like npx). No clone still:

curl -sSL https://raw.githubusercontent.com/Samthesurf/kwin-mcp/main/setup.sh | bash -s hermes
# or, if you have uv installed locally:
uvx --from git+https://github.com/Samthesurf/kwin-mcp kwin-mcp setup hermes --uvx

Both paths run a preflight first. If a system dep is missing it prints the exact install command and stops, so you never get a half-wired config.

Do not do both. Path A installs the kwin-mcp binary and wires it as command: kwin-mcp. Path B wires it as uvx --from git+... kwin-mcp. If you already did pipx install, just use kwin-mcp setup; no uvx or git clone is needed.

System deps (one-time, either path)

sudo pacman -S kdotool spectacle                 # Arch
sudo usermod -aG input "$USER"                   # allow /dev/uinput
# log out and back in so the new group applies

Not on Arch? See the Dependencies table below.

What it can do

Windows are identified by a stable KDE window UUID of the form {xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx} (exactly what kdotool prints).

Readiness report (doctor) and safety contract

doctor / kwin-mcp-doctor

Run kwin-mcp --doctor (or kwin-mcp-doctor) to get a single structured JSON document describing the desktop, the windowing backend (with a live window list probe), the input path, AT-SPI, the screenshot path, and XDG portal availability. It ends with a readiness summary carrying explicit blockers and a recommended_next_step, so an MCP host or a human can render one report instead of parsing prose:

kwin-mcp --doctor | jq .readiness

The same report is exposed as the doctor MCP tool.

MCP safety annotations

Since v0.2 every tool carries an MCP ToolAnnotations so hosts can warn before invoking a mutating tool:

Annotations are safety hints, not an authorization system. Treat any call that could submit, delete, send, or purchase as requiring user approval.

Computer History

A port of Cua Driver's encrypted, metadata-only Computer History preview (libs/cua-driver/docs/computer-history-*.md). It gives you a local, inspectable record of what kwin-mcp did, when, and which app it targeted, without turning the server into a screen recorder or keylogger.

Privacy boundary (permanent): history records only fixed-field metadata. It never stores screenshots, typed text, clipboard contents, raw tool arguments or results, accessibility trees, window titles, URLs, or file paths. Every event is a CloudEvents 1.0 envelope on urn:kwin-mcp:schema:history-event:v0.

Encrypted at rest: each event is sealed with AES-256-GCM before any bytes hit disk (no plaintext fallback). The key is a 256-bit in-memory secret; delete destroys the key and erases the store.

Opt-in, off by default. Nothing is recorded until you enable it. Agents can only read history (history_status, history_query); capture lifecycle, retention, and deletion are owned locally (mirroring Cua's history_control_requires_local_cli).

Enable it from the server process (e.g. via the history_control tool, or a local CLI), then let an agent query bounded slices:

{ "tool": "history_control", "arguments": { "operation": "enable" } }
{ "tool": "history_query",   "arguments": { "limit": 50, "since_sequence": 1 } }

Recorded events cover the 14 mutating/action tools (click, drag, type_text, paste, press_key, scroll, click_element, perform_action, set_value, focus_element, activate, raise_window, minimize, close_window) as action_started / action_completed envelopes, classified by effect (confirmed, partial, unverifiable, suspected_noop, refused, failed) and route (synthetic_events, trusted_input, global_input, accessibility, system_api).

Dependencies

System packages (must be installed on the machine)

These are the KDE/Wayland tools the server shells out to. Install with your distro's package manager.

On Arch this machine already had kdotool, spectacle, grim, ydotool, slurp, and busctl available.

Kernel / group requirements (input)

Synthetic input is sent through a virtual device on /dev/uinput. You must:

1. Be a member of the input group:

   groups | grep -w input || sudo usermod -aG input "$USER"
   # then log out and back in
   

2. Have write access to /dev/uinput (group input owns it: crw-rw---- root input). No root daemon (ydotoold) is required because python-uinput opens the device directly as a group member.

Verify with:

ls -l /dev/uinput          # should show group 'input' with rw
id -nG | tr ' ' '\n' | grep -x input   # should print 'input'

Python packages

python -m venv .venv
. .venv/bin/activate
pip install -r requirements.txt
# AT-SPI element/action/value targeting + semantic clicks work out of the box:
# kwin-mcp talks to AT-SPI directly over D-Bus via jeepney (already a
# dependency), so no pyatspi is required. On distros where the legacy pyatspi
# module happens to be installed, it is used as a fallback backend.

Installed and verified on this build: mcp 1.28.1, python-uinput 1.0.1, Pillow 12.3.0 (Python 3.14).

Running

. .venv/bin/activate

# dependency preflight (also run automatically by setup.sh)
python server.py --check

# JSON readiness report
python server.py --doctor

# stdio MCP server (for Claude/Codex/Hermes MCP clients)
python server.py

# or via the convenience wrapper
python run.py

# Streamable HTTP transport on 127.0.0.1:8080
python server.py --http 8080

The smoothest path is the one-command kwin-mcp setup described in ## Install — no local venv needed.

Wiring into an MCP client (what kwin-mcp setup does)

kwin-mcp setup <agent> preflights, then injects the correct entry into your agent's config (and never half-wires). The manual equivalent is to point the client at the installed command:

{
  "mcpServers": {
    "kwin-mcp": { "command": "kwin-mcp", "args": [] }
  }
}

If you used --uvx (no pip install), the wired entry is instead { "command": "uvx", "args": ["--from", "git+https://github.com/Samthesurf/kwin-mcp", "kwin-mcp"] }.

Supported agents: hermes (/.hermes/config.yaml), claude (/.claude.json), codex (/.codex/config.toml), cursor, vscode (settings.json), opencode (/.config/opencode/opencode.json), openclaw, antigravity, pi, zed (context_servers), windsurf. Run kwin-mcp setup list for the full table with paths. ./setup.sh is still supported as a legacy alias (./setup.sh hermes --uvx).

Manual examples:

• Claude Code: claude mcp add kwin-mcp -- kwin-mcp (pip path) or claude mcp add kwin-mcp -- uvx --from git+https://github.com/Samthesurf/kwin-mcp kwin-mcp (uvx path)

The server is self-sufficient about its environment: when an MCP client does not forward DBUS_SESSION_BUS_ADDRESS / WAYLAND_DISPLAY / DISPLAY / XDG_RUNTIME_DIR, the server discovers the correct session values from /run/user/<uid>/ so kdotool and spectacle always work.

No API keys, no network calls, no cloud. Everything runs locally against your compositor.

Running from a local checkout (alternative)

If you prefer a local venv instead of uvx:

python -m venv .venv && . .venv/bin/activate
pip install -r requirements.txt
python server.py            # stdio MCP server
python server.py --check   # dependency preflight

How it works (and the Wayland caveats)

On Wayland there is no X server between apps and the compositor, so input cannot be injected "into a specific window" the way cua-driver does on X11. The bridge follows a focus-then-inject model:

1. kdotool windowactivate <uuid> raises and focuses the target window.
2. The virtual pointer (a python-uinput device) is moved to the target coordinate. Because the compositor applies mouse acceleration and uinput only emits relative motion, movement is closed-loop: read the real cursor, emit a bounded delta, re-read, repeat until within ~3 px. This makes absolute positioning deterministic.
3. The click / key / drag is emitted on the now-focused window.

What this costs versus X11 (inherent to Wayland, not a bug):

• No background targeting. The window must be focused first; the real cursor moves. It is not invisible the way background X11 input can be.
• Single cursor. Parallel multi-pointer drags (cua's parallel_mouse_drag) are not available on Wayland.
• Secure-input surfaces (some password fields, the lock screen) may reject synthetic input.
• Small focus race. Between focusing and injecting there is a brief window where focus could shift; the code waits ~250 ms after activation.

Screenshots use spectacle in background/non-interactive mode. On KDE Wayland --background can occasionally race the compositor and capture the lock-screen splash instead of the live desktop; the capture path adds a settle delay and a variance-based validation that retries up to 3 times, so the returned frame is always the real desktop.

AT-SPI (get_window_state, click_element, perform_action, set_value, semantic clicks) works for GTK/Qt/KDE apps that expose an accessibility tree. It talks to AT-SPI directly over D-Bus (via jeepney, a pure-Python client), so it needs no pyatspi and works on Arch; the legacy pyatspi module is used only as a fallback if present. It degrades gracefully to coordinate input when no AT-SPI backend is available.

Project layout

kwin-mcp/
├── server.py              # MCP server (FastMCP) exposing all tools
├── run.py                 # convenience entry point
├── requirements.txt
├── pyproject.toml
├── mcp-config.example.json
├── README.md
└── kwin_bridge/
    ├── __init__.py
    ├── windows.py         # kdotool wrapper: enumerate/geometry/focus/close
    ├── screenshot.py      # spectacle wrapper + crop + retry/validate
    ├── input.py           # /dev/uinput virtual pointer+keyboard, closed-loop move
    ├── a11y.py            # AT-SPI front-end (semantic resolve / action / value)
    ├── atspi_dbus.py      # pure-D-Bus AT-SPI backend (jeepney, no pyatspi)
    ├── doctor.py          # structured JSON readiness report
    └── preflight.py       # actionable dependency check

Testing

A quick smoke test against the live desktop:

. .venv/bin/activate
python - <<'PY'
from kwin_bridge import windows, screenshot, input as inp
ws = windows.list_windows()
print("windows:", len(ws))
wid = ws[0].window_id
print("capturing", wid)
p = screenshot.capture_window(wid, "/tmp/test.png")
print("shot:", p)
inp.click_window(wid, 100, 100)
inp.type_text("hello from kwin-mcp")
PY

License

MIT. Use it, fork it, ship it.