hypruse 0.1.0

Computer use for Hyprland — an MCP server that gives AI agents native hands on your Wayland desktop

hypruse

Computer use for Hyprland. An MCP server that gives AI agents native hands on your Wayland desktop — workspaces, windows, mouse, keyboard, screenshots.

No ydotool daemon. No root. No portals. No X11.

Why

Computer use exists on macOS and Windows. On Linux there is effectively nothing: the Claude Desktop Linux beta explicitly ships without screen control, Anthropic's reference implementation is an X11 container, and the existing Wayland attempts lean on setuid uinput hacks or GNOME-only portals.

Meanwhile Hyprland already exposes everything an agent needs, better than any accessibility bridge: a complete IPC surface for state and window management, and first-class Wayland protocols for input. hypruse just wires them to MCP:

• Semantic first. desktop returns the real window/workspace tree — addresses, classes, titles, geometry — in one call. The agent switches workspaces and focuses windows the way you do (instantly, over IPC), not by squinting at pixels.
• Vision when it matters. Screenshots of a monitor, an exact window crop, or a zoomed region, with the geometry/scale metadata to map any pixel back to a clickable coordinate.
• Native input. Clicks and scrolls are spoken directly over the Wayland wire (zwlr_virtual_pointer_v1); typing goes through wtype's virtual keyboard with a proper XKB keymap — unicode-safe on any layout.

How it works

agent (Claude Code, or any MCP client)
   │ stdio
   ▼
hypruse
   ├── hyprctl -j ········▶ desktop state: monitors, workspaces, windows
   ├── hyprctl dispatch ··▶ focus / move / close / launch / movecursor
   ├── grim ··············▶ screenshots: monitor, window crop, region
   ├── wtype ·············▶ keyboard (zwp_virtual_keyboard_v1, real XKB keymap)
   └── raw Wayland wire ··▶ click & scroll (zwlr_virtual_pointer_v1)

Design choices, defended:

• No ydotool / uinput. That path needs a daemon, udev rules or root, and types US scancodes that break on other layouts. hypruse is just another Wayland client of your compositor — same standing as wlrctl.
• No portals. xdg-desktop-portal-hyprland does not implement the RemoteDesktop portal (InputCapture is capture, not injection), so anything built on libei/portals silently degrades on Hyprland. hypruse doesn't try.
• Cursor positioning via hyprctl dispatch movecursor (global logical coordinates, exact on any monitor layout), with only button/axis events on the virtual pointer — sidestepping the known multi-monitor bugs of absolute virtual-pointer motion (hyprwm/Hyprland#6749).

Tools

tool	what it does
desktop	One-call semantic snapshot: monitors, workspaces, windows (address/class/title/geometry), active window, cursor
screenshot	Focused monitor, exact window crop by address, or x,y,WxH region — returns image + coordinate-mapping metadata
pointer	move / click / drag / scroll in global coordinates
keyboard	Type literal text (unicode-safe) or press combos: ctrl+shift+t, super+enter, F5
hypr	Switch workspace, focus/move/close windows, fullscreen, floating — pure IPC, milliseconds
launch	Start an app (optionally silent on another workspace), wait for its window, return its address — detects single-instance apps (browsers) whose window ignores exec rules, and moves it to the requested workspace

Install

Requirements: Hyprland, grim, wtype (most Hyprland setups already have both), and uv.

Claude Code:

claude mcp add -s user hypruse -- uvx hypruse

From a source checkout:

claude mcp add -s user hypruse -- uv run --directory /path/to/hypruse hypruse

Any other MCP client: run uvx hypruse as a stdio server.

Claude Desktop (Linux beta)

The Linux beta ships without Anthropic's first-party computer use — but stdio MCP servers work in chat, which makes hypruse the workaround. In ~/.config/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "hypruse": {
      "command": "uvx",
      "args": ["hypruse"],
      "env": { "HYPRUSE_SCREENSHOT_MODE": "image" }
    }
  }
}

Two Desktop-specific notes: use image mode (Desktop renders inline MCP images and has no file-read tool), and the app must run natively inside your Hyprland session so the server inherits WAYLAND_DISPLAY/HYPRLAND_INSTANCE_SIGNATURE — from a VM or container it cannot reach your compositor. If your Desktop install bypasses tool-approval prompts, treat the Waybar indicator + panic keybind as mandatory, not optional.

Security model

Read this section before installing. hypruse hands an agent your mouse, your keyboard, your screen contents, and an app launcher. The layers that keep that sane:

1. Approval — MCP clients gate tool calls. In Claude Code, allowlist the read-only tools (desktop, screenshot) and leave pointer/keyboard/hypr/launch on ask-first until you trust a workflow.
2. Visibility — the server maintains an activity beacon ($XDG_RUNTIME_DIR/hypruse/state.json); the shipped Waybar module is invisible when idle and shows 󰚩 while an agent has hands on your desktop.
3. Interruption — click the indicator, or bind a panic key: bind = SUPER SHIFT, BackSpace, exec, pkill -f hypruse. Killing it mid-action is safe: button press/release pairs never span tool calls, so it cannot die holding a button.
4. The seat is shared. There is one cursor and one keyboard focus, and Hyprland's focus-follows-mouse means a cursor move alone can retarget keystrokes. Don't type while an agent is driving — watch the indicator.
5. Scope — stdio only (no network listener), no clipboard access, nothing persisted except the beacon. A screenshot sees everything visible: treat an agent session like screen sharing.

Performance

Measured on a live session (Hyprland 0.55, 1080p, 20 windows): desktop ~30 ms, workspace/window dispatch ~10–20 ms, screenshots ~0.5 s. If tool calls feel slow, it is almost certainly the MCP approval prompt in front of each call, not the server — allowlist the tools you trust and the latency disappears. Claude Code (.claude/settings.json):

{
  "permissions": {
    "allow": [
      "mcp__hypruse__desktop",
      "mcp__hypruse__screenshot",
      "mcp__hypruse__hypr"
      // add pointer/keyboard/launch once you trust your workflows
    ]
  }
}

Coordinates

Everything speaks Hyprland's global logical coordinates — the space hyprctl cursorpos and window at use. Screenshots are pixel-space; each capture returns geometry and scale so global = origin + pixel / scale. On scale 1.0 monitors (most setups) image pixels are global coordinates.

In image mode, captures automatically fit the host's result-size limit (Claude Desktop caps tool results at 1 MB): format degrades before resolution — native PNG, then full-res JPEG, then stepped downscale — because full-res JPEG reads UI text better than half-res PNG. The applied scale is folded into the returned metadata, so coordinate mapping stays exact; tune with HYPRUSE_MAX_IMAGE_BYTES, or pass scale for a deliberate zoom-out.

By default the screenshot tool writes a PNG under $XDG_RUNTIME_DIR/hypruse/ and returns its path — MCP hosts with a file reader (Claude Code's Read) render it natively. This default exists because some hosts (Claude Code ≤ 2.1.x among them) serialize inline MCP image blocks to base64 text the model can't see; we verified this empirically rather than trusting the spec. HYPRUSE_SCREENSHOT_MODE=image switches to inline image content blocks for hosts that render them correctly.

Development

uv sync --group dev
uv run pytest            # unit tests, no compositor needed
uv run pytest -m e2e --override-ini addopts=   # live seat-safe checks
uv run python scripts/e2e_input.py             # supervised: takes the seat ~10s

The input e2e is deliberately manual — it borrows your cursor and keyboard, counts down, proves click/scroll/type delivery by reading the target terminal's screen back over kitty remote control, and restores your focus.

Roadmap

Now — distribution

• PyPI (uvx hypruse) and AUR (hypruse, hypruse-git)
• Demo GIF, then awesome-mcp-servers / awesome-hyprland listings

Next — trust & accuracy

• hypruse doctor — first-run diagnostics (deps, session reachability, virtual-pointer handshake)
• Click-by-text via OCR (Tesseract) — click labels, not estimated pixels; works on any app
• Read-only mode — disable input tools for a safe first run
• Headless-Hyprland CI — real end-to-end tests in GitHub Actions

Then — breadth & depth

• sway / niri support — the wire client already speaks the wlr protocols; needs an IPC layer alongside hyprctl.py (PRs very welcome, help wanted)
• AT-SPI element tree — click by accessible name, read GTK/Qt UIs without vision
• Clipboard (wl-clipboard), settle / wait-for-stable, discrete-axis scroll
• Multi-monitor and fractional-scaling hardening

Alternatives, honestly

project	approach	where it falls short on Hyprland
computer-use-linux	AT-SPI + portals, ydotool fallback	GNOME-first; RemoteDesktop portal is dead on Hyprland, screenshots portal-first
hyprmcp	hyprctl wrapper	no screenshots, no input
wayland-mcp (×2)	setuid evemu + bundled VLM	archived / stale; evemu needs elevated setup
Anthropic computer-use-demo	X11 + xdotool in Docker	a sandboxed demo, not your desktop

License

MIT