Remote CUA Bridge

Control a remote Mac's desktop from any MCP-capable agent — Hermes, Claude, Cursor, Codex — via cua-driver over SSH.

Keep your agent on a central machine (a VM, a server, your desk PC) and drive the desktop of a Mac elsewhere on your network. No agent runs on the Mac — just a one-time, 2-minute cua-driver install. No daemons to babysit — the bridge starts the driver when needed and stops it when idle.

┌──────────────────────────┐         ┌──────────────────────────┐
│ Your agent host (VM/PC)  │         │ MacBook / Mac mini       │
│                          │         │                          │
│  Hermes/Claude/Cursor…   │  SSH    │  cua-driver (MCP daemon) │
│        │                 │────────►│   Accessibility grants   │
│        ▼                 │  call   │   Screen Recording       │
│  remote-cua-bridge       │         │                          │
│  (MCP server)            │         │   starts on demand       │
└──────────────────────────┘         │   exits when idle        │
                                     └──────────────────────────┘

Why this exists

Hermes (and most agents) can drive the local desktop through cua-driver — but cua-driver has no remote transport. If your agent lives on a central VM (sessions, memory, cron all there) and you also want it to see and click your laptop's screen, the built-in tool can't reach it.

This bridge is the missing link: a thin MCP server that translates the familiar computer_use action vocabulary into cua-driver call commands executed over SSH.

Features

• Same vocabulary as built-in computer use — capture, click, type, key, scroll, drag, list_apps, focus_app, wait
• Screenshots inline — captures return the PNG plus a numbered element tree, no temp-file approval dances
• Auto-resolves windows — macOS needs window_id; the bridge finds it for you
• Lazy daemon start — if the Mac's cua-driver isn't running, the bridge starts it over SSH, waits, retries. First call after a reboot ~5s.
• Idle daemon exit — after CUA_BRIDGE_IDLE_MINUTES (default 15, 0 = disabled) of no calls, the bridge stops the remote daemon. It comes back automatically on the next action.
• Key-only SSH — BatchMode=yes, no passwords over the wire.

Onboarding (agent-guided — 30 seconds)

This is designed to be installed by your agent, for your agent.

1. Copy this one block:

Fetch the skill at this URL, save it as a Hermes skill (use your
skill_manage tool), then load it and follow its instructions to set me
up to control my Mac:

https://raw.githubusercontent.com/bonut88/remote-cua-bridge/main/skills/remote-cua-bridge-setup/SKILL.md

2. Paste it into Hermes and hit send.

That's it. Hermes fetches the skill, installs it, then walks you through:

1. Ask for the Mac's SSH address + username
2. Test SSH, set up keys if missing
3. Install cua-driver on the Mac over SSH
4. Wait for you to click the macOS permission dialogs (the only human step)
5. Install the bridge (pip or source), register it with Hermes, verify with a live capture

Alternative (power users, from a terminal):

hermes skills install https://raw.githubusercontent.com/bonut88/remote-cua-bridge/main/skills/remote-cua-bridge-setup/SKILL.md

Then in a new Hermes session: "set me up to control my Mac"

Updating

Agent-guided (same as install — 30 seconds):

Paste this into Hermes:

Fetch the latest setup skill from
https://raw.githubusercontent.com/bonut88/remote-cua-bridge/main/skills/remote-cua-bridge-setup/SKILL.md,
then ask your agent to update the bridge. It detects the install method,
checks for a newer version, upgrades in place, updates the skill, checks
MCP registration for new env vars, and verifies with doctor — no
re-onboarding needed, nothing lost.

Manual:

# pip install (once published):
pip install --upgrade remote-cua-bridge

# Source install:
cd remote-cua-bridge && git pull && uv sync

Then run remote-cua-bridge doctor to verify, and start a new session.

Manual setup

1. On the Mac (one-time, ~2 minutes)

# Install cua-driver
/bin/bash -c "$(curl -fsSL https://cua.ai/driver/install.sh)"

# Start it once so macOS asks for permissions
open -n -g -a CuaDriver --args serve

Then grant Accessibility and Screen Recording to CuaDriver.app in System Settings → Privacy & Security. Verify:

/Users/you/.local/bin/cua-driver permissions status   # both ✅

2. SSH from your agent host to the Mac

# On your agent host — copy your SSH key to the Mac
ssh-copy-id you@my-mac

# Verify a one-shot command works
ssh you@my-mac "echo hello"    # → hello

Non-interactive SSH needs your key in the Mac's ~/.ssh/authorized_keys. If you use an SSH alias (~/.ssh/config), the bridge can use it directly.

3. Install the bridge

pip install remote-cua-bridge
# or
uv tool install remote-cua-bridge
# or run from source
uv run remote-cua-bridge

Configuration

Environment variables:

Variable	Default	Meaning
CUA_BRIDGE_MAC_HOST	(required)	SSH host of the Mac (alias from ~/.ssh/config or user@ip)
CUA_BRIDGE_MAC_USER	(required)	SSH username
CUA_BRIDGE_IDLE_MINUTES	15	Minutes of no calls before the remote daemon is stopped. 0 disables.
CUA_BRIDGE_DAEMON_START_CMD	open -n -g -a CuaDriver --args serve	How to start the remote daemon

You can also pass host= per tool call to target a specific Mac.

Hermes integration

hermes mcp add remote-cua-bridge \
  --command uv \
  --env CUA_BRIDGE_MAC_HOST=my-mac CUA_BRIDGE_MAC_USER=you \
  --args run --directory /path/to/remote-cua-bridge remote-cua-bridge

The tool appears as mcp_remote_cua_bridge_mac_computer_use in new sessions.

Other MCP clients (Claude Code, Cursor, Codex): register the server the same way you'd register any stdio MCP server:

{
  "mcpServers": {
    "remote-cua-bridge": {
      "command": "remote-cua-bridge",
      "env": { "CUA_BRIDGE_MAC_HOST": "my-mac", "CUA_BRIDGE_MAC_USER": "you" }
    }
  }
}

Usage

capture(action="list_apps")                     # what's running on the Mac
capture(action="capture", app="Safari")          # screenshot + element tree
capture(action="click", element=7, pid=4242)     # click by element index
capture(action="click", coordinate=[100, 200])   # or by pixel
capture(action="type", text="hello", pid=4242)   # type text
capture(action="key", keys="cmd+s")              # key combo
capture(action="scroll", direction="down", amount=3)
capture(action="drag", from_coordinate=[10,10], to_coordinate=[300,300])
capture(action="focus_app", app="Finder")

Security notes

• SSH is key-only with BatchMode=yes; no credentials cross the wire beyond your existing SSH key.
• The Mac's cua-driver runs in standard permission mode by default.
• Every mutating action still goes through your agent's approval flow.
• This is a powerful capability — only expose it on networks you trust.

Tools

remote-cua-bridge doctor

One-command diagnostic for the whole chain. Tells you exactly what's wrong:

CUA_BRIDGE_MAC_HOST=my-mac CUA_BRIDGE_MAC_USER=you remote-cua-bridge doctor

Checks: SSH reachability, cua-driver install + version, macOS permissions (Accessibility, Screen Recording), a live list_apps probe, a capture probe (screenshot + element tree), and prints the exact hermes mcp add command for your setup.

Add --json for machine-readable output.

Testing

uv sync
uv run pytest tests/ -m "not smoke"      # unit tests
uv run pytest tests/test_smoke_stdio.py -v -s -m smoke   # live E2E over SSH

License

MIT