Agent Browser Bridge

Let an AI agent drive the Microsoft Edge window you already have open, on the active tab on your screen. On Windows the same daemon also drives the desktop itself: screenshots, mouse and keyboard for the dialogs and native apps a browser extension cannot reach.

Automating a clean browser profile drops existing cookies and auth state. Agent Browser Bridge attaches to your running browser instead, keeping active logins and extensions available while sending trusted Chrome DevTools Protocol events directly to the page.

A single background daemon provides the MCP server, CLI, and Python library.

Tested on Microsoft Edge. The extension uses standard MV3 and speaks CDP, so other Chromium browsers may work, but Edge is the only tested target.

Install

pip install agent-browser-bridge

Python 3.10 or newer, with no third-party packages at install time or runtime. The optional desktop extra adds Pillow for the Windows desktop half (pip install "agent-browser-bridge[desktop]").

Windows

When agent-bridge isn't recognized after install (Python's Scripts folder isn't on your PATH), run commands with py -m:

py -m agent_browser_bridge daemon start
py -m agent_browser_bridge setup

To add Scripts to your user PATH automatically, run py -m agent_browser_bridge setup --fix-path and reopen your terminal. If you're using pipx, run pipx ensurepath. To check launcher and environment health, run py -m agent_browser_bridge doctor.

Then install the companion extension directly from the Microsoft Edge Add-ons store (Agent Browser Bridge) or the Chrome Web Store, or load the extension from the local folder while developing:

agent-bridge extension open     # opens the bundled extension directory

Turn on Developer mode at edge://extensions, select the opened extension folder, and pick that directory. Start the daemon and verify Edge connected:

agent-bridge daemon start
agent-bridge daemon status

Daemon: running on 127.0.0.1:18999 (PID 24188)
Extension: 2.6.1 (connected)

CLI commands and Python's with Edge() start the daemon on first use if it isn't running, but daemon start brings the extension online right away.

Upgrading from Edge Agent Bridge

pip install -U edge-agent-bridge moves an existing install over: it installs agent-browser-bridge and leaves a small compatibility shim so edge-bridge, edge, python -m edge_agent_bridge and from edge_agent_bridge import Edge keep working. From then on, upgrade with pip install -U agent-browser-bridge. Nothing else changes: same port, same data directory, same MCP server key and tool names, and the store extension updates in place under its new name.

Wire it into an agent

agent-bridge setup

That detects the MCP clients on your machine and registers the server with each one. Claude Code, Cursor, Windsurf, Gemini CLI, Codex CLI and VS Code are covered. JSON config files are backed up before they're touched, and a file that doesn't parse is left alone and reported.

To see the config without writing anything:

agent-bridge mcp-config --client claude
agent-bridge mcp-config --client cursor

The server speaks stdio JSON-RPC and exposes 34 tools, all prefixed edge_ so they don't collide with Playwright MCP in a mixed setup. docs/agent-guide.md is written for the agent rather than for you: snapshot first, act by ref, re-snapshot after navigation.

Snapshot and refs

The agent reads the page as a tree of roles, names and current values, each interactive node carrying a short id:

$ agent-bridge snapshot
tab 1459 "Files - Example App" http://localhost:8080/files
page "Files - Example App" url=http://localhost:8080/files
- heading "Files" level=1
- textbox "Search" [e1] value=""
- button "Upload" [e2]
- file "Choose file" [e3] hidden
- table "Files"
  - row "12080 spec.pdf 2026-09-01" [e4]
    - button "Open" [e5]

Every action that takes a target accepts one of those refs, so click e5 hits the button that was actually named in the tree. A ref that no longer resolves returns stale_ref, and a ref from a snapshot taken before the last navigation returns stale_snapshot. Neither one silently clicks something else, which is the failure mode that makes text matching unusable on dense pages.

Values come back too, which is what makes "fill in the rest of this form" work: the agent can see which fields you already filled. Passwords are masked. Hidden input[type=file] nodes are listed anyway and tagged hidden, because upload buttons almost always hide the real input. Inside an iframe a ref carries its frame, as in f31e1, including cross-origin frames.

Python

from agent_browser_bridge import Edge

with Edge() as browser:
    snap = browser.snapshot()
    print(snap["text"])

    browser.fill("e1", "spec")
    browser.click("e5")
    browser.wait(text="Opened")
    print(browser.console())

The daemon starts on first use. Edge pins itself to the tab of its first result, so a later call can't drift onto a different tab because you switched windows in the meantime. Bare CLI commands share one daemon-side pin instead: switch and new set it, tab-less commands keep acting on that tab, and --tab stays a one-shot override; several agents at once still need one session each. Methods return the raw result dict and never raise on a failed action: check success, read code and error.

Command line

agent-bridge snapshot                      # tree with refs; --full adds static text
agent-bridge snapshot --compact --max-nodes 50  # terse refs-only lines; 0 = unlimited
agent-bridge elements                      # interactive nodes with coordinates
agent-bridge click e5                      # ref, CSS selector or visible text
agent-bridge fill e1 "AI agents"
agent-bridge type "slow typing" --delay 30 # per-key events, for autocomplete widgets
agent-bridge key Enter
agent-bridge select e6 --label 日本語
agent-bridge upload e3 C:/tmp/spec.pdf    # WSL /mnt/<drive>/... paths translate automatically
agent-bridge wait --text Opened
agent-bridge wait --idle                   # no in-flight requests; ignores WebSockets
agent-bridge screenshot out.jpg --of e4
agent-bridge eval "document.title"         # prints the returned value (here: the page title)
agent-bridge console                       # console output, exceptions, dialogs
agent-bridge nav https://example.com
agent-bridge new https://example.com --group Agent
agent-bridge close --tab 1459
agent-bridge history search "quarterly report"  # search titles/URLs; --max-results, --start-time/--end-time
agent-bridge history delete https://example.com/page
agent-bridge group list                     # tab groups; --window-id filters
agent-bridge group move 1459 1460 --title Agent --color blue   # omit --group-id for a new group
agent-bridge group ungroup 1459
agent-bridge batch '[{"action":"click","target":"e5"},{"action":"sleep","ms":50},{"action":"fill","target":"e1","text":"spec"}]'
agent-bridge run steps.json --stop-on-error  # batch from a file; one JSON line per step
agent-bridge session start --name agent-a  # prints sessionToken=<uuid>; --new always mints
agent-bridge session list                  # named pins; prune drops closed-tab sessions
agent-bridge session status --session <uuid>
agent-bridge session stop --session <uuid>
agent-bridge click e5 --session <uuid>   # session pin; --tab is a one-shot override
agent-bridge click e5 --no-fallback      # disable the ref→text→scan→coords ladder
agent-bridge doctor                      # diagnose daemon, extension, token, pairing
agent-bridge logs --tail 50              # daemon log (--follow streams)
agent-bridge daemon stop                 # only if /status confirms bridge.pid; --force for a hung daemon

--json prints the raw result as a single line and sets the exit code: 0 on success, 1 when the action failed, 2 when the daemon is unreachable, 3 on a usage error. --no-highlight turns off the ring drawn around the element about to be acted on.

Each CLI call pays roughly 600 ms of Python startup. For anything repetitive use MCP, the Python API, or run (one process for a whole step file).

Desktop (Windows)

pip install "agent-browser-bridge[desktop]"

The daemon drives the Windows desktop through the same token and port as the browser. An agent takes a screenshot, reads it, acts with coordinates read off that image, then takes another screenshot to confirm. OS file dialogs, installers, native applications and pages the extension cannot attach to all become reachable.

agent-bridge desktop shot                  # writes <data dir>/shot.png, prints view/display/scale
agent-bridge desktop shot look.png --monitor 2 --region 100,80,400,120   # one part, for small text
agent-bridge desktop screen                # list displays (numbered left to right) without capturing
agent-bridge desktop click 640,360 --button right --count 2 --modifiers ctrl
agent-bridge desktop move 640,360          # reveal a hover state, then shot, then click
agent-bridge desktop drag 100,100 400,300
agent-bridge desktop drag 100,100 400,300 --through "200,50;300,150"   # one stroke through waypoints
agent-bridge desktop scroll down 3 --at 640,360
agent-bridge desktop window               # which window has the keyboard, before typing
agent-bridge desktop type "hello"
agent-bridge desktop key ctrl+s --repeat 2
agent-bridge desktop hold shift 2
agent-bridge desktop wait 1.5
agent-bridge desktop cursor                # where the pointer is, in both spaces
agent-bridge desktop notify "filling the form now" --seconds 5   # a notice on the overlay
agent-bridge desktop record start --name expense-claim --describe "file one expense"
agent-bridge desktop record stop           # writes <data dir>/recordings/expense-claim/SKILL.md

MCP tools: desktop_shot, desktop_screen, desktop_cursor, desktop_window, desktop_click, desktop_move, desktop_drag, desktop_scroll, desktop_type, desktop_key, desktop_hold, desktop_notify, desktop_record. desktop_shot returns the image inline plus the coordinate space and the foreground window. Python: Edge().send("desktop_click", {"at": [640, 360]}).

While a desktop action runs, a small strip in the top-right corner of the primary display says so. It never takes focus and stays out of screenshots on Windows 10 2004 and later; EDGE_BRIDGE_NO_OVERLAY=1 turns it off. desktop notify puts an agent's own short notice there.

desktop record watches you instead of the agent: it keeps your clicks and keys, a screenshot per click, and on stop writes a SKILL.md of numbered steps with the images, which an agent can follow later with the same tools. The strip turns red for the whole recording and refuses to start without the strip. The bridge's own injected input is left out, so an agent can keep working while you demonstrate. Text composed through an IME never reaches a keyboard hook, so Japanese or Chinese input is not captured; the recorder stops itself after 2000 events or 30 minutes of activity.

Coordinates are view pixels: the pixel space of the last shot of that display, which is scaled down to a 1280 long edge by default. The daemon scales them back up; never convert yourself. A point outside the view is refused, not clamped. Every shot result carries space (view, display, or region); a region shot is a crop with its own origin and says not to click from it. Pass --space display to use real display pixels. Each display has its own view starting at 0,0; --monitor all captures every display as one wide image for finding things, then capture the one display before clicking.

Windows only: elsewhere every desktop action answers desktop_unsupported, and without Pillow desktop_unavailable. Start the daemon from your own shell: one started by a scheduled task or a service cannot move the cursor, and an elevated one cannot pair with Edge, though it still drives unelevated desktop apps. agent-bridge doctor reports both halves.

Security model

The trust boundary is your OS user account. Anything running as you can already read your files and your browser profile, so the daemon doesn't try to defend against it.

• The daemon binds 127.0.0.1 only, and checks the Host header against the loopback names so a DNS rebinding attempt gets a 421 rather than a command.
• /exec requires the token written to the data directory at first start, mode 0600 on POSIX. A request carrying an Origin or Sec-Fetch-* header is refused outright, so a page you are browsing can't reach the daemon even if it guesses the token.
• /ws requires an Origin beginning chrome-extension://, which keeps stray local clients off the extension channel, though a second OS user on a shared machine could forge that header. If that is your situation, run agent-bridge daemon start --require-pairing and paste the token into the extension popup once; unpaired sockets are then rejected.
• WebSocket frames above 16 MB close the connection, and the socket carries a read timeout, so a local client can't make the daemon buffer without bound or pin a worker thread forever.
• Nothing leaves the machine: no telemetry, no outbound requests of any kind.

desktop record installs keyboard and mouse hooks for as long as it runs, so everything you type, passwords included, lands in events.jsonl and, as Type "..." steps, in SKILL.md in your data directory. It starts only on an explicit record start, the red strip stays up until record stop (an agent's notices appear inside that red strip), and nothing is uploaded. Scrub the skill before sharing it and delete the folder when it has served its purpose.

eval runs arbitrary JavaScript in the page with your session. It's there on purpose, and it's the reason to think about which agent you hand this to. Actions that destroy state need an explicit tab id: tab_close without one is an error rather than a guess.

The extension asks for debugger, which is what makes input trusted and screenshots possible. While it is attached, Edge shows its "is debugging this browser" bar under a Chromium policy no extension can dismiss.

Measured benchmarks

Against live Edge tabs, from tests/e2e/test_benchmark.py. These are one machine's numbers; the timings move with the host, so the suite treats them as advisory and gates on the correctness assertions instead. EDGE_BRIDGE_BENCH_STRICT=1 makes the timings gate too.

python -m pytest tests/e2e/test_benchmark.py -q -s

Development

python -m pytest -q -m "not e2e"   # hermetic, no browser needed
python -m pytest -q -m e2e         # drives a real Edge
python scripts/build_extension.py  # syncs the manifest version, zips to dist/

The hermetic suite runs a real daemon against a fake extension over a real WebSocket, so it covers the wire protocol without Edge installed. CI runs it on Windows, Linux and macOS and the E2E suite against a real Edge on Windows; releases are automated, see docs/release.md.

Support

Built by Shanewas Ahmed. If it saves you time, sponsorship funds keeping the CDP handlers current with Chromium releases:

• GitHub Sponsors

License

MIT. See LICENSE.