Glasswarp Python SDK

See any screen. Control any PC. One API.

Glasswarp gives your agent eyes and hands on a real Windows machine; you supply the brain (model, prompts, CV, solvers). Example task logic lives under examples/ and outcome-named starters under templates/ — not in the platform. Public plans: Free / Builder / Growth / Enterprise. Game Mode at /play is a frozen demo of the same capture pipeline.

pip install glasswarp
# optional SoM / image helpers:
pip install "glasswarp[grounding]"
# optional glasswarp-demo CLI + bundled showcase scripts:
pip install "glasswarp[demos]"

Published on PyPI: pypi.org/project/glasswarp (0.1.1). Release process: docs/PYPI_PUBLISH.md.

Want Claude / Cursor with no project code? Use the remote MCP server instead — glasswarp.com/mcp · docs.

Quickstart

from glasswarp import GlasswarpClient

gw = GlasswarpClient(api_key="gw_live_sk_...")

# List your rigs
rigs = gw.list_rigs()
rig = rigs[0]

# Start a desktop session
session = gw.create_session(rig_id=rig.id)

# Take a screenshot (returns JPEG bytes)
screenshot = gw.screenshot(session.session_id)
with open("screen.jpg", "wb") as f:
    f.write(screenshot.jpeg)

# Send input
gw.click(session.session_id, x=500, y=400)
gw.type_text(session.session_id, "Hello from Python!")
gw.key_press(session.session_id, "Enter")

# End the session
gw.end_session(session.session_id)

Configuration

API reference

Rigs

• list_rigs() — all rigs accessible to this key
• get_rig(rig_id) — single rig detail

Sessions

• create_session(rig_id, mode="desktop") — start a session
• get_session(session_id) — current status
• list_sessions() — all active sessions
• end_session(session_id) — end and trigger safety_restore

Visual

• screenshot(session_id, max_width=None, quality=None, x=y=w=h=None) — JPEG; optional ROI crop
• dirty_rects(session_id) — DXGI dirty rects since last poll (VP0); available=false on GDI ⇒ assume changed
• list_targets(session_id) — sparse click targets from host UIA (P8); empty if none
• observe(session_id, mark=True, targets=None, …) — one RTT: JPEG + dirty + targets + capture_mode (SoM when marked)

Grounding (SoM)

Agents should click targets, not guessed pixels:

from glasswarp import targets_from_grid, som_annotate

# Host UIA targets (P8) or client CV/grid:
targets = targets_from_grid(x0=100, y0=200, x1=900, y1=800, rows=8, cols=8)
obs = gw.observe(session.session_id, max_width=1280, mark=True, targets=targets)
# obs.jpeg is Set-of-Mark annotated; model returns target id →
gw.click_target(session.session_id, "12", targets=targets)

• som_annotate(jpeg, targets, native_width=…, native_height=…) — draw numbered marks (needs pip install pillow)
• targets_from_grid(…) — build cell targets for a rectangular board
• click_target(session_id, target_or_id, targets=…) — click bbox center

Apps

• launch_app(session_id, path, args=None) — spawn a process on the rig (returns pid)
• kill_app(session_id, pid=None) — kill session-launched apps (pid optional)

Input

• send_input(session_id, events) — batch input events
• click(session_id, x, y, button) — click at coordinates
• type_text(session_id, text) — type a string
• key_press(session_id, key) — press a key
• move_mouse(session_id, x, y) — move cursor
• scroll(session_id, dx, dy) — scroll

Stream

• stream(session_id) — WebRTC signaling credentials for a live 60fps feed

Templates

Outcome-named starters in templates/ (observe → think → act → verify):

• templates/observe_think_act/ — minimal agent-loop skeleton
• templates/notepad_type/ — Notepad via UIA + type
• templates/live_view_hitl/ — hold a session for Console Live View

Demos CLI (tight loops — not MCP click-per-cell)

export GLASSWARP_API_KEY=gw_live_sk_...
glasswarp-demo list
glasswarp-demo minesweeper
glasswarp-demo mona-lisa

Same entrypoint: python -m glasswarp.demos minesweeper. Prefer this (or the scripts below) for click-speed loops; use MCP tools for ad-hoc UI. Guide: Ways to run agents.

Examples

See examples/ for complete runnable scripts:

• quickstart.py — connect, screenshot, click, type, disconnect
• vp0_smoke.py — dirty rects + ROI screenshot smoke
• vp0_observe.py — shared VP0 helper (dirty skip + ROI coalesce) for agent loops
• paint_mark_demo.py — an agent draws a themed GLASSWARP outline in Paint (custom colors + native drag). See Paint quickstart.
• paint_mona_lisa_demo.py — an agent reproduces the Mona Lisa in Paint (observe→think→act→verify: vision placement, UIA colors, tile fills, Save As→Desktop). VNC-hard differentiator. See Mona Lisa quickstart.
• notepad_uia_demo.py — Notepad via UIA targets + type. See Notepad quickstart.
• minesweeper_solver_demo.py — Minesweeper demo (CV + solver). See quickstart.
• gemini_agent_loop.py — Minesweeper demo with Gemini + SoM → click_target → verify
• claude_agent_loop.py — observe → Claude → act; VP0 on by default (VP0=0 for full frames)

License

Apache-2.0