screenagent-ai 0.3.0

Computer Use SDK for Python — build screen-controlling AI agents in a few lines of code

screenagent

Control any macOS app with Claude — Python SDK + CLI.

Browser Use and Skyvern only work inside the browser. screenagent uses macOS Accessibility API + CGEvent for native input, so it works with System Settings, Finder, Notes, Calculator, and any app.

Install

pip install screenagent

Requires macOS and Python 3.11+.

Usage 1: CLI

Run directly from the terminal. Works with Claude Code out of the box.

# Control native apps (finds and launches via Spotlight automatically)
screenagent run "Open Calculator and compute 42 * 17"
screenagent run "Open System Settings and switch to Dark Mode"

# Browser automation
screenagent run "Open Chrome, go to youtube.com, search for ycombinator"
screenagent run --app "Google Chrome" "Go to google.com and search for AI news"

# Individual actions (no API key needed)
screenagent screenshot --file screen.png
screenagent ax-tree "Google Chrome"
screenagent click 640 400
screenagent type "hello world"
screenagent key return --modifiers command

Usage 2: Python SDK

3-line agent

from screenagent import Agent

agent = Agent()
result = agent.run("Open System Settings and switch to Dark Mode")
print(result.summary)
print(result.success)

Component functions (no API key needed)

from screenagent import screenshot, click, type_text, key_press, get_ui_tree

png_bytes = screenshot()
click(640, 400)
type_text("hello world")
key_press("return")

tree = get_ui_tree("Google Chrome")
print(tree.to_text())

Configuration

Environment Variable	Default	Description
ANTHROPIC_API_KEY	—	Claude API key (required for agent)
AGENT_MODEL	claude-sonnet-4-6	Model to use
AGENT_MAX_STEPS	20	Maximum agent loop iterations
AGENT_COMPUTER_USE	true	Use Claude computer-use tool
CDP_PORT	9222	Chrome DevTools Protocol port

Also supports .env files.

Requirements

• macOS (Quartz CGEvent, Accessibility API)
• Python 3.11+
• Accessibility permission granted to your terminal/IDE
• Chrome with --remote-debugging-port=9222 for CDP features (optional)

License

MIT