acuser

acuser is a local macOS observation and control CLI for AI agents. It combines structured macOS APIs with mouse/keyboard control and screenshots as a visual fallback.

Install for development

python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e .

The package currently targets macOS and requires Python 3.10 or newer.

Agent guidance

The built-in agent guide is the canonical usage reference:

acuser agent-help
acuser agent-help --format json

The Markdown guide explains the observe → act → verify loop. The JSON form uses schema acuser.agent-command-catalog.v1 and is intended for wrappers, skills, and other agents that need to discover commands without parsing terminal prose.

Recommended agent loop

1. Check access with acuser permissions.
2. Establish context with acuser acclimate or query apps and windows.
3. Focus the target application.
4. Prefer Accessibility inspection and semantic clicks.
5. Use coordinates only after resolving current window bounds.
6. Use a screenshot when the UI is visual or Accessibility data is insufficient.
7. Verify important actions with a fresh query or screenshot.

Never reuse stale coordinates after an app switch, scroll, window move, display change, or layout change.

Observation commands

acuser apps                         # app metadata, PIDs, bundle IDs, state
acuser apps --names                 # names only
acuser windows                      # window IDs, owners, titles, bounds
acuser inspect Safari --depth 3     # Accessibility tree
acuser dock                        # Dock geometry and ordered items
acuser permissions                  # macOS permission status
acuser acclimate --output /tmp/state.json
acuser state --input /tmp/state.json
acuser screenshot WINDOW_ID /tmp/window.png

All successful commands emit JSON except agent-help in its default Markdown mode. Coordinates are global macOS logical screen points. Screenshot pixels may be higher density and must be scaled before using them as click coordinates.

Control commands

acuser launch Arc
acuser focus Arc
acuser move 500 300
acuser click 500 300
acuser click --app Notes --title "New Note"
acuser type 'hello from an agent'
acuser press enter
acuser hotkey command l
acuser hotkey command+shift 4
acuser scroll -6
acuser scroll 3 --horizontal 2
acuser drag 400 300 900 600 --duration 0.8

click --app ... --title ... uses the macOS Accessibility API and is generally more stable than coordinate clicking. type uses paste-and-restore by default; --method events is a fallback for controls that reject paste. drag emits interpolated left-button drag events for gesture-sensitive controls.

Permissions

The process needs macOS Accessibility permission for cross-application inspection and control. Screen Recording permission may be required for window capture. These are granted in System Settings → Privacy & Security and normally persist after the first approval.

acuser permissions
acuser permissions --prompt

Safety model

Observation commands are read-only, except that acclimate writes a local state file and screenshot writes an image. Input commands can affect the active application and may submit forms, edit documents, send messages, or trigger external actions. Agents should identify the target, act deliberately, and verify the result. Do not expose passwords, tokens, or sensitive data in screenshots or typed command arguments.

Development checks

python -m py_compile acuser.py
acuser --help
acuser agent-help --format json