pyautoassist

Playwright-style desktop automation with native OS accessibility backends.

pyautoassist brings the developer experience of Playwright to desktop automation. No more time.sleep(). No more fragile coordinate-based clicking. Just explicit locators, native auto-waiting, and real OS accessibility APIs.

import pyautoassist

app = pyautoassist.using_backend()

# XPath-style locators targeting OS accessibility properties
app.locator("//Button[@Name='Submit']").click()
app.locator("//Edit[@AutomationId='email-input']").fill("Hello World")

# CSS-style shorthand
app.locator("button.submit-btn").click()
app.locator("edit-text#main-input").fill("Hello")

# Auto-waiting: no time.sleep() needed
app.locator("//Button[@Name='Save']").click()  # waits until clickable

Why pyautoassist?

Installation

# Core (auto-detects platform)
pip install pyautoassist

# Platform-specific extras
pip install pyautoassist[windows]   # Windows UIA
pip install pyautoassist[macos]     # macOS Accessibility
pip install pyautoassist[linux]     # Linux AT-SPI2

Quick Start

import pyautoassist

# Create an automation session
app = pyautoassist.using_backend()

# Find a window
notepad = app.open("Notepad")

# Use locators (auto-waits for element)
app.locator("//Edit").fill("Hello, pyautoassist!")

# Chain locators within elements
notepad.locator("//MenuItem[@Name='File']").click()
notepad.locator("//MenuItem[@Name='Save']").click()

Selectors

XPath Style

Target elements by their OS accessibility properties:

# By control type and name
app.locator("//Button[@Name='Submit']")
app.locator("//Edit[@Name='Username']")

# By AutomationId (most reliable)
app.locator("//Button[@AutomationId='btn-submit']")
app.locator("//Edit[@AutomationId='email-input']")

# By ClassName
app.locator("//Pane[@ClassName='Notepad']")

# Chained: nested elements
app.locator("//Pane[@ClassName='Notepad']//Button[@Name='File']")

CSS Style

Shorthand selectors for quick access:

# Type + ID (AutomationId)
app.locator("button#submit-btn")

# Type + class (ClassName)
app.locator("button.submit")

# Type + attribute
app.locator('button[name="OK"]')

# Child combinator
app.locator("pane > button")

Supported Control Types

button, edit/edit-text/textbox, pane, window/dialog, menu, menuitem, checkbox, radio/radiobutton, combobox/dropdown, list, listitem, tree, treeitem, toolbar, tab, tabitem, image, hyperlink, slider, progressbar, scrollbar, group, tooltip, statusbar, header, separator, document, dataitem, custom

Auto-Waiting

Every action on a Locator automatically waits for the element to be ready:

# Waits up to 30s for the button to exist, be visible, and be enabled
app.locator("//Button[@Name='Submit']").click()

# Custom timeout
app.locator("//Button[@Name='Submit']").with_timeout(5000).click()

# Wait for specific states
app.locator("//Button[@Name='Loading']").visible().click()
app.locator("//Edit[@Name='Email']").enabled().click()

Element Actions

el = app.locator("//Button[@Name='Submit']")

el.click()              # Left click
el.double_click()       # Double click
el.right_click()        # Context menu
el.fill("text")         # Set value (input fields)
el.type_text("text")    # Type character by character
el.clear()              # Clear input
el.select_option(label="Option 1")  # Dropdown selection
el.focus()              # Move keyboard focus
el.hover()              # Move mouse to element
el.press_key("Enter")   # Send key press

Window Management

# List all windows
for w in app.get_windows():
    print(f"{w.title} (PID: {w.pid})")

# Find by title
notepad = app.find_window(title="Notepad")
notepad.focus()
notepad.move(100, 100)
notepad.resize(800, 600)
notepad.close()

# Find by PID
window = app.find_window(pid=12345)

Codegen Recorder

Generate automation code by clicking around:

# Start recording (Windows)
pyautoassist record

# Save to file
pyautoassist record --output recorded.py

# Specify backend
pyautoassist record --backend macos

The recorder hooks into OS mouse events, inspects the accessibility tree at each click location, and prints pyautoassist-style code in real time.

Architecture

pyautoassist/
  __init__.py              # Public API
  _impl/
    pyautoassist.py           # Main entry point (pyautoassist class)
    locator.py             # Playwright-style Locator with auto-wait
    element.py             # Element wrapper with actions
    window.py              # Window management
    selector.py            # CSS/XPath-style selector engine
    wait.py                # Auto-wait polling loop
    types.py               # Data types and enums
    backend.py             # Abstract backend interface
  backends/
    windows_backend.py     # Windows UIA (UIA3 COM API)
    macos_backend.py       # macOS AXUIElement (PyObjC)
    linux_backend.py       # Linux AT-SPI2 (GObject)
  recorder/
    cli.py                 # Codegen CLI tool

License

MIT