pyguitest

Python GUI automation for real desktop applications.

Automate and test Linux and BSD desktop applications from Python — even when the application has no automation API.

Pyguitest provides one Python API for mouse, keyboard, window, screenshot, and accessible UI automation across Wayland, X11, and XWayland. It is the Python successor to X11::GUITest.

Use it to:

• 🧪 Build reliable desktop GUI tests
• 🤖 Automate repetitive desktop tasks
• 🖱️ Control applications like a real user
• 🔎 Find and interact with accessible UI elements
• 📸 Capture screenshots and failure artifacts
• 🧰 Diagnose desktop automation environments
• 🔄 Modernize applications and test suites built around older X11 automation tools
• 🎬 Choreograph screen action for film, TV and stage — a character's typing and clicking, landing on cue

Status: every capability implemented across all backends, covering every X11::GUITest export. Much of it has been run against real GNOME Wayland and X11 sessions; some of it has not, and docs/validation.md says exactly which is which, so nothing here has to be taken on trust.

Because desktops differ in what they permit, what a session can do is discovered at runtime rather than assumed — gui.supports(...) is how you ask, and docs/developers/design.md is why the API is shaped that way instead of being a one-to-one port.

New here? docs/getting-started.md is five minutes from nothing to a working script. docs/recipes.md answers "how do I…", and docs/troubleshooting.md answers "why didn't that work".

Install

Requires Python 3.10 or newer.

pip install pyguitest              # core; no dependencies
pip install 'pyguitest[atspi]'     # + element automation

Or from a checkout, which is the same thing with a path instead of a name:

git clone https://github.com/ctrondlp/pyguitest.git
cd pyguitest
pip install .
pip install '.[atspi]'

You do not need -e; that flag is for developing this package, and is covered in CONTRIBUTING.md.

None are required. The package imports and runs with nothing else installed. What you add depends on which backend has to serve your desktop — extras (atspi, x11, uinput, eiinput, dev), a few distribution packages pip cannot supply, and sometimes a tool on PATH. Rather than work that out from a document, ask the machine:

pyguitest doctor

It detects your distribution and prints the exact commands. For the whole picture — a per-backend requirements matrix, the distribution package table, and how capture chooses a path — see docs/install.md. Injecting input has its own setup (/dev/uinput permissions, the ydotool daemon, libei, portal consent): docs/input.md.

Usage

import pyguitest

gui = pyguitest.connect()

# Widgets by what they are and what they are called -- the recommended way.
gui.button("OK").click()
gui.text_field("Name").set_text("Ada Lovelace")
gui.dropdown("Country").choose("Norway")

# Windows by title -- a plain string, or a compiled regex -- and by app id,
# which is what survives a title that changes with the document.
window = gui.find_window("Editor")
editor = gui.find_window(app_id="org.gnome.TextEditor")

# Coordinates and keys, when you need them.
gui.move_mouse(500, 300)
gui.click()
gui.type_text("Hello")
gui.send_keys("^(a)^(c)")  # Ctrl-A, Ctrl-C

# Motion the toolkit can see, for drag-and-drop and hover.
gui.drag((120, 400), (600, 400))

Matching on role and name survives the application being moved or resized, unlike clicking at (842, 612). elements()/element() take more than role and name -- enabled/visible filter on state, name/description take a compiled regex instead of an exact string, and predicate is an escape hatch for anything else (an ancestor/descendant check, say):

from pyguitest import Role

gui.elements(role=Role.PUSH_BUTTON, enabled=True)
gui.element(name=re.compile(r"^Save"))
gui.element(role=Role.CHECK_BOX, within=gui.window_element("Preferences"))

Ask before depending on anything that varies by desktop:

from pyguitest import Capability

if gui.supports(Capability.WINDOW_GEOMETRY):
    x, y, w, h = gui.geometry(window)

connect() never raises on a limited desktop — a session with few capabilities is the normal case, and supports() is how you find out.

A session is usually several backends at once: elements from AT-SPI, injection from a CLI adapter, capture from another. CompositeBackend merges their capabilities and routes each call to whichever member provides it, so callers see one object. backend.providers() shows the routing.

Screenshots

gui.screenshot("desktop.png")  # the whole desktop
gui.screenshot("editor.png", window=window)  # one window
gui.screenshot("corner.png", region=(0, 0, 400, 300))

region is (x, y, width, height) in screen coordinates — the same tuple gui.geometry(window) returns, on every backend. You never write a tool's own rectangle syntax; whichever tool the session picked gets its own built for it. window is served two ways, and the difference shows in the image: under X11 the window's own pixels are read, so anything stacked on top of it is absent; everywhere else the rectangle is looked up and cut out of a full-screen shot, which does include whatever is covering it. gui.supports(Capability.WINDOW_CAPTURE) tells you which you are getting.

Automatically, when a test fails. Nothing captures on its own — a screenshot has to be taken while the failure is still propagating, because by the time an except: block runs the application under test is usually gone. Wrap the part you want documented:

with gui.capture_on_failure("artifacts"):
    gui.button("Save").click()
    assert gui.element(name="Saved")

Nothing is written when the block succeeds. On failure the image lands in artifacts/ (or $PYGUITEST_SCREENSHOT_DIR, or the temporary directory), its path is attached to the exception as .screenshot, and the original exception is re-raised untouched, so the test runner still reports the real failure. A screenshot that itself fails is recorded on the exception as .screenshot_error and swallowed — it never replaces the failure it was trying to document.

Examples

Runnable scripts in examples/, each degrading with an explanation when the desktop cannot do what it asks:

python3 examples/01_what_can_i_do.py     # start here
python3 examples/03_widgets.py           # buttons, text boxes, dropdowns
python3 examples/06_a_real_test.py       # the one to copy: a unittest suite

Tools

pyguitest                     # what this desktop can actually do
pyguitest doctor              # what to install to unlock more
pyguitest debug               # everything needed to diagnose a bug report
pyguitest inspect             # the accessible tree of every open window
pyguitest migrate script.pl   # what porting a Perl script involves
pyguitest record              # hand off to pyguitest-recorder, if installed

All six also work as python -m pyguitest … without installing.

pyguitest debug is what to paste into a bug report: package and Python versions, every environment probe (not only the ones that came back true), each detected tool's own --version, and whether the process is running inside a Flatpak, toolbox, or other container -- which changes what every other probe on this list actually sees. Add --json for a machine-readable form.

pyguitest inspect walks the accessible tree of every open window and prints it, grouped by application -- the tool for seeing what gui.button(...) or gui.element(role=..., name=...) actually has to match against, without writing a script first. --window TITLE_REGEX narrows it to one application; --json gives the same tree as machine-readable data, the same split debug uses.

The migration scanner reports the tier of every X11::GUITest call in a source file and exits non-zero if any call has no Wayland path, so a port can be gated in CI.

pyguitest record is an alias for pyguitest-recorder, which records desktop activity and writes the pyguitest script for it. That tool is a separate package -- pyguitest does not depend on it, and installing pyguitest does not install it -- so the subcommand reports how to install it if it is absent. Everything after record is passed straight through, so pyguitest record --help is the recorder's own help and every one of its flags works unchanged.

Documentation

Start here

• docs/getting-started.md — five minutes to a working script, which API to reach for, and what X11, Wayland and XWayland each change
• docs/recipes.md — task-shaped answers: waiting properly, forms, windows, screenshots, CI, and an X11::GUITest cheat sheet
• docs/troubleshooting.md — symptom first: nothing found, nothing typed, nothing captured

Reference

• docs/api.md — the full API reference: every public class, method and enum, with the capability each one needs
• docs/install.md — what each backend needs, per distribution, and how capture picks a path
• docs/input.md — injecting pointer and keyboard input: permissions, daemons, keymap safety, libei and the portal
• docs/validation.md — what has been run against a real desktop, and what has not
• docs/ai-assistants.md — rules for a coding assistant generating pyguitest code
• testable-guis.md — how to build a GUI that can be tested at all: the accessibility work that lets a test name a button instead of clicking a coordinate. Written to be handed to application developers; lives in the recorder repository

Design and internals — docs/developers/: why the API is not a port, the audit of all 50 X11::GUITest exports it derives from, the two ADRs, the repository structure, and the protocol gaps worth taking upstream.

Contributing

Tests, lint, types, CI and the D-Bus suite: CONTRIBUTING.md.

License

GPL-2.0-or-later. See LICENSE.