pyguitest

Python GUI automation for real desktop applications.

Automate and test Linux, BSD and Microsoft Windows desktop applications from Python — even when the application has no automation API.

Pyguitest provides one Python API for mouse, keyboard, window, screenshot, and accessible UI automation across Wayland, X11, XWayland and Windows. It is the Python successor to X11::GUITest.

Microsoft Windows support arrives in 0.11.0, the first release that imports there at all. Two backends drive it: win32 — screens, input, windows, window events, screen capture and the clipboard, through ctypes with no dependency — and uia, the element tree, through UI Automation behind the windows extra. The suite passes on Windows 11 (build 26200) and four live runs have driven a real interactive desktop — but on one US-layout, single-monitor, unelevated machine, so multi-monitor arithmetic, windows()'s z-order claim, AltGr and dead keys are still open. docs/validation.md records which is which, and docs/developers/adr-003-windows.md what was decided and why.

Use it to:

• 🧪 Build reliable desktop GUI tests
• 🤖 Automate repetitive desktop tasks
• 🖱️ Control applications like a real user
• 🔎 Find and interact with accessible UI elements
• 📸 Capture screenshots and failure artifacts
• 🧰 Diagnose desktop automation environments
• 🔄 Modernize applications and test suites built around older X11 automation tools
• 🎬 Choreograph screen action for film, TV and stage — a character's typing and clicking, landing on cue

Status: every capability implemented across all backends, covering every X11::GUITest export. Much of it has been run against real GNOME, KDE, sway, Xfce, X11, GhostBSD and Windows 11 sessions; some of it has not, and docs/validation.md says exactly which is which, so nothing here has to be taken on trust.

Because desktops differ in what they permit, what a session can do is discovered at runtime rather than assumed — gui.supports(...) is how you ask, and docs/developers/design.md is why the API is shaped that way instead of being a one-to-one port.

New here? docs/getting-started.md is five minutes from nothing to a working script. docs/recipes.md answers "how do I…", and docs/troubleshooting.md answers "why didn't that work".

What pyguitest is for

One Python API for driving an application that has no automation API of its own — a desktop program somebody else wrote, which you cannot add a test hook to. It works at the accessibility layer first, so a button is found by its role and name rather than by where it happens to be drawn, and falls back to coordinates only where a toolkit publishes nothing. That is the reason gui.supports() exists: the API does not promise everything works everywhere, it tells you what this session can do.

That shape decides what it is good at and what it is not:

• Good at applications with no API of their own, tests that have to survive a redesign or a theme change, and working out why something is not being found on a desktop you do not control.
• Not a test framework. It is a library: no pytest plugin, no fixtures, no runner. Use whatever you already use, and it will fit underneath it.
• Not for browsers or mobile. Web pages have WebDriver and Playwright, phones have their own tooling. This drives desktop windows on Linux, the BSDs and Microsoft Windows; macOS has no backend.
• Not the first choice where the application can help. If a program ships a command line, a documented API or an in-app test hook, driving that is faster, more stable, and says what the test means instead of what it clicked. A toolkit's own test support is closer to the application than this can be.
• Not a recorder. pyguitest-recorder is a separate package that writes pyguitest scripts from a recorded session, and pyguitest does not depend on it.

Compared to other tools

These overlap less than a search result makes it look, and for several jobs one of the others is the better answer.

Use PyAutoGUI if you need macOS, or you want image matching behind a small API and your desktop is X11 or Windows. It is the most widely used of these by a wide margin, and that is worth real money in answered questions.

Use pywinauto if the target is Windows and only Windows. It is mature and Windows-shaped throughout, with far more Windows-specific knowledge behind it than a package spanning four desktops can carry.

dogtail is not really a competitor. The atspi extra installs it, and the element tree on Linux is dogtail underneath. Use it directly if AT-SPI on Linux is all you need and you would rather have one less layer.

What is left is the awkward middle, and it is the only thing pyguitest claims to be better at: one API that keeps working when the session underneath changes — X11 to Wayland, GNOME to KDE to sway, Linux to Windows — and gui.supports() to ask what the session in front of you can actually do instead of discovering it by failing. If you do not have that problem, one of the above is a shorter road.

These projects move, and the table is our reading of them rather than theirs — check their own documentation before deciding on it.

Install

Requires Python 3.10 or newer.

pip install pyguitest              # core; no dependencies
pip install 'pyguitest[atspi]'     # + element automation
pip install "pyguitest[windows]"   # + element automation on Windows (comtypes)

Or from a checkout, if you are working from the source tree:

git clone https://github.com/ctrondlp/pyguitest.git
cd pyguitest
pip install .
pip install '.[atspi]'

You do not need -e; that flag is for developing this package, and is covered in CONTRIBUTING.md.

None are required. The package imports and runs with nothing else installed. What you add depends on which backend has to serve your desktop — extras (atspi, x11, uinput, eiinput, windows, dev), a few distribution packages pip cannot supply, and sometimes a tool on PATH. Rather than work that out from a document, ask the machine:

pyguitest doctor

It prints the exact commands — naming your distribution's packages on Linux and the BSDs, and answering in Windows terms on Windows. For the whole picture — a per-backend requirements matrix, the distribution package table, and how capture chooses a path — see docs/install.md. Injecting input has its own setup (/dev/uinput permissions, the ydotool daemon, libei, portal consent): docs/input.md.

What works where

Which backend serves which part of a session. This is the shape of the answer, not a promise about your machine: what a session actually assembled is what connect() reports, what backend.providers() lists, and what pyguitest doctor prints.

Two things the table cannot say. Whether an application publishes anything to AT-SPI is up to the application, and testable-guis.md is about that side of it. And which of these paths has actually been run against a real desktop is in docs/validation.md — that is the file to read before trusting any row here, and it is written to be read that way. That file is where the Windows row above needs reading twice: win32 and uia are registered and composed exactly like every backend above them, and both have driven a real Windows 11 desktop — but on one machine, with one layout and one monitor, so the row says what is implemented and validation.md says what has been measured. docs/developers/adr-003-windows.md records the design and the alternatives rejected along the way.

Usage

import pyguitest

gui = pyguitest.connect()

# Widgets by what they are and what they are called -- the recommended way.
gui.button("OK").click()
gui.text_field("Name").set_text("Ada Lovelace")
gui.dropdown("Country").choose("Norway")

# Windows by title -- a plain string, or a compiled regex -- and by app id,
# which is what survives a title that changes with the document.
window = gui.find_window("Editor")
editor = gui.find_window(app_id="org.gnome.TextEditor")

# Coordinates and keys, when you need them.
gui.move_mouse(500, 300)
gui.click()
gui.type_text("Hello")
gui.send_keys("^(a)^(c)")  # Ctrl-A, Ctrl-C

# Motion the toolkit can see, for drag-and-drop and hover.
gui.drag((120, 400), (600, 400))

Matching on role and name survives the application being moved or resized, unlike clicking at (842, 612). elements()/element() take more than role and name -- enabled/visible filter on state, name/description take a compiled regex instead of an exact string, and predicate is an escape hatch for anything else (an ancestor/descendant check, say):

from pyguitest import Role

gui.elements(role=Role.PUSH_BUTTON, enabled=True)
gui.element(name=re.compile(r"^Save"))
gui.element(role=Role.CHECK_BOX, within=gui.window_element("Preferences"))

Ask before depending on anything that varies by desktop:

from pyguitest import Capability

if gui.supports(Capability.WINDOW_GEOMETRY):
    x, y, w, h = gui.geometry(window)

connect() never raises on a limited desktop — a session with few capabilities is the normal case, and supports() is how you find out.

A session is usually several backends at once: elements from AT-SPI, injection from a CLI adapter, capture from another. CompositeBackend merges their capabilities and routes each call to whichever member provides it, so callers see one object. backend.providers() shows the routing.

Screenshots

gui.screenshot("desktop.png")  # the whole desktop
gui.screenshot("editor.png", window=window)  # one window
gui.screenshot("corner.png", region=(0, 0, 400, 300))

region is (x, y, width, height) in screen coordinates — the same tuple gui.geometry(window) returns, on every backend. You never write a tool's own rectangle syntax; whichever tool the session picked gets its own built for it. window is served two ways, and the difference shows in the image: under X11 the window's own pixels are read, so anything stacked on top of it is absent; everywhere else the rectangle is looked up and cut out of a full-screen shot, which does include whatever is covering it. gui.supports(Capability.WINDOW_CAPTURE) tells you which you are getting.

Automatically, when a test fails. Nothing captures on its own — a screenshot has to be taken while the failure is still propagating, because by the time an except: block runs the application under test is usually gone. Wrap the part you want documented:

with gui.capture_on_failure("artifacts"):
    gui.button("Save").click()
    assert gui.element(name="Saved")

Nothing is written when the block succeeds. On failure the image lands in artifacts/ (or $PYGUITEST_SCREENSHOT_DIR, or the temporary directory), its path is attached to the exception as .screenshot, and the original exception is re-raised untouched, so the test runner still reports the real failure. A screenshot that itself fails is recorded on the exception as .screenshot_error and swallowed — it never replaces the failure it was trying to document.

Examples

Runnable scripts in examples/, each degrading with an explanation when the desktop cannot do what it asks:

python3 examples/01_what_can_i_do.py     # start here
python3 examples/03_widgets.py           # buttons, text boxes, dropdowns
python3 examples/06_a_real_test.py       # the one to copy: a unittest suite

Tools

pyguitest                     # what this desktop can actually do
pyguitest doctor              # what to install to unlock more
pyguitest debug               # everything needed to diagnose a bug report
pyguitest inspect             # the accessible tree of every open window
pyguitest migrate script.pl   # what porting a Perl script involves
pyguitest record              # hand off to pyguitest-recorder, if installed

All six also work as python -m pyguitest … without installing.

pyguitest debug is what to paste into a bug report: package and Python versions, every environment probe (not only the ones that came back true), each detected tool's own --version, and whether the process is running inside a Flatpak, toolbox, or other container -- which changes what every other probe on this list actually sees. Add --json for a machine-readable form.

pyguitest inspect walks the accessible tree of every open window and prints it, grouped by application -- the tool for seeing what gui.button(...) or gui.element(role=..., name=...) actually has to match against, without writing a script first. --window TITLE_REGEX narrows it to one application; --json gives the same tree as machine-readable data, the same split debug uses.

The migration scanner reports the tier of every X11::GUITest call in a source file and exits non-zero if any call has no Wayland path, so a port can be gated in CI.

pyguitest record is an alias for pyguitest-recorder, which records desktop activity and writes the pyguitest script for it. That tool is a separate package -- pyguitest does not depend on it, and installing pyguitest does not install it -- so the subcommand reports how to install it if it is absent. Everything after record is passed straight through, so pyguitest record --help is the recorder's own help and every one of its flags works unchanged.

Documentation

Start here

• docs/getting-started.md — five minutes to a working script, which API to reach for, and what X11, Wayland and XWayland each change
• docs/recipes.md — task-shaped answers: waiting properly, forms, windows, screenshots, CI, and an X11::GUITest cheat sheet
• docs/troubleshooting.md — symptom first: nothing found, nothing typed, nothing captured

Reference

• docs/api.md — the full API reference: every public class, method and enum, with the capability each one needs
• docs/install.md — what each backend needs, per distribution, and how capture picks a path
• docs/input.md — injecting pointer and keyboard input: permissions, daemons, keymap safety, libei and the portal
• docs/validation.md — what has been run against a real desktop, and what has not
• docs/ai-assistants.md — rules for a coding assistant generating pyguitest code
• testable-guis.md — how to build a GUI that can be tested at all: the accessibility work that lets a test name a button instead of clicking a coordinate. Written to be handed to application developers; lives in the recorder repository

Design and internals — docs/developers/: why the API is not a port, the audit of all 50 X11::GUITest exports it derives from, the two ADRs, the repository structure, and the protocol gaps worth taking upstream.

Contributing

Tests, lint, types, CI and the D-Bus suite: CONTRIBUTING.md.

License

GPL-2.0-or-later. See LICENSE.