Rust-accelerated PyAutoGUI 0.9.54

This fork preserves the established PyAutoGUI 0.9.54 API while adding an optional user-mode Rust fast path on Windows.

import pyautogui

pyautogui.moveTo(500, 300, duration=0.2)
pyautogui.click()
pyautogui.write("hello", interval=0.03)
image = pyautogui.screenshot(region=(0, 0, 800, 600))
box = pyautogui.locateOnScreen("button.png", confidence=0.9)

No Rust-specific import or new calling convention is required. If the extension is absent or one native operation fails, that operation silently returns to the original Win32 ctypes or PyScreeze implementation.

Windows fast paths

• Batched SendInput keyboard and mouse events
• Virtual-desktop coordinates and negative monitor origins
• Per-monitor V2 DPI awareness
• Reusable CreateDIBSection GDI capture contexts
• Parallel hierarchical grayscale SAD template matching
• Deadline-based smooth movement with scoped 1 ms timer resolution
• Low-level failsafe monitoring
• GIL release around native capture, matching, input, and movement

The project contains no kernel driver, service, test-signing workflow, or special boot requirement.

Universal visual location

Still images, GIF representative frames, DPI/zoom variants, and moving targets share an adaptive locate pipeline. Large or highly changed references can use the external 359k-parameter TinyLocate network. Its framework-free CUDA runtime is compiled into the Rust extension; the 721 KiB model remains a separate file. High-confidence neural hits contribute a small, bounded in-memory appearance bank, so a moving or animated target can adapt without writing captures to disk.

python tools/install_tinylocate_model.py path\to\tinylocate-v1.tln
python tools/smoke_tinylocate_runtime.py path\to\tinylocate-v1.tln

Existing calls stay unchanged:

box = pyautogui.locateOnScreen("target.png", confidence=0.8)
boxes = list(pyautogui.locateAllOnScreen("target.gif", confidence=0.8))

Model training uses PyTorch only in the development environment and exports the custom TLN1 FP16 container. Runtime loads neither PyTorch nor an ONNX engine. See PRODUCT_VISION.md for routing, accuracy, and latency gates.

Requirements

• Python 3.9+
• Windows wheels use CPython ABI3
• Building from source on Windows requires stable Rust, MSVC Build Tools, and a Windows SDK

Optional vision dependencies are available with:

python -m pip install -e ".[vision]"

Build and test

python -m pip install -e ".[dev,benchmark]"
python tools/check.py --build

The default test suite is hermetic and does not emit operating-system input. See DEVELOPING.md for architecture, benchmarks, CI, and release checks.

Upstream PyAutoGUI documentation remains applicable to the public API: https://pyautogui.readthedocs.io/