alphawindow 0.1.1

Windows-first window automation abstractions with pluggable input and output modes.

AlphaWindow

AlphaWindow is a Windows-first automation abstraction layer. It exposes one session API for window operations while allowing callers to choose different input and output modes.

AlphaWindow is meant to model target-window automation, not global "computer use" control. The same high-level operations can be backed by no-op inspection, window messages, UI Automation, foreground input, guarded foreground input, or optional hook-assisted virtual input.

Scope

AlphaWindow separates four concerns:

• Window discovery and state: WindowResolver, WindowSelector, and WindowSnapshot.
• Output: state inspection, direct window capture, desktop-region capture, hook telemetry, or no output.
• Input: dry-run, window messages, UIA, global foreground input, guarded foreground input, isolated hook input, or virtualized hook input.
• Safety contracts: profile compatibility, explicit fallback modes, hook requirements, prefetch TTL, and fake backends for deterministic tests.

The project currently focuses on the abstraction and compatibility layer. Native Win32, pynput, UIA, and hook implementations are intentionally kept behind protocols so callers can plug in their own adapters without changing automation code. The exception is an experimental PyWinRT-backed Windows Graphics Capture path that can capture a native Direct3D surface, read it back to CPU memory, or keep a persistent frame cache for low-latency repeated screenshots.

Supported Today

• Built-in I/O profiles for every declared input and output mode, including inspect, observe, desktop_observe, dry_run, win32_message, message_only, uia_control, uia_visual, global_input, foreground_visual, guarded_input, guarded_desktop, isolated_observe, isolated_telemetry, virtual_window, virtual_telemetry, and virtual_input_only.
• A WindowSession API for prefetch, capture, observe, click, key_down, key_up, and type_text.
• Generic delegation backends for message, UIA, foreground, guarded foreground, state, desktop-region, hook-telemetry, and no-output modes.
• Compatibility checks so a backend cannot silently fall back to a broader side-effect mode.
• Hook-aware isolated and virtualized input backends that accept pluggable hook managers and virtual input transports.
• Deterministic prefetch behavior: snapshots are reused within prefetch_ttl_seconds, rejected when stale, and revalidated through a resolver is_valid hook when provided.
• DPI-aware snapshots and capture requests: WindowSnapshot.dpi is carried into operation normalization and DirectX capture requests.
• Relative point normalization: session.click(0.5, 0.5) resolves against the current target window rectangle.
• HDR-aware DirectX/Windows Graphics Capture model: DirectXCaptureOutputBackend builds capture requests for an injected DirectXCapturePipeline, selects FP16/scRGB for HDR targets, selects BGRA8/sRGB for SDR, and preserves frame DPI and SDR white-level metadata.
• CPU frame conversion: CpuFrame supports FP16/scRGB to SDR BGRA8 tone mapping and pure-Python PNG export.
• Experimental native WGC pipeline: WinrtWgcCapturePipeline uses PyWinRT, creates a Direct3D11 device, creates a GraphicsCaptureItem from hwnd, starts a free-threaded WGC frame pool, returns a CaptureFrame containing the native IDirect3DSurface, and can read it back with capture_cpu().
• Persistent native WGC frame cache: WinrtWgcFrameCache keeps the WGC session and D3D11 device alive, tracks the newest frame, can wait for a bounded fresh frame, and can run a before_readback hook while holding that frame so external labels can be sampled against the same image moment.
• PyPI packaging for pip install alphawindow and typed package data via py.typed.
• Tests that cover profile coverage, compatibility failures, prefetch reuse/refresh behavior, hook backend contracts, DirectX/HDR capture contracts, native WGC dependency behavior, generic backends, packaging import behavior, and the publish workflow.

Built-in Profiles

Profile	Input mode	Output mode	Current behavior
inspect	dry_run	state	Inspect resolver state only.
observe	dry_run	window_capture	Observe/capture without input.
desktop_observe	dry_run	desktop_region	Observe through desktop-region output.
dry_run	dry_run	none	Record planned operations without side effects.
win32_message	message	window_capture	Adapter slot for window-message input plus window capture.
message_only	message	none	Adapter slot for message input without output.
uia_control	uia	state	Adapter slot for UIA control operations.
uia_visual	uia	window_capture	Adapter slot for UIA input plus visual capture.
global_input	foreground	desktop_region	Adapter slot for pynput/SendInput-style foreground input.
foreground_visual	foreground	window_capture	Foreground input with target-window output.
guarded_input	guarded_foreground	window_capture	Adapter slot for foreground input with focus/cleanup guards.
guarded_desktop	guarded_foreground	desktop_region	Guarded foreground input with desktop-region capture.
isolated_observe	isolated	window_capture	Requires a hook-capable input backend; can isolate real mouse input before optional delegated input.
isolated_telemetry	isolated	hook_telemetry	Requires hook support for isolation and telemetry.
virtual_window	virtualized	window_capture	Requires hook support and a virtual input transport.
virtual_telemetry	virtualized	hook_telemetry	Requires hook support for virtual input and telemetry.
virtual_input_only	virtualized	none	Requires hook support; routes virtual input without output capture.

Considered Design Surface

These capabilities are represented in the model and can be supplied by external adapters today:

• Native window resolution by hwnd, title, class name, PID, visibility, DPI, and process/thread metadata.
• Direct window output such as PrintWindow, DWM thumbnails, BitBlt, or application-specific capture.
• HDR-aware DirectX/Windows Graphics Capture output with explicit pixel format, color space, buffer count, free-threaded frame-pool preference, DPI, and SDR white-level metadata.
• Native WGC capture through PyWinRT for callers that want an IDirect3DSurface, a tone-mapped CPU image, or a persistent latest-frame cache for high-frequency capture loops.
• Desktop-region output for modes where the target must be foreground or compositor-visible.
• Window-message input for controls that accept PostMessage/SendMessage style input.
• Windows UI Automation input for standard controls.
• Foreground input through pynput, SendInput, or equivalent global input libraries.
• Guarded foreground input that restores focus/cursor state after an operation.
• Optional hook-assisted mouse and keyboard windowization, including target-local telemetry.
• Per-window or per-process virtual input transports for multi-window automation without taking over the primary mouse.

Not Supported Yet

The package still keeps most native adapters out of core. In particular, it does not currently include:

• A built-in pywin32 WindowResolver.
• Color-managed HDR preview or HDR-preserving image/video encoders.
• Advanced tone-mapping controls beyond the built-in ACES-style SDR conversion.
• Built-in PostMessage, UIA, pynput, or SendInput input backends.
• A production DLL injection or hook manager.
• A ready-made virtual input IPC transport.
• OCR, image matching, accessibility tree querying, or high-level wait/retry helpers.
• Non-Windows native backends.
• Kernel drivers, privilege escalation, protected-process bypasses, anti-cheat bypasses, or game-specific automation logic.

Hook-capable profiles fail closed unless the provided backend advertises hook capability. VirtualizedInputBackend also requires a VirtualInputTransport.

Planned Features

• pywin32-based resolver and state backend for common window selectors.
• Native capture adapters for classic window capture and desktop-region capture.
• Broader Windows Graphics Capture resize handling.
• HDR-preserving image encoders and richer color-management metadata.
• pynput/SendInput foreground input backend and guarded foreground focus restoration.
• Window-message and UIA adapters for common controls.
• A documented hook manager interface with a sample user-mode hook implementation.
• A sample virtual input transport suitable for target-window mouse/keyboard windowization.
• More examples showing how to share one automation script across dry-run, message, foreground, isolated, and virtualized modes.
• Expanded integration tests for real Windows adapters while keeping unit tests deterministic through fake backends.

Install

From PyPI:

pip install alphawindow

From this checkout:

pip install .

Native Windows adapters are optional:

pip install "alphawindow[native]"

Import

import alphawindow

from alphawindow import (
    AutomationConfig,
    DirectXCaptureConfig,
    WgcFrameReadbackResult,
    WinrtWgcCapturePipeline,
    WinrtWgcFrameCache,
    InputMode,
    OutputMode,
    WindowSelector,
    WindowSession,
    get_profile,
)

print(alphawindow.__version__)
print(get_profile("observe").input_mode is InputMode.DRY_RUN)
print(DirectXCaptureConfig().hdr_mode.value)
print(WgcFrameReadbackResult.__name__)
print(WinrtWgcCapturePipeline.__name__)
print(WinrtWgcFrameCache.__name__)

Native WGC Screenshot

from alphawindow import DirectXCaptureConfig, HdrCaptureMode, WindowSnapshot
from alphawindow.native_wgc import WinrtWgcCapturePipeline

target = WindowSnapshot(
    hwnd=123456,
    title="Target",
    rect=(0, 0, 1920, 1080),
    dpi=144,
    metadata={"hdr_enabled": True, "sdr_white_level_nits": 203.0},
)
request = DirectXCaptureConfig(hdr_mode=HdrCaptureMode.AUTO).create_request(target)

pipeline = WinrtWgcCapturePipeline(timeout_seconds=2.0)
frame = pipeline.capture_cpu(target, request, tone_map=True)
frame.save_png("target.png")
pipeline.close()

For repeated capture, keep the WGC session alive and read back the latest frame:

from alphawindow.native_wgc import WinrtWgcFrameCache

def sample_labels(frame_info):
    return {"sequence": frame_info["sequence"], "labels": []}

cache = WinrtWgcFrameCache.start_for(target, request, timeout_seconds=2.0)
try:
    result = cache.readback_latest_cpu(
        tone_map=True,
        fresh_max_age_ms=8.0,
        fresh_wait_ms=16.0,
        before_readback=sample_labels,
    )
    result.cpu.save_png("target.png")
    print(result.info["size"], result.info["content_size"], result.hook_result)
finally:
    cache.close()