py-a11y-ui 0.1.0

A cross-platform UI automation adapter for AI Agents.

# 👁️ PyA11y (Python Accessibility Adapter)

**A cross-platform UI automation library designed for AI Agents.**

PyA11y bridges the gap between AI models (like GPT-4, Claude) and operating system UIs. Instead of relying on fragile coordinate-based clicks or expensive screenshot analysis, PyA11y hooks directly into the OS Accessibility API to generate a semantic, lightweight JSON representation of the screen.

> **Current Status:**
> * ✅ **Windows:** Fully supported (via UIAutomation)
> * 🚧 **Linux:** In Development (planned X11/Atspi support)
> * 📅 **macOS:** Planned

## 🚀 Installation

```bash
pip install pya11y

⚡ Quick Start

PyA11y separates the "Driver" (the OS interface) from your logic. Here is how to snapshot the current window:

import time
from pya11y import get_ui_driver, ScanConfig

# 1. Configure the scanner
# Scan 10 levels deep, ignore clutter like ScrollBars
config = ScanConfig(max_depth=10)

# 2. Get the driver for your OS
driver = get_ui_driver(config)

print("Switch to your target app in 3 seconds...")
time.sleep(3)

# 3. Capture the semantic tree
ui_data = driver.capture()

# 4. Use the data
print(f"Captured Window: {ui_data['name']}")

if 'children' in ui_data and len(ui_data['children']) > 0:
    first_child = ui_data['children'][0]
    print(f"First Element: {first_child['name']}")
    print(f"Coordinates: {first_child['bbox']}")

🧠 Why PyA11y?

Most automation tools (like PyAutoGUI) are "blind"—they just click X,Y coordinates. If a window moves, they fail.

PyA11y provides Context:

1. Semantic Tree: It tells you "There is a 'Submit' button at [200, 400]".
2. Token Efficient: Outputs clean JSON, not heavy screenshots.
3. Resilient: If the window moves, the underlying tree updates automatically.

🔧 Configuration

You can fine-tune the scanner to save tokens or get more detail:

config = ScanConfig(
    max_depth=12,              # How deep to scan nested elements
    include_offscreen=False,   # Ignore hidden elements to save space
    require_text=True          # Ignore elements with no labels
)

🤝 Contributing

We are actively looking for contributors to help build the Linux and macOS adapters!

1. Fork the repo
2. Create your feature branch (git checkout -b feature/linux-adapter)
3. Submit a Pull Request