haindy 0.1.0

Autonomous AI Testing Agent with multi-agent architecture

HAINDY

Computer-use for coding agents. HAINDY gives AI coding tools (Claude Code, Codex CLI, OpenCode, and others) the ability to see the screen, click, type, and test real applications across desktop, Android, and iOS.

pip install haindy
haindy setup

What it does

Your coding agent calls HAINDY to interact with a live UI. Every command returns structured JSON.

haindy session new --desktop                                       # start a session
haindy act "click the Login button" --session <ID>                 # execute an action
haindy test "sign in and verify the dashboard loads" --session <ID> # run a multi-step test
haindy screenshot --session <ID>                                   # capture current state
haindy session close --session <ID>                                # clean up

Under the hood, each action goes through a computer-use AI provider (OpenAI, Google Gemini, or Anthropic Claude) that takes a screenshot, reasons about the UI, and performs real OS-level input -- mouse, keyboard, scroll -- against the actual application. No DOM hooks, no selectors, no browser automation.

Supported platforms

Platform	Automation method
Linux/X11	uinput + xdotool + ffmpeg
macOS	pynput + mss
Android	ADB
iOS	idb

haindy session new --desktop
haindy session new --android --android-serial emulator-5554
haindy session new --ios --ios-udid <UDID>

Agent integration

HAINDY ships with bundled skills that haindy setup installs automatically for detected AI CLIs. Once installed, your coding agent can discover and use HAINDY directly.

Supported CLIs: Claude Code, Codex CLI, OpenCode.

act vs test

• act -- execute a single action ("click the submit button", "type hello into the search field")
• test -- plan and execute a multi-step scenario with outcome validation ("sign in, navigate to settings, verify the dark mode toggle works")

Use test whenever you care about verifying a result, not just performing an action.

Session variables

Store values your agent can reference across commands:

haindy session set USERNAME [e-mail] --session <ID>
haindy session set --value-file credentials.txt --secret --session <ID>
haindy session vars --session <ID>

Batch mode

HAINDY also includes a pipeline of specialized AI agents that can plan and execute tests autonomously from a requirements file -- no coding agent required:

Requirements ──> Scope Triage ──> Test Planner ──> Situational Agent ──> Test Runner ──> Report

haindy run --plan requirements.txt --context context.txt
haindy run --mobile --plan requirements.txt --context context.txt    # Android
haindy run --ios --plan requirements.txt --context context.txt       # iOS

This produces an HTML report with screenshots, pass/fail results, and a JSONL execution log.

Configuration

Credentials

haindy auth login openai      # stored in system keychain
haindy auth login google
haindy auth login anthropic
haindy auth status             # verify

Providers

HAINDY uses two providers independently: one for planning/analysis, one for computer-use actions.

haindy provider set openai                   # planning/analysis
haindy provider set-computer-use google      # computer-use

Settings file

Create ~/.haindy/settings.json for persistent non-secret configuration:

{
  "agent": { "provider": "openai" },
  "computer_use": { "provider": "google" },
  "openai": { "model": "gpt-5.4", "computer_use_model": "gpt-5.4" },
  "google": { "model": "gemini-3-flash-preview", "computer_use_model": "gemini-3-flash-preview" },
  "anthropic": { "model": "claude-sonnet-4-6", "computer_use_model": "claude-sonnet-4-6" },
  "execution": { "automation_backend": "desktop" },
  "logging": { "level": "INFO" }
}

Environment variables override all other sources. See .env.example for the full list.

Platform prerequisites

Platform	Requirements
Linux/X11	ffmpeg, xdotool, xclip, /dev/uinput access
macOS	Grant Accessibility + Screen Recording to your terminal (System Settings > Privacy & Security)
Android	adb installed, device/emulator reachable
iOS (macOS)	brew install idb-companion, device paired

haindy doctor checks all of these for you. See docs/RUNBOOK.md for detailed setup.

Development

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.lock
pip install -e ".[dev]"

ruff check .          # lint
ruff format --check . # format check
mypy haindy           # type check
pytest                # tests

Architecture

Directory	Purpose
haindy/tool_call_mode/	Tool-call CLI, daemon, IPC, session state
haindy/agents/computer_use/	Multi-provider computer-use session orchestrator
haindy/agents/	Scope triage, test planner, situational, action, and test runner agents
haindy/desktop/	Linux/X11 automation (uinput, xdotool, ffmpeg)
haindy/macos/	macOS automation (pynput, mss)
haindy/mobile/	Android (ADB) and iOS (idb) automation
haindy/config/	Settings, env vars, settings file loader
haindy/orchestration/	Multi-agent workflow coordination
haindy/monitoring/	JSONL logging, HTML report generation

Contributing

See CONTRIBUTING.md for development guidelines and how to submit changes.

License

MIT