droid-use 0.1.0

Android automation toolkit with screenshot grounding, ADB actions, and Airtest helpers for AI-authored scripts.

droid-use

droid-use is a Python package for Android UI automation driven by screenshots, grounding, and ADB.

It focuses on two related workflows:

• Direct Android device control through ADB helpers such as screenshot capture, tap, long press, swipe, text input, and key events.
• Screenshot grounding that turns a natural-language instruction plus a phone screenshot into a concrete device action or bounding box, with visual overlays and evaluator feedback loops.

It also includes Airtest-oriented utilities so an AI agent can help write or refine Android automation scripts that rely on visual matching and OCR.

Main capabilities

• ADB-backed Android interaction helpers
• Screenshot-based coordinate grounding for taps, long presses, swipes, and bbox selection
• Coordinate conversion helpers for pixel and normalized coordinate systems
• Grounding orchestration with operator/evaluator loops and artifact logging
• Airtest helper utilities for AI-authored automation scripts

Installation

pip install droid-use

For local development:

pip install -e .

Example

from droid_use import get_screenshot, interact_with_screen

get_screenshot("/tmp/screen.png")
result = interact_with_screen(
    "/tmp/screen.png",
    "点击底部中间的登录按钮",
    max_rounds=3,
)

print(result)

Airtest-oriented usage

The package also ships helper modules under droid_use.airtest_utils for workflows where an AI agent writes or edits Airtest-style Android automation scripts.