apppilot 1.2.26

Modern, Intelligent Mobile Automation Framework (Compiled)

🚀 AppPilot Mobile — The Master Guide

Welcome to the world-class documentation for AppPilot Mobile. This framework is designed to be the "Playwright for Mobile" — a Python-first, human-readable automation engine that handles the complexity of mobile UI so you can focus on writing tests.

---

🏗️ 1. Professional Architecture (POM)

AppPilot is optimized for the Page Object Model (POM). By following this structure, your tests will be readable, maintainable, and stable.

1.1 BaseWindow — The Core Elements

Define common system-level elements (buttons, toolbars) here so every page can use them without re-locating.

class BaseWindow:
    def __init__(self, mobile: Mobile):
        self.mobile = mobile
        
    # Standard buttons found across many screens
    @property
    def ok(self): return self.mobile.button("OK")
    
    @property
    def cancel(self): return self.mobile.button("Cancel")
    
    @property
    def save(self): return self.mobile.button("Save")
    
    @property
    def back_btn(self): return self.mobile.button("Back")

    def go_back(self):
        self.mobile.press_back()
        return self

1.2 BasePage — Smart Synchronization

All specific app pages inherit from BasePage.

class BasePage(BaseWindow):
    def __init__(self, mobile: Mobile):
        super().__init__(mobile)
        
    def wait_for_load(self):
        """Standard sync point for every page transition."""
        self.mobile.wait_for_page_ready()
        return self

1.3 Specific Page (e.g., SettingsPage)

class SettingsPage(BasePage):
    def open_display_settings(self):
        self.mobile.tap("Display")
        return DisplaySettingsPage(self.mobile)
        
    def toggle_wifi_off(self):
        # Using the base button if 'Save' appears
        self.mobile.checkbox("Wi-Fi").uncheck()
        if self.mobile.find_text("Apply"):
            self.save.tap() # Clean access to self.save from BaseWindow

---

🧠 2. Smart & AI-Powered Features

AppPilot goes beyond basic automation by using an Intelligent Layer to handle real-world mobile UI challenges.

2.1 Intelligent Locator Engine (Scoring & Dynamic Selection)

Stop fighting with duplicate IDs or "distractor" elements. AppPilot uses a scoring engine to find exactly what you want.

• Example: If you search for "Search", it prioritizes the clickable search button and ignores "Search History" or "Search Labs" automatically.
• Priority: Accessibility ID > Visible Text > Resource ID > Content Description.

Handling Elements Without Text Dynamically

You do not need separate methods (like tap_xpath() or tap_by_id()) for elements without text. The exact same public methods (mobile.tap(), mobile.fill(), etc.) handle all targets dynamically:

• Accessibility ID / Content Description:

  mobile.tap("shopping_cart_icon")  # Taps by accessibility label even if there is no text.
  

• Resource ID:

  mobile.tap("com.example.app:id/submit_btn")  # Identifies the developer-assigned element ID.
  

• Placeholder / Hint Text:

  mobile.fill("Enter your password", "secret123")  # Fills inputs identified by empty placeholder text.
  

• XPath Fallback:

  mobile.tap("//android.widget.FrameLayout[1]/android.widget.Button[2]")  # Triggers automatically if starting with "//".
  

2.2 Auto-Scroll Search

Never worry if an element is off-screen. Methods like tap() and fill() will automatically:

1. Check if the element is visible.
2. If not, perform a smart swipe (Samsung optimized).
3. Retry the search.
4. Fail only after 5 failed scrolls (configurable).

2.3 AI Test Generation (TEST_GEN_PROMPT.md)

Every project initialized with apppilot setup includes a specialized AI prompt.

1. Copy the contents of TEST_GEN_PROMPT.md.
2. Paste it into ChatGPT or Gemini.
3. Describe your test case (e.g., "Login, then go to Profile and change name").
4. The AI will generate a complete, valid AppPilot Python test for you.

---

🖥️ 3. Live Monitoring & Evidence

3.1 Live Desktop Mirroring

See your phone's screen in a window on your desktop during execution using the integrated scrcpy support.

• Enable: Set mirror_screen: true in your apppilot.yaml.

3.2 Automated Evidence

• Hardware-Accelerated Recording (MP4): Every test is recorded with zero performance drop. Includes professional 1s padding.
• Smart Screenshots: Captures specific UI states into .apppilot/screenshots/.

---

🛠️ 4. Developer & Diagnostic Tools

AppPilot includes powerful tools to help you debug and optimize your automation scripts in real-time.

4.1 UI Inspector (mobile.inspect())

Prints a detailed, categorized map of every element on the screen.

• Categorization: Automatically identifies INPUT, BUTTON, TEXT, IMAGE, CHECKBOX, and more.
• Precision: Includes exact screen bounds [x1,y1][x2,y2] for every element.
• Usage:

mobile.inspect()

Output Example:

[1] BUTTON: text='', desc='Google Search', id='', bounds='[45,744][182,882]', class='android.widget.Button'
[2] INPUT: text='', desc='', id='', bounds='[180,744][630,882]', class='android.widget.EditText'

4.2 Device Diagnostic (mobile.diagnose())

Provides a complete health check of the device and the automation environment.

• System Info: Model, Android version, and Serial.
• App State: Package and Activity currently in focus.
• Performance: Real-time Memory (MB) and CPU (%) usage.
• Usage:

mobile.diagnose()

4.3 Relational Selectors (Maestro-Style)

Find elements relative to other elements when they lack clear IDs or text.

• Methods: below(), above(), left_of(), right_of().
• Usage:

# Fill an input field to the right of the "Search" button
mobile.right_of("Search").input().fill("AppPilot Mobile")

# Tap a button below a specific label
mobile.below("Username").button("Login").tap()

4.4 Programmatic Element Attributes

Retrieve detailed properties and visual characteristics of elements dynamically. Methods return a dictionary with text, bounds, clickable state, average color, and relative coordinates.

• Methods: get_element_attributes(target), get_focused_element(), get_active_element(), get_selected_element().
• Usage:

# Retrieve all properties of an element (returns a dict)

  ### Code Example

    def test_element_details(mobile):
        # 1. Launch your application
        mobile.launch_app("com.jio.consumer.jiothings")
    
        # 2. Get attributes of a specific element by text, ID, or description
        card_attributes = mobile.get_element_attributes("Tap this card to start Vehicle Sharing")
    
        # The method returns a standard Python dictionary
        print("\n--- Card Attributes ---")
        print(f"Class Name:          {card_attributes.get('class')}")
        print(f"Text Value:          {card_attributes.get('text')}")
        print(f"Resource ID:         {card_attributes.get('resource-id')}")
        print(f"Accessibility Desc:  {card_attributes.get('content-desc')}")
        
        # Coordinates & Layout properties
        bounds = card_attributes.get('bounds')  # returns tuple (x1, y1, x2, y2)
        print(f"Coordinates Bounds:  {bounds}")
        print(f"Relative Center:     X={card_attributes.get('rel_cx') * 100:.1f}%, Y={card_attributes.get('rel_cy') * 100:.1f}%")
        
        # State flags
        print(f"Clickable state:     {card_attributes.get('clickable')}")
        print(f"Checked state:       {card_attributes.get('checked')}")
        print(f"Focused state:       {card_attributes.get('focused')}")
        print(f"Enabled state:       {card_attributes.get('enabled')}")
        
        # Visual & Color analysis
        print(f"Avg Hex Color:       {card_attributes.get('avg_color')}")     # e.g., '#e9e9e9'
        print(f"Avg RGB Colors:      {card_attributes.get('avg_rgb')}")       # e.g., (233, 233, 233)
        print(f"Image Complexity:    Variance={card_attributes.get('variance')}")
        print(f"Icon/Image detected: {'Yes' if card_attributes.get('is_complex') else 'No'}")
    
    
        # 3. Retrieve properties of the currently active/focused element
        active_el = mobile.get_focused_element()  # or mobile.get_active_element()
        
        print("\n--- Focused Element ---")
        print(f"Active Class:        {active_el.get('class')}")
        print(f"Active Text:         {active_el.get('text')}")
        print(f"Is Focused:          {active_el.get('focused')}")
        # Chained One-Liner Example:
        #You can also chain everything together in a single line:

        # Directly get the parent card's background color starting from the child title's Resource ID
        parent_color = mobile.element("com.jio.consumer.jiothings:id/tv_title_share_vehicle").parent().get_attributes().get("avg_color")
        print(f"Parent Card Background Color: {parent_color}")
         # 2. Child of Child (Grandchild element - e.g. first child of the first child)
        # You can specify indexes (0-indexed) for child navigation
        grandchild_element = element.child(0).child(0)
        grandchild_attrs = grandchild_element.get_attributes()
        
        print("\n--- Grandchild Attributes ---")
        print(f"Class: {grandchild_attrs.get('class')}")
        print(f"Text:  {grandchild_attrs.get('text')}")

        # 3. Sibling navigation (getting the next or specific sibling by index)
        # Get the second sibling element of the title text element:
        second_sibling = mobile.element("com.jio.consumer.jiothings:id/tv_title_share_vehicle").sibling(1)
        print(f"Second Sibling Text: {second_sibling.get_attributes().get('text')}")
        
        # Get all siblings of the element:
        all_siblings = mobile.element("com.jio.consumer.jiothings:id/tv_title_share_vehicle").siblings()
        print(f"Total Siblings Found: {len(all_siblings)}")

        # 4. Mixing Parent, Child, and Sibling chaining
        # Go up to the parent, locate a sibling at index 2, and tap it:
        sibling_sub_element = element.parent().sibling(2).child(0)
        sibling_sub_element.tap()
    

---

📊 5. Complete API Reference

Category	Method	Example	Description
Core	tap(target) / click()	mobile.click("Login")	Smart tap/click with auto-scroll.
	fill(target, text) / set_text()	mobile.set_text("User", "admin")	Locates label and fills input.
	clear(target) / clear_text()	mobile.clear_text("Search")	Wipes text from a field.
	double_tap(t) / double_click()	mobile.double_click("Map")	Rapid two-tap gesture.
	long_press(target)	mobile.long_press("File")	1s touch-and-hold.
	dropdown(target)	mobile.dropdown("Theme").select(option="Dark")	Opens dropdown and selects by option text or index.
Diagnostics	inspect()	mobile.inspect()	Prints UI tree to console.
	diagnose()	mobile.diagnose()	Full device & app health check.
	get_current_app()	app = mobile.get_current_app()	Returns package/activity.
	get_performance()	perf = mobile.get_performance()	Returns CPU/Memory stats.
	get_focused_element()	active = mobile.get_focused_element()	Gets properties of currently focused element.
	get_active_element()	active = mobile.get_active_element()	Alias for get_focused_element().
	get_selected_element()	active = mobile.get_selected_element()	Alias for get_focused_element().
Device Mgmt	connect_device(serial)	mobile.connect_device("192.168.1.5:5555")	Connects via ADB TCP/IP.
	disconnect_device()	mobile.disconnect_device()	Disconnects device.
	get_device_info()	info = mobile.get_device_info()	OS, Model, Manufacturer.
	install_app(path)	mobile.install_app("app.apk")	Installs APK.
	uninstall_app(pkg)	mobile.uninstall_app("com.app")	Uninstalls package.
	close_app(pkg)	mobile.close_app("com.app")	Force stops application.
	clear_app_data(pkg)	mobile.clear_app_data("com.app")	Resets app state.
	grant_permissions()	mobile.grant_permissions(pkg, ["camera"])	Auto-allow permissions.
	set_location(lat, lon)	mobile.set_location(12.9, 77.5)	Spoofs GPS coordinates.
	set_orientation(mode)	mobile.set_orientation("landscape")	Portrait/Landscape switch.
Keyboard	send_keys(text)	mobile.send_keys("Hello")	Direct text to focused field.
	hide_keyboard()	mobile.hide_keyboard()	Dismisses soft keyboard.
	show_keyboard()	mobile.show_keyboard()	Forces keyboard to open.
	press_key(key)	mobile.press_key("ENTER")	Hardware key (BACK, HOME, etc).
Gestures	swipe(x1,y1,x2,y2)	mobile.swipe(100,500,900,500)	Raw coordinate movement.
	scroll_to_text(text)	mobile.scroll_to_text("Accept")	Scrolls until text found.
	drag_and_drop(s, t)	mobile.drag_and_drop("App", "Bin")	Moves element.
	flick(x1,y1,x2,y2)	mobile.flick(500,800,500,200)	Fast swipe gesture.
Discovery	get_page_source()	xml = mobile.get_page_source()	Returns UI hierarchy XML.
	get_all_texts()	mobile.get_all_texts_displayed()	Returns all visible strings.
	get_text(target)	mobile.get_text("Title")	Reads text from element.
	get_element_attributes(t)	attrs = mobile.get_element_attributes("Btn")	Gets detailed layout, state, and visual properties.
	get_attributes()	attrs = mobile.element("Btn").get_attributes()	Chained attributes retriever for elements.
Media	take_screenshot(n)	mobile.take_screenshot("err")	Captures full screen.
	capture_element(t,n)	mobile.capture_element_screenshot("Btn","c")	Crops to element bounds.
	start_recording()	mobile.start_screen_recording()	Captures MP4 video.
	capture_photo()	mobile.capture_photo()	Shutter + Auto-OK.
State	is_enabled(target)	mobile.is_enabled("Submit")	Clickable state check.
	is_visible(target)	mobile.is_visible("Title")	Visibility check.
Wait	wait_for_page_ready()	mobile.wait_for_page_ready()	Waits for UI stability.
	wait_until_visible()	mobile.wait_until_visible("OK")	Polling wait until visible.
	wait_until_gone()	mobile.wait_until_gone("Load")	Polling wait until vanished.
Validation	verify_text(text)	mobile.verify_text("Saved")	Assert exact text exists.
	verify_present(t)	mobile.verify_element_present("id")	Assert element exists.
Advanced	execute_shell()	mobile.execute_shell_command("ls")	Raw shell access.
	execute_adb()	mobile.execute_adb("devices")	Raw ADB command.
	find_by_image()	mobile.find_by_image_click("p.png")	Visual Match + Click.

---

🏗️ 6. CLI Tooling Reference

Command	Example	Description
init	apppilot init	Scaffolds a new project.
setup	apppilot setup --project_folder tests	Initializes project in specific folder.
--overwrite	apppilot setup --overwrite	Overwrites apppilot.yaml with latest defaults.
doctor	apppilot doctor	Dependency and tool path health check.
devices	apppilot devices	Lists all connected device serials.
interactive	apppilot interactive	Live REPL: Run commands one-by-one.
record	apppilot record	Visual Live Inspector & Recorder: Point-and-click to inspect nodes and record tests.
run	apppilot run	Standard test execution.

| report | apppilot report | Generates/Serves HTML test report. |

---

🎨 6.5 Visual Live Inspector & Recorder (apppilot record)

AppPilot includes a graphical Point-and-Click Test Recorder designed to write tests visually without looking up complex element paths or coordinates.

How to Use the Visual Recorder:

1. Connect your Android device and confirm it is detected using apppilot devices.
2. Start the recorder:

   apppilot record
   

3. Interactive Inspection:
   • Move your mouse over the screen mirror to see element boundaries highlighted in Red.
   • Left-click on any element to select it (highlighted in Blue).
4. Choose Your Selector & Action:
   • If multiple elements overlap, use the Clicked Elements Listbox to pick the correct child/parent node.
   • Choose your preferred locator strategy (Text, ID, Short ID, or relative XPath) in the Locator Options Listbox.
   • Click Tap or Fill Text to perform the action live on the physical device. The screen will automatically refresh after execution.
5. Generate Script:
   • Your actions are logged live. Click Copy to Clipboard or Save to File to retrieve the complete generated Python test script (recorded_test.py).

---

📂 7. Configuration (apppilot.yaml)

# apppilot.yaml
tools:
  adb_path: 'C:\path\to\adb.exe'
  scrcpy_path: 'C:\path\to\scrcpy.exe'
  log_path: '.apppilot/logs'  # Custom log folder

mirror_screen: true
auto_scroll: true
timeout: 30
recorder_refresh_delay: 1.5  # Time (in seconds) to wait for screen transitions before refreshing the recorder view

Note for Windows Users: Always use single quotes ' for paths to avoid errors with backslashes.

---

🛠️ 8. Discovery & Debugging Example

When you are automating a new app and don't know the IDs, use this "Discovery Script" to find everything you need.

import pytest
from apppilot.api.mobile import Mobile

def test_discovery_session(mobile):
    # 1. Launch the app and check health
    mobile.launch_app("com.android.chrome")
    mobile.diagnose()  # Prints CPU/Memory and current Activity
    
    # 2. Navigate to a complex page
    mobile.navigate("https://www.google.com")
    mobile.wait_for_page_ready()
    
    # 3. Inspect the UI to find hidden locators
    # This will print every INPUT, BUTTON, and TEXT with coordinates
    mobile.inspect()
    
    # 4. Use Relational Selectors for elements without IDs
    # "Find the input to the right of the Google Search icon"
    mobile.right_of("Google Search").input().fill("AppPilot Mobile")
    
    # 5. Check performance after interaction
    perf = mobile.get_performance()
    print(f"Current App Memory Usage: {perf['memory_mb']} MB")
    
    mobile.press_key(66) # Enter
    mobile.screenshot("discovery_result")

---

© 2026 AppPilot Mobile Framework | Built for Engineers.