apppilot 1.2.36

Modern, Intelligent Mobile Automation Framework (Compiled)

🚀 AppPilot Mobile — The Master Guide

Welcome to the world-class documentation for AppPilot Mobile. This framework is designed to be the "Playwright for Mobile" — a Python-first, human-readable automation engine that handles the complexity of mobile UI so you can focus on writing tests.

---

🏗️ 1. Professional Architecture (POM)

AppPilot is optimized for the Page Object Model (POM). By following this structure, your tests will be readable, maintainable, and stable.

1.1 BaseWindow — The Core Elements

Define common system-level elements (buttons, toolbars) here so every page can use them without re-locating.

1.1 Screen Object Model (SOM) — Mobile-First Architecture

While Page Objects work, mobile applications natively consist of Screens (backed by Android Activities/Fragments or iOS ViewControllers). The Screen Object Model (SOM) architecture leverages dynamic runtime identification to ensure robust screen transition verification.

1.1.1 BaseScreen & The Screen Registry

By defining a unique SCREEN_IDENTIFIER (such as the Activity class name on Android or ViewController name on iOS), you build self-verifying transition checks right into page flows.

class BaseScreen:
    # A substring mapping to the current Android Activity or ViewController name
    SCREEN_IDENTIFIER = None 

    def __init__(self, mobile: Mobile):
        self.mobile = mobile

    def verify_on_screen(self):
        """Ensures that the active OS screen matches the expected screen."""
        current_screen = self.mobile.current_screen_name
        if self.SCREEN_IDENTIFIER and self.SCREEN_IDENTIFIER not in current_screen:
            raise WrongScreenError(
                f"Expected to be on screen '{self.__class__.__name__}' "
                f"({self.SCREEN_IDENTIFIER}), but currently on '{current_screen}'"
            )
        return self

1.1.2 Screen Transition Example

In this model, the LoginScreen verifies its starting state, fills in credentials, taps submit, and then instantiates and returns the next screen (DashboardScreen) after verifying the transition was successful:

class LoginScreen(BaseScreen):
    SCREEN_IDENTIFIER = ".ui.login.LoginActivity"

    def login(self, username, password) -> DashboardScreen:
        self.verify_on_screen()
        
        self.mobile.input("Username").fill(username)
        self.mobile.input("Password").fill(password)
        self.mobile.button("Login").tap()
        
        dashboard = DashboardScreen(self.mobile)
        dashboard.verify_on_screen()  # Automatically verifies successful transition!
        return dashboard

---

🧠 2. Smart & AI-Powered Features

AppPilot goes beyond basic automation by using an Intelligent Layer to handle real-world mobile UI challenges.

2.1 Intelligent Locator Engine (Scoring & Dynamic Selection)

Stop fighting with duplicate IDs or "distractor" elements. AppPilot uses a scoring engine to find exactly what you want.

• Example: If you search for "Search", it prioritizes the clickable search button and ignores "Search History" or "Search Labs" automatically.
• Priority: Accessibility ID > Visible Text > Resource ID > Content Description.

Handling Elements Without Text Dynamically

You do not need separate methods (like tap_xpath() or tap_by_id()) for elements without text. The exact same public methods (mobile.tap(), mobile.fill(), etc.) handle all targets dynamically:

• Accessibility ID / Content Description:

  mobile.tap("shopping_cart_icon")  # Taps by accessibility label even if there is no text.
  

• Resource ID:

  mobile.tap("com.example.app:id/submit_btn")  # Identifies the developer-assigned element ID.
  

• Placeholder / Hint Text:

  mobile.fill("Enter your password", "secret123")  # Fills inputs identified by empty placeholder text.
  

• XPath Fallback:

  mobile.tap("//android.widget.FrameLayout[1]/android.widget.Button[2]")  # Triggers automatically if starting with "//".
  

2.2 Auto-Scroll Search

Never worry if an element is off-screen. Methods like tap() and fill() will automatically:

1. Check if the element is visible.
2. If not, perform a smart swipe (Samsung optimized).
3. Retry the search.
4. Fail only after 5 failed scrolls (configurable).

2.3 AI Test Generation (TEST_GEN_PROMPT.md)

Every project initialized with apppilot setup includes a specialized AI prompt.

1. Copy the contents of TEST_GEN_PROMPT.md.
2. Paste it into ChatGPT or Gemini.
3. Describe your test case (e.g., "Login, then go to Profile and change name").
4. The AI will generate a complete, valid AppPilot Python test for you.

---

🖥️ 3. Live Monitoring & Evidence

3.1 Live Desktop Mirroring

See your phone's screen in a window on your desktop during execution using the integrated scrcpy support.

• Enable: Set mirror_screen: true in your apppilot.yaml.

3.2 Automated Evidence

• Hardware-Accelerated Recording (MP4): Every test is recorded with zero performance drop. Includes professional 1s padding.
• Smart Screenshots: Captures specific UI states into .apppilot/screenshots/.

---

🛠️ 4. Developer & Diagnostic Tools

AppPilot includes powerful tools to help you debug and optimize your automation scripts in real-time.

4.1 UI Inspector (mobile.inspect())

Prints a detailed, categorized map of every element on the screen.

• Categorization: Automatically identifies INPUT, BUTTON, TEXT, IMAGE, CHECKBOX, and more.
• Precision: Includes exact screen bounds [x1,y1][x2,y2] for every element.
• Usage:

mobile.inspect()

Output Example:

[1] BUTTON: text='', desc='Google Search', id='', bounds='[45,744][182,882]', class='android.widget.Button'
[2] INPUT: text='', desc='', id='', bounds='[180,744][630,882]', class='android.widget.EditText'

4.2 Device Diagnostic (mobile.diagnose())

Provides a complete health check of the device and the automation environment.

• System Info: Model, Android version, and Serial.
• App State: Package and Activity currently in focus.
• Performance: Real-time Memory (MB) and CPU (%) usage.
• Usage:

mobile.diagnose()

4.3 Relational Selectors (Maestro-Style)

Find elements relative to other elements when they lack clear IDs or text.

• Methods: below(), above(), left_of(), right_of().
• Usage:

# Fill an input field to the right of the "Search" button
mobile.right_of("Search").input().fill("AppPilot Mobile")

# Tap a button below a specific label
mobile.below("Username").button("Login").tap()

4.4 Programmatic Element Attributes

Retrieve detailed properties and visual characteristics of elements dynamically. Methods return a dictionary with text, bounds, clickable state, average color, and relative coordinates.

• Methods: get_element_attributes(target), get_focused_element(), get_active_element(), get_selected_element().
• Usage:

# Retrieve all properties of an element (returns a dict)

  ### Code Example

    def test_element_details(mobile):
        # 1. Launch your application
        mobile.launch_app("com.jio.consumer.jiothings")
    
        # 2. Get attributes of a specific element by text, ID, or description
        card_attributes = mobile.get_element_attributes("Tap this card to start Vehicle Sharing")
    
        # The method returns a standard Python dictionary
        print("\n--- Card Attributes ---")
        print(f"Class Name:          {card_attributes.get('class')}")
        print(f"Text Value:          {card_attributes.get('text')}")
        print(f"Resource ID:         {card_attributes.get('resource-id')}")
        print(f"Accessibility Desc:  {card_attributes.get('content-desc')}")
        
        # Coordinates & Layout properties
        bounds = card_attributes.get('bounds')  # returns tuple (x1, y1, x2, y2)
        print(f"Coordinates Bounds:  {bounds}")
        print(f"Relative Center:     X={card_attributes.get('rel_cx') * 100:.1f}%, Y={card_attributes.get('rel_cy') * 100:.1f}%")
        
        # State flags
        print(f"Clickable state:     {card_attributes.get('clickable')}")
        print(f"Checked state:       {card_attributes.get('checked')}")
        print(f"Focused state:       {card_attributes.get('focused')}")
        print(f"Enabled state:       {card_attributes.get('enabled')}")
        
        # Visual & Color analysis
        print(f"Avg Hex Color:       {card_attributes.get('avg_color')}")     # e.g., '#e9e9e9'
        print(f"Avg RGB Colors:      {card_attributes.get('avg_rgb')}")       # e.g., (233, 233, 233)
        print(f"Image Complexity:    Variance={card_attributes.get('variance')}")
        print(f"Icon/Image detected: {'Yes' if card_attributes.get('is_complex') else 'No'}")
    
    
        # 3. Retrieve properties of the currently active/focused element
        active_el = mobile.get_focused_element()  # or mobile.get_active_element()
        
        print("\n--- Focused Element ---")
        print(f"Active Class:        {active_el.get('class')}")
        print(f"Active Text:         {active_el.get('text')}")
        print(f"Is Focused:          {active_el.get('focused')}")
        # Chained One-Liner Example:
        #You can also chain everything together in a single line:

        # Directly get the parent card's background color starting from the child title's Resource ID
        parent_color = mobile.element("com.jio.consumer.jiothings:id/tv_title_share_vehicle").parent().get_attributes().get("avg_color")
        print(f"Parent Card Background Color: {parent_color}")
         # 2. Child of Child (Grandchild element - e.g. first child of the first child)
        # You can specify indexes (0-indexed) for child navigation
        grandchild_element = element.child(0).child(0)
        grandchild_attrs = grandchild_element.get_attributes()
        
        print("\n--- Grandchild Attributes ---")
        print(f"Class: {grandchild_attrs.get('class')}")
        print(f"Text:  {grandchild_attrs.get('text')}")

        # 3. Sibling navigation (getting the next or specific sibling by index)
        # Get the second sibling element of the title text element:
        second_sibling = mobile.element("com.jio.consumer.jiothings:id/tv_title_share_vehicle").sibling(1)
        print(f"Second Sibling Text: {second_sibling.get_attributes().get('text')}")
        
        # Get all siblings of the element:
        all_siblings = mobile.element("com.jio.consumer.jiothings:id/tv_title_share_vehicle").siblings()
        print(f"Total Siblings Found: {len(all_siblings)}")

        # 4. Mixing Parent, Child, and Sibling chaining
        # Go up to the parent, locate a sibling at index 2, and tap it:
        sibling_sub_element = element.parent().sibling(2).child(0)
        sibling_sub_element.tap()
    

---

📊 5. Complete API Reference

Category	Method	Example	Description
Core	tap(target) / click()	mobile.click("Login")	Smart tap/click with auto-scroll.
	fill(target, text) / set_text()	mobile.set_text("User", "admin")	Locates label and fills input.
	clear(target) / clear_text()	mobile.clear_text("Search")	Wipes text from a field.
	double_tap(t) / double_click()	mobile.double_click("Map")	Rapid two-tap gesture.
	long_press(target)	mobile.long_press("File")	1s touch-and-hold.
	dropdown(target)	mobile.dropdown("Theme").select(option="Dark")	Opens dropdown and selects by option text or index.
Diagnostics	inspect()	mobile.inspect()	Prints UI tree to console.
	diagnose()	mobile.diagnose()	Full device & app health check.
	get_current_app()	app = mobile.get_current_app()	Returns package/activity.
	get_performance()	perf = mobile.get_performance()	Returns CPU/Memory stats.
	get_focused_element()	active = mobile.get_focused_element()	Gets properties of currently focused element.
	get_active_element()	active = mobile.get_active_element()	Alias for get_focused_element().
	get_selected_element()	active = mobile.get_selected_element()	Alias for get_focused_element().
Device Mgmt	connect_device(serial)	mobile.connect_device("192.168.1.5:5555")	Connects via ADB TCP/IP.
	disconnect_device()	mobile.disconnect_device()	Disconnects device.
	get_device_info()	info = mobile.get_device_info()	OS, Model, Manufacturer.
	install_app(path)	mobile.install_app("app.apk")	Installs APK.
	uninstall_app(pkg)	mobile.uninstall_app("com.app")	Uninstalls package.
	close_app(pkg)	mobile.close_app("com.app")	Force stops application.
	clear_app_data(pkg)	mobile.clear_app_data("com.app")	Resets app state.
	grant_permissions()	mobile.grant_permissions(pkg, ["camera"])	Auto-allow permissions.
	set_location(lat, lon)	mobile.set_location(12.9, 77.5)	Spoofs GPS coordinates.
	set_orientation(mode)	mobile.set_orientation("landscape")	Portrait/Landscape switch.
Keyboard	send_keys(text)	mobile.send_keys("Hello")	Direct text to focused field.
	hide_keyboard()	mobile.hide_keyboard()	Dismisses soft keyboard.
	show_keyboard()	mobile.show_keyboard()	Forces keyboard to open.
	press_key(key)	mobile.press_key("ENTER")	Hardware key (BACK, HOME, etc).
Gestures	swipe(x1,y1,x2,y2)	mobile.swipe(100,500,900,500)	Raw coordinate movement.
	scroll_to_text(text)	mobile.scroll_to_text("Accept")	Scrolls until text found.
	drag_and_drop(s, t)	mobile.drag_and_drop("App", "Bin")	Moves element.
	flick(x1,y1,x2,y2)	mobile.flick(500,800,500,200)	Fast swipe gesture.
Discovery	get_page_source()	xml = mobile.get_page_source()	Returns UI hierarchy XML.
	get_all_texts()	mobile.get_all_texts_displayed()	Returns all visible strings.
	get_text(target)	mobile.get_text("Title")	Reads text from element.
	get_element_attributes(t)	attrs = mobile.get_element_attributes("Btn")	Gets detailed layout, state, and visual properties.
	get_attributes()	attrs = mobile.element("Btn").get_attributes()	Chained attributes retriever for elements.
Media	take_screenshot(n)	mobile.take_screenshot("err")	Captures full screen.
	capture_element(t,n)	mobile.capture_element_screenshot("Btn","c")	Crops to element bounds.
	start_recording()	mobile.start_screen_recording()	Captures MP4 video.
	capture_photo()	mobile.capture_photo()	Shutter + Auto-OK.
State	is_enabled(target)	mobile.is_enabled("Submit")	Clickable state check.
	is_visible(target)	mobile.is_visible("Title")	Visibility check.
Wait	wait_for_page_ready()	mobile.wait_for_page_ready()	Waits for UI stability.
	wait_until_visible()	mobile.wait_until_visible("OK")	Polling wait until visible.
	wait_until_gone()	mobile.wait_until_gone("Load")	Polling wait until vanished.
Validation	verify_text(text)	mobile.verify_text("Saved")	Assert exact text exists.
	verify_present(t)	mobile.verify_element_present("id")	Assert element exists.
Advanced	execute_shell()	mobile.execute_shell_command("ls")	Raw shell access.
	execute_adb()	mobile.execute_adb("devices")	Raw ADB command.
	find_by_image()	mobile.find_by_image_click("p.png")	Visual Match + Click.

---

🏗️ 6. CLI Tooling Reference

Command	Example	Description
init	apppilot init	Scaffolds a new project.
setup	apppilot setup --project_folder tests	Initializes project in specific folder.
--overwrite	apppilot setup --overwrite	Overwrites apppilot.yaml with latest defaults.
doctor	apppilot doctor	Dependency and tool path health check.
devices	apppilot devices	Lists all connected device serials.
interactive	apppilot interactive	Live REPL: Run commands one-by-one.
run	apppilot run	Standard test execution.
report	apppilot report	Generates/Serves HTML test report.

📂 7. Configuration (apppilot.yaml)

# apppilot.yaml
tools:
  adb_path: 'C:\path\to\adb.exe'
  scrcpy_path: 'C:\path\to\scrcpy.exe'
  log_path: '.apppilot/logs'  # Custom log folder

mirror_screen: true
auto_scroll: true
timeout: 30
recorder_refresh_delay: 1.5  # Time (in seconds) to wait for screen transitions before refreshing the recorder view

Note for Windows Users: Always use single quotes ' for paths to avoid errors with backslashes.

---

🛠️ 8. Discovery & Debugging Example

When you are automating a new app and don't know the IDs, use this "Discovery Script" to find everything you need.

import pytest
from apppilot.api.mobile import Mobile

def test_discovery_session(mobile):
    # 1. Launch the app and check health
    mobile.launch_app("com.android.chrome")
    mobile.diagnose()  # Prints CPU/Memory and current Activity
    
    # 2. Navigate to a complex page
    mobile.navigate("https://www.google.com")
    mobile.wait_for_page_ready()
    
    # 3. Inspect the UI to find hidden locators
    # This will print every INPUT, BUTTON, and TEXT with coordinates
    mobile.inspect()
    
    # 4. Use Relational Selectors for elements without IDs
    # "Find the input to the right of the Google Search icon"
    mobile.right_of("Google Search").input().fill("AppPilot Mobile")
    
    # 5. Check performance after interaction
    perf = mobile.get_performance()
    print(f"Current App Memory Usage: {perf['memory_mb']} MB")
    
    mobile.press_key(66) # Enter
    mobile.screenshot("discovery_result")

---

© 2026 AppPilot Mobile Framework | Built for Engineers.