🚀 WebPilot Engine v2.3.0

Enterprise Autonomous Browser Engine, Universal Form Inspector & AI Agent Web Runtime

WebPilot Engine is a high-performance, enterprise-grade autonomous browser engine and AI agent web runtime designed to inspect, interact with, populate, and operate complex modern web applications, single-page apps (SPAs), dynamic Shadow DOMs, and enterprise ATS/ERP widgets (Workday, SAP SuccessFactors, Taleo, Oracle Cloud, custom modal flows) with zero site-specific selectors or hardcoded scripts.

🏛️ 3-Tier Zero-Elevation Multi-Process Architecture

To overcome the fundamental operating system limitation where terminal runners and subshells terminate background browser child processes upon command completion (such as the Windows Job Object JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE), WebPilot implements a hardened Zero-Elevation Three-Tier Process Architecture:

 ┌────────────────────────────────────────────────────────────────────────┐
 │               WebPilot CLI & REPL Shell (webpilot / wp)                │
 └───────────────────────────────────┬────────────────────────────────────┘
                                     │ HTTP REST (127.0.0.1:9333)
                                     ▼
 ┌────────────────────────────────────────────────────────────────────────┐
 │              Layer 1: Master Supervisor Host (Daemon)                  │
 │   • Ultra-Lightweight IPC Controller (~15 MB RAM)                      │
 │   • Zero Playwright Imports (Starts instantaneously)                  │
 │   • 30-Minute Inactivity Watchdog Sentinel (Automatic RAM release)     │
 │   • Detached Background Auto-Spawn via Windows WMI / POSIX double-fork │
 └───────────────────────────────────┬────────────────────────────────────┘
                                     │ stdin / stdout (Isolated JSON-RPC)
                                     ▼
 ┌────────────────────────────────────────────────────────────────────────┐
 │                Layer 2: Operational Worker Process                     │
 │   • Playwright Runtime + SessionCoordinator                            │
 │   • Persistent in-memory browser session maintained across CLI calls   │
 │   • Decoupled DOM Services (Field, Auth, Reactive, Inspection)         │
 │   • Multi-Stage Verification Barrier & Dynamic Section Expander        │
 └───────────────────────────────────┬────────────────────────────────────┘
                                     │ Process Tree Attachment
                                     ▼
 ┌────────────────────────────────────────────────────────────────────────┐
 │                   Layer 3: Chromium Browser Engine                     │
 │   • Chromium Main Process + GPU Process + Sandboxed Renderers          │
 │   • Deterministic Cascading Termination (Zero Zombie Leaks)            │
 └────────────────────────────────────────────────────────────────────────┘

⚡ Architectural Pillars:

1. Persistent Browser Session Across Disconnected CLI Invocations: Execute multi-step sequential interactions (open ➔ fill profile ➔ upload resume ➔ press submit) without closing the browser, losing session state, or re-authenticating.
2. Zero Administrative Elevation (Standard User): Operates 100% in unprivileged standard user space. Zero Administrator / UAC escalation required.
3. Deterministic Cascading Termination & Auto-Healing: Terminating Layer 2 instantly tears down the entire Chromium process tree with zero zombie process leaks. Layer 1 automatically auto-spawns a fresh Layer 2 worker in ~300ms.
4. Memory Watchdog Sentinel: Automatically reclaims all browser resources and cleanly exits after 30 minutes of idle inactivity.

🐍 Python SDK: Programmatic AI Agent Usage

WebPilot provides a first-class, ergonomic Python SDK for AI agents, automation pipelines, and Python applications via import webpilot:

from webpilot import WebPilot

# Connects to background supervisor automatically (or auto-spawns detached host)
with WebPilot() as wp:
    # 1. Inspect Form Schema (returns JSON dictionary of inputs, dropdowns, buttons)
    schema = wp.inspect("https://careers.example.com/apply")

    # 2. Fill Fields with 0.0% False Success Physical Truth Verification
    result = wp.apply(
        url="https://careers.example.com/apply",
        fields={
            "First Name": "Alice",
            "Last Name": "Nakamura",
            "Email": "alice@example.com",
            "Degree": "Master's",
            "Authorized to work": "Yes"
        },
        press="Submit Application"
    )

    if result.success:
        print(f"Application confirmed! Confirmed fields: {len(result.confirmed_fields)}")
    else:
        print(f"Application failed verification: {result.unconfirmed_fields}")

    # 3. Probe Specific Element or Dropdown Options
    probe = wp.find("Country", options=True)
    print("Available options:", probe.output_lines)

    # 4. Multi-Tab Session Management
    tabs = wp.list_tabs()
    wp.switch_tab(0)

⚡ The Performance Breakthrough (9x Native Speedup)

In version 2.0.11 and 2.0.12, WebPilot underwent a comprehensive timing and latency overhaul, transitioning from legacy synthetic delays to reactive, event-driven DOM readiness:

• Elimination of Artificial Pauses: Replaced blind 3,000ms pauses (PAUSE_DOM_CONTENT_LOADED_MS, PAUSE_REACTIVE_STABILIZE_MS) with deterministic condition checking and DOM event listeners.
• Bounded Actionability Timeouts: Added strict 1,000ms locator timeouts to prevent Playwright from hanging for 30 seconds on non-actionable or overlayed buttons (e.g. S10 overlay trap).
• Conditional Accordion Expansion: Section expansions only wait if closed accordions were actually detected and opened.

📊 Benchmark Execution Latency Comparison:

🛡️ The Truth Barrier: 0.0% False Success Guarantee

A major vulnerability in AI browser agents and web automation frameworks is False Success (Silent Failure): claiming an operation succeeded when inputs were silently rejected, cleared by reactive JavaScript, or blocked by invisible validation errors.

WebPilot enforces a strict Multi-Stage Physical Truth Verification Pipeline:

[Fill Attempt] ➔ [1. Inline DOM Re-read] ➔ [2. Pre-Submit Verification Barrier] ➔ [Button Click / Submit] ➔ [3. Post-Interaction Verification Barrier]

1. Inline DOM Re-read (verify_field_value): Immediately reads the physical DOM property after setting an input to confirm that the value took effect.
2. Pre-Submit Verification Barrier (_reverify_fields): Re-checks all previously confirmed fields right before triggering submit buttons to catch delayed resets.
3. Post-Interaction Verification Barrier: Re-verifies all fields after button clicks to detect adversarial scripts or form handlers that clear or revert values upon clicking (e.g. S29, S44).
4. Dynamic Repeating Section Expansion (expand_dynamic_section): Automatically identifies and clicks dynamic repeater triggers (Add, Add another, +) when target fields do not yet exist in the DOM (e.g. S27).
5. Categorized Telemetry Accounting: Categorizes every field deterministically as confirmed, unconfirmed, or failed. If even a single field is unconfirmed, WebPilot reports operation failure rather than making speculative success claims.

🧹 Clean Architecture & SOLID Engineering (v2.3.0)

Following an extensive project-wide architectural refactoring milestone, the codebase adheres to strict enterprise software standards:

• Strict Multi-Tier Layering & Separation of Concerns:
  • Presentation Layer: CLI Dispatcher (cli.py), Modular Subcommands (cli_commands/), Modular CLI Parser (cli_parser.py), MCP Server (mcp_server.py), MCP Tool Dispatchers (mcp_tools.py), REPL Shell (supervisor/shell.py), Telemetry (common/telemetry.py).
  • Orchestration & Supervision: Master Daemon (master_daemon.py), HTTP Handler (supervisor/handler.py), Inactivity Watchdog (supervisor/watchdog.py), Worker Daemon (supervisor/worker_process.py), Worker Execution Handlers (supervisor/worker_handlers.py), Pipeline Orchestration (flow_orchestrator.py).
  • Domain Services: Field Interactions (FieldInteractionService), Form Application Pipeline (FormApplicationPipeline), Target Descriptor Parser (TargetDescriptorParser), DOM Data Matcher (DOMDataMatcher), Schema Merger (SchemaMerger), Inspection (InspectionService), Reactive Interactions (ReactiveInteractionService), Page Lifecycle (PageLifecycleService).
  • Infrastructure & Adapters: Browser Adapter (BrowserAdapter), Tab & Window Manager (TabManager), Playwright Fallbacks (PlaywrightLocatorFallback).
• Modular DOM Script Engine (adapters/dom/):
  • Decomposed legacy 1,000+ line monolithic scripts into focused, single-responsibility modules: lifecycle.py, navigation.py, set_field.py, picklist.py, verify.py, inspection.py, probe.py, reactive.py.
• Universal Page Lifecycle Paradigm:
  • Eliminated bespoke auth hacks; all portals and login pages are managed uniformly through event-driven DOM readiness and accordion expansion in PageLifecycleService.
• Tab & Multi-Window Isolation:
  • Fully abstracted browser context, tab lifecycle, popup tracking, and route interception into TabManager.
• DRY Consolidation & Single Source of Truth:
  • Pipeline phases consolidated in FormApplicationPipeline with deterministic 3-stage physical truth verification.
  • Eliminated circular imports and inline imports in compliance with PEP 8.

🔬 Empirical Adversarial Benchmark Suite (50 Scenarios)

The repository includes a complete, fully reproducible 50-Scenario Adversarial Benchmark Suite under eval/ containing real-world edge cases, anti-automation traps, and complex widget patterns:

Certified Evaluation Results (v2.0.12 on Linux Cloud VPS):

Reproducing the Benchmark:

# 1. Start the local evaluation lab server
python eval/lab/server.py --port 8900 &

# 2. Run the monitored benchmark suite
python eval/scripts/run_suite.py --tool webpilot --wp wp --runs 1 --label v2012 --out-dir eval/results

The benchmark fixtures, expected JSON contracts, and in-depth analytical reports are accessible in:

• eval/lab/scenarios.py — Definitions of all 50 test scenarios.
• eval/docs/REPORT.md — Comprehensive benchmark evaluation report.
• eval/docs/FALSE_SUCCESS_ANATOMY.md — Architectural breakdown of false success prevention.
• eval/docs/DEFECTS.md — Documented edge cases, trap analysis, and resolution history.

🚀 Quick Start Guide

Installation:

# Clone the repository
git clone https://github.com/AbdulkarimSSS/webpilot.git
cd webpilot

# Install in editable mode
pip install -e .

# Install Playwright browser binaries
playwright install chromium

You can use either the full webpilot command or the shortcut wp:

1. Inspect Any Web Page

Extract a token-optimized, clean schema of inputs, dropdowns, and buttons:

wp inspect --url "https://example.com/portal" --output "schema.json" --screenshot "page.png"

2. Autonomous Form Filling & Submission

Fill fields using JSON data payloads or inline flags:

# Using inline arguments
wp apply --url "https://example.com/login" \
         --fill "username=demo@example.com" \
         --fill "password=MySecurePassword123!" \
         --press "Sign In" \
         --submit

# Using JSON payload file
wp apply --url "https://example.com/job-application" \
         --data "applicant_data.json" \
         --press "Submit Application"

3. Step-by-Step Multi-Turn Execution

Commands run sequentially against the same active persistent browser session:

# Step 1: Open target portal
wp apply --url "https://example.com/register"

# Step 2: Fill personal details
wp apply --fill "first_name=John" --fill "last_name=Doe"

# Step 3: Fill contact info
wp apply --fill "email=john.doe@example.com" --fill "phone=+1234567890"

# Step 4: Click Next, detect dynamic expansion, and complete registration
wp apply --press "Next" --submit --screenshot "complete.png"

4. Interactive REPL Shell

Launch the sub-second interactive automation shell:

wp shell

webpilot> open https://example.com/login
webpilot> fill username=john.doe@example.com
webpilot> fill password=MyPassword123!
webpilot> press Sign In
webpilot> tabs
webpilot> screenshot dashboard.png
webpilot> exit

5. Supervisor Service Management

wp service status   # View real-time PID telemetry for Layer 1, Layer 2, and Layer 3
wp service restart  # Cascading kill of Layer 2 & 3, auto-respawns in ~300ms
wp service stop     # Cleanly terminate all processes and reclaim 100% of memory

🧪 Unit Testing & Verification

WebPilot includes a comprehensive test suite covering IPC wire contracts, supervisor lifecycle, redaction, and DOM interaction services:

pytest tests/

All 58 unit tests execute in < 28 seconds:

tests/test_contracts.py .......                                          [ 12%]
tests/test_engine_facade.py .....                                        [ 20%]
tests/test_live_session.py ...                                           [ 25%]
tests/test_orchestration.py ......                                       [ 36%]
tests/test_redaction.py ........                                         [ 50%]
tests/test_services.py ..............                                    [ 74%]
tests/test_supervisor.py ................                                [100%]

============================== 58 passed in 27.42s ==============================

🔒 Security & Privacy

• Automatic Credential Redaction: Passwords, API tokens, and sensitive inputs are automatically masked as ***REDACTED*** in CLI output, logs, and telemetry cards.
• Standard User Security: Requires zero Administrator / root permissions; never triggers UAC prompts or elevates tokens.
• Local Loopback Isolation: The Master Supervisor listens strictly on 127.0.0.1:9333.

📄 License

This project is licensed under the MIT License — free for commercial, research, and personal use.