agent-failure-doctor 6.3.2

Local-first failure diagnosis for AI browser automation, Playwright, crawler, and RPA runs.

Agent Failure Doctor

中文文档

Diagnose Playwright failures locally, apply one scoped locator repair, and keep the change only when the same real test command passes. The verified Playwright locator repair is deliberately narrow and evidence-bound.

The public core deliberately does two things:

1. Turn a trace, log, screenshot, or failed-run directory into an evidence report.
2. Repair an explicitly supplied stale locator, rerun the real test, and emit a truthful verified, failed_rolled_back, or manual_restore_required result.

git clone https://github.com/tobybgy-lsd/web-agent-runtime-bench.git
cd web-agent-runtime-bench
python -m pip install agent-failure-doctor
failure-doctor diagnose .\examples\failed_runs\proxy_network_error --out .\report
failure-doctor bench

Verified locator repair

failure-doctor heal .\test-results\failed-case `
  --project . `
  --target tests\checkout.spec.ts `
  --old-locator "button.old-submit" `
  --new-locator "button[data-testid=submit]" `
  --out .\heal-report `
  --test-command npx playwright test tests\checkout.spec.ts

The command writes the diagnosis, exact diff, source backup, sanitized command logs, exit code, and verification report. It blocks ambiguous or unsupported repairs. A non-zero rerun restores the original file by default. It never marks a proposal as repaired merely because a file was edited. See Verified repair contract.

Current milestone: Agent Failure Doctor v6.3.2 Release Identity Patch. Current stable line: v6.3.2.

P98 gate: passed. P98 master gate passed. See validation/dashboard.md for the validation dashboard.

Previous stable line: Agent Failure Doctor v6.3.1 Verified Repair Release. Earlier stable line: Agent Failure Doctor v6.1 Composite Diagnosis Runtime Triage Release. Earlier stable line: Agent Failure Doctor v3.9.0 Local Failure Knowledge Base Pack. Previous P95 stable: Agent Failure Doctor v2.4.1 P95 Alignment & Missing Tracks Pack.

Fast path (v6.3)

Install only the local diagnosis and benchmark core:

python -m pip install agent-failure-doctor
failure-doctor doctor
failure-doctor diagnose .\examples\failed_runs\proxy_network_error --out .\report
failure-doctor heal --help
failure-doctor bench
failure-doctor console

The default CLI promotes only six entries: diagnose, heal, bench, console, doctor, and advanced. The existing full command set remains available under failure-doctor advanced and through its original command names.

Install optional capabilities only where they run:

python -m pip install "agent-failure-doctor[web]"
python -m pip install "agent-failure-doctor[android]"
python -m pip install "agent-failure-doctor[ocr-paddle]"
python -m pip install "agent-failure-doctor[enterprise]"

The built-in lite-core benchmark is offline and uses packaged, sanitized failure artifacts. Challenge/CAPTCHA cases are detection and manual-handoff tests, not bypasses.

Lifecycle commands: diagnose / plan / verify / run. Classic lifecycle: diagnose -> plan -> AI handoff / patch proposal -> verify -> sanitize/share. Compatibility track: Agent Failure Doctor v4.2.0 Plugin SDK & Adapter Ecosystem Pack. Previous P95 stable: v2.4.1.

Specialist Android, OCR, enterprise, adapter, and LAN Worker capabilities are optional compatibility tracks. They are intentionally absent from the default help; use failure-doctor advanced when you actually operate those components.

Experimental LAN Worker v1

The next line adds an opt-in LAN execution protocol for registered workers: single-use pairing, heartbeats, capability matching, persistent task leases, fencing tokens, ordered logs, checkpoints, verified evidence upload, and truthful completion contracts. Workers execute only locally configured task_type=module:function allowlist entries; the controller cannot send an arbitrary shell command. See LAN Worker v1. Existing RPA masters can use the lifecycle and preflight contract in LAN Worker Adapter SDK.

v6.1 Composite Diagnosis Runtime Triage

python -m tools.validation.run_android_composite_runtime_triage_validation
python -m tools.validation.run_p98_master_gate

v6.1 adds Android/APK composite runtime triage for multi-factor failures such as device offline plus permission dialog plus locator drift. It ranks primary root cause, secondary causes, and downstream symptoms, then emits a repair order that fixes device connectivity, permission or WebView context blockers before locator repair. The validation gate covers 40 local synthetic cases with 40/40 primary root cause, secondary ordering, downstream suppression, and repair order checks.

v6.0 Mobile Automation Stable Standardization

failure-doctor android-author record start --app com.example.mock --out .\recording
failure-doctor android-pilot project-init --app com.example.mock --out .\pilot
failure-doctor android-dx bundle-create --input .\failed_run --out .\dx_bundle
failure-doctor android-playbook list --out .\playbooks
failure-doctor android-real-pilot init --out .\real_pilot_private
failure-doctor android-lab init --out .\android_lab
failure-doctor mobile-stability check-android-cli --out .\mobile_stability

v6.0 closes the Android v5.4-v5.9 road-to-stable line with workflow authoring, visual assertions, human review, app-specific pilot scaffolding, deep diagnostic bundles, root-cause playbooks, private real-APK pilot support, mock device-lab hardening, and stable mobile CLI/schema/plugin ABI checks. All Android mobile surfaces remain local-only, dry-run by default, sanitized by default, and keep state-changing actions, app tampering, privileged execution, external upload, raw browser profile reads, and local credential store reads blocked by default.

Android Real Device Farm & Business Workflow Ops

failure-doctor android-ops farm init --out .\android_farm
failure-doctor android-ops farm discover --farm .\android_farm --out .\device_inventory
failure-doctor android-ops template scaffold --type post_image_text_save_draft --out .\flows\post_image_text.yml
failure-doctor android-ops data bind --flow .\flows\post_image_text.yml --data .\tasks.csv --out .\bound_tasks
failure-doctor android-ops scheduler plan --farm .\android_farm --queue .\bound_tasks --out .\schedule_plan

Android Ops adds local device farm management, Appium session orchestration, device lease/recovery, business workflow templates, task scheduling, replay, flaky detection, compatibility reports, mutation guard, and production runbooks for authorized Android automation workflows. It is dry-run by default, save-draft first, blocks final submit and production state mutation by default, and does not provide abuse circumvention, device identity manipulation, APK tampering, runtime instrumentation, privilege escalation, external upload, or pooling-based evasion guidance.

v5.2 Android APK Production Hardening

failure-doctor android-pro profile init --package com.example.mock --out .\android_profile
failure-doctor android-pro locator-registry build --ui-dump .\ui.xml --out .\registry
failure-doctor android-pro flow scaffold --template post_image_text_dry_run --out .\flow.yml
failure-doctor android-pro flow lint --flow .\flow.yml --profile .\android_profile\android_app_profile.json --out .\lint
failure-doctor android-pro onboarding-check --profile .\android_profile\android_app_profile.json --flow .\flow.yml --out .\onboarding

v5.2 adds an Android Pro production-hardening layer for authorized/mock app automation evidence. It keeps workflows local-only and dry-run by default, keeps final action execution off by default, blocks coordinate-primary locators, and reports locator healing candidates for manual review instead of auto-applying them.

v5.1 Android APK UI Automation Adapter

failure-doctor android doctor --out .\android_doctor
failure-doctor android validate-flow .\examples\android_apk_cases\post_image_text_dry_run\input\flow.yml --out .\android_flow
failure-doctor android dump-ui .\examples\android_apk_cases\ui_tree_basic\input\ui.xml --out .\android_ui
failure-doctor diagnose .\examples\android_apk_cases\permission_dialog_blocked\input --adapter android-apk --out .\android_diag
failure-doctor collect --adapter android-apk --input .\examples\android_apk_cases\permission_dialog_blocked\input --out .\android_pack

v5.1 adds a local-only Android APK UI evidence adapter for authorized/mock app automation failures. It normalizes Appium-style logs, ADB/uiautomator XML, logcat summaries, screenshots, and flow files into the same diagnose -> plan -> verify lifecycle. Flow execution is dry-run by default; final submit is blocked unless explicitly approved. The adapter does not modify APKs, access private app directories, use external OCR uploads, or provide captcha, account, device, or platform-risk bypass guidance.

v5.0 Stable Platform Commands

failure-doctor adapter api diagnose --input .\newman_report.json --out .\api_report
failure-doctor deploy health --workspace .\.failure-doctor-enterprise --out .\health_report
failure-doctor stability check-api --out .\stability_report

v5.0 locks the stable CLI, schema registry, plugin ABI, benchmark case format, public case manifest, and enterprise policy baseline. See docs/STABILITY_POLICY.md, docs/SCHEMA_REGISTRY.md, and docs/PLUGIN_ABI_STABILITY.md.

Real User Case Program & Public Benchmark

failure-doctor case intake --input .\raw_failure_pack --out .\sanitized_case
failure-doctor case publish-check --case .\sanitized_case
failure-doctor issue-pack create --input .\raw_failure_pack --out .\issue_pack
failure-doctor benchmark run --suite public-safe --out .\benchmark_public
failure-doctor benchmark compare --baseline .\benchmark_public --candidate .\benchmark_public --out .\benchmark_compare

v4.3 adds a public-safe case intake path and deterministic benchmark runner. Cases use public_case/v1 manifests, sanitized inputs, expected diagnosis metadata, and publish checks before anything is shared. Benchmarks run local synthetic suites only and produce benchmark_manifest.json, benchmark_summary.md/json, case_results.jsonl, failures.md, regression_diff.json, and open_this_first_benchmark.md.

See docs/REAL_USER_CASE_PROGRAM.md, docs/CASE_CONTRIBUTION_GUIDE.md, docs/PUBLIC_BENCHMARK_SPEC.md, and docs/BENCHMARK_RUNNER.md.

Plugin SDK

failure-doctor plugin scaffold --type framework-adapter --name toy_adapter --out .\plugins\toy_adapter
failure-doctor plugin validate .\plugins\toy_adapter
failure-doctor plugin install .\plugins\toy_adapter --workspace .\.failure-doctor-plugins
failure-doctor plugin enable toy_adapter --workspace .\.failure-doctor-plugins
failure-doctor plugin run toy_adapter --workspace .\.failure-doctor-plugins --input .\sample_artifacts --out .\plugin_report

The Plugin SDK lets teams extend Agent Failure Doctor with local-only framework adapters, evidence adapters, diagnosis rule packs, industry packs, Console extensions, CI extensions, KB patterns, reasoning tools, report exporters, and validation packs.

Plugin safety rules:

• disabled by default
• manifest required
• permissions required
• local-only by default
• no upload by default
• no network by default
• no shell by default
• no raw evidence access by default
• safety gate and enterprise policy always apply
• plugin outputs are candidates; the core validator remains final authority

Enterprise Governance

failure-doctor enterprise init --workspace .\.failure-doctor-enterprise
failure-doctor enterprise user add --workspace .\.failure-doctor-enterprise --username alice --role developer
failure-doctor console --enterprise --workspace .\.failure-doctor-enterprise --open

Enterprise governance adds local users, role-based access control, approvals, audit ledger, policy enforcement, project/team isolation, shared KB governance, and sanitized audit exports. It remains local-only by default, binds to 127.0.0.1 unless --allow-lan is explicit, requires local auth for enterprise console mode, and does not enable cloud sync, telemetry, or external APIs.

Agent bootstrap: failure-doctor agent-bootstrap --target all --project ..

See validation/dashboard.md.

Quickstart

python -m pip install agent-failure-doctor
git clone https://github.com/tobybgy-lsd/web-agent-runtime-bench.git
cd web-agent-runtime-bench
failure-doctor diagnose .\examples\failed_runs\proxy_network_error --out .\report
failure-doctor plan .\report --out .\fix_plan
failure-doctor collect --project . --preset auto --out .\failure_doctor_auto_report `
  --auto-diagnose --auto-handoff --auto-sanitize
failure-doctor ocr-evidence extract --input .\screenshot.png --out .\ocr_report --provider mock_ocr
failure-doctor ocr-evidence compare --ocr .\ocr_report --dom .\dom_snapshot.html --out .\ocr_compare
failure-doctor visual-runtime diagnose --input .\visual_run --out .\visual_report --no-dom
failure-doctor visual-runtime compare --baseline .\dom_agent_run --candidate .\vlm_agent_run --out .\compare_report
failure-doctor regulated-eval --suite all --out .\regulated_report
failure-doctor full-chain-eval --input .\failed_run --out .\full_chain_report
failure-doctor console --import-report .\full_chain_report --open
failure-doctor ci run --input .\full_chain_report --out .\ci_report --fail-on high
failure-doctor ci templates --out .\ci_templates
failure-doctor kb init --path .\.failure-doctor-kb
failure-doctor kb import-report --kb .\.failure-doctor-kb --report .\full_chain_report
failure-doctor diagnose .\failed_run --kb .\.failure-doctor-kb --hybrid-reasoning --out .\report
failure-doctor reason --input .\report --out .\report\hybrid_reasoning
failure-doctor agent-bootstrap --target all --project .

Hybrid Evidence Reasoning

v4.0 adds local, evidence-bound reasoning:

failure-doctor reason --input .\report --out .\report\hybrid_reasoning
failure-doctor root-cause --input .\report --out .\root_cause_report
failure-doctor causal-chain --input .\report --out .\causal_chain_report
failure-doctor diagnose .\failed_run --kb .\.failure-doctor-kb --hybrid-reasoning --out .\report

The deterministic diagnosis remains the source of truth. Every reasoning claim must cite a sanitized evidence id. Optional local reasoners never download models automatically and never upload raw evidence.

Local Failure Knowledge Base

v3.9 adds a local-only failure knowledge base:

failure-doctor kb init --path .\.failure-doctor-kb
failure-doctor kb import-report --kb .\.failure-doctor-kb --report .\failure_doctor_auto_report
failure-doctor kb search --kb .\.failure-doctor-kb --query "selector timeout after login redirect"
failure-doctor kb match --kb .\.failure-doctor-kb --report .\failure_doctor_auto_report --out .\kb_match_report
failure-doctor kb export --kb .\.failure-doctor-kb --out .\kb_export --sanitized-only
failure-doctor diagnose .\failed_run --kb .\.failure-doctor-kb --out .\report

Use a local KB to match similar historical failures, reuse verified fix candidates, and turn team troubleshooting history into a private, searchable knowledge asset. The KB is local-only and sanitized-only by default: no cloud sync, no external embedding API, no raw secret storage, no private local training content, and verified fixes are suggestions, not auto-applied patches.

CI/CD Integration

v3.8 adds local CI gates and starter templates:

failure-doctor ci run --input .\report --out .\ci_report --fail-on high
failure-doctor ci validate --input .\ci_report --out .\ci_validation
failure-doctor ci templates --out .\ci_templates

The CI pack writes ci_manifest.json, ci_summary.md/json, severity_decision.json, gate_decision.json, audit_manifest.json, and open_this_first_ci.md. It is local-first, does not upload raw artifacts, and blocks private training markers or unsafe recommendation markers before external sharing. See docs/CI_CD_INTEGRATION.md.

Local Web Console

v3.7 adds a local-only report console:

failure-doctor console
failure-doctor console --host 127.0.0.1 --port 8765 --workspace .\.failure-doctor-console
failure-doctor console --import-report .\report --open

The console binds to 127.0.0.1 by default, serves bundled assets only, uses no telemetry, requires a local token for POST actions, and hides raw local evidence by default. It is a report viewer and workflow cockpit for diagnosis, evidence, safety, AI handoff, patch proposal previews, visual runtime, OCR/document, regulated workflow, batch/fleet, and full-chain reports. See docs/LOCAL_WEB_CONSOLE.md.

Optional browser runtimes are capability-detected rather than assumed. The current backend contract keeps Playwright as the deterministic default and reserves adapters for BrowserAct and Ego Browser only when their local CLIs are actually installed. See docs/BROWSER_BACKENDS.md.

Release tracks: v6.3.2 Release Identity is the current package stable line; v6.3.1 Verified Repair and v6.3.0 Lite UX & Bench Core are previous stable lines. v6.2.0 is the earlier Local RPA Ops line.

Command groups: diagnosis, repair planning, verification, collection, OCR evidence, visual runtime, patch proposal, fleet batch, and safe sharing.

Optional visual evidence: screenshot.png / dom_snapshot.html / click_coordinates.json / ocr_excerpt.txt.

Regulated Industry & Pure Visual Agent Evaluation

v3.6 adds local-only regulated workflow mock evaluation and full-chain agent evaluation:

failure-doctor regulated-eval --suite finance --out .\regulated_report
failure-doctor regulated-eval --suite government --out .\regulated_report
failure-doctor regulated-eval --suite healthcare --out .\regulated_report
failure-doctor regulated-eval --suite all --out .\regulated_report
failure-doctor full-chain-eval --input .\failed_run --out .\full_chain_report --include-safety --include-ocr --include-visual --include-regulated

These suites use synthetic local mock evidence only. They do not access real finance, government, healthcare, patient, citizen, bank, or customer systems. They are evidence-quality and shareability checks, not legal or regulatory compliance advice.

OCR & Document Evidence

Use OCR evidence for screenshots, PDFs, forms, tables, reports, RPA exports, ecommerce SKU tables, ERP exports, and document-heavy automation failures.

failure-doctor ocr-evidence extract --input .\screenshot.png --out .\ocr_report --provider mock_ocr
failure-doctor ocr-evidence compare --ocr .\ocr_report --dom .\dom_snapshot.html --out .\ocr_compare
failure-doctor ocr-evidence compare-vlm --ocr .\ocr_report --vlm .\vlm_responses.jsonl --out .\ocr_vlm_compare

Default provider is mock/local. No cloud OCR and no screenshot/PDF upload happen by default. Cloud OCR requires explicit --allow-cloud-ocr and safety evaluation. Sensitive documents are blocked or require sanitization. OCR evidence is supporting evidence, not ground truth.

Visual Agent Runtime Observability

failure-doctor visual-runtime diagnoses offline screenshot-driven agent runs: screenshot capture, image payload cost, VLM observation metadata, action grounding, coordinate clicks, DPR/viewport mapping, stale observations, OCR semantics, and optional DOM/visual conflict.

It is local-only by default: no external VLM calls, no screenshot upload, no real platform access, and no access-control defeat, identity-masking, protected challenge automation, or behavior/pointer-path recipes.

failure-doctor visual-runtime diagnose --input .\visual_run --out .\visual_report --no-dom
failure-doctor visual-runtime profile --input .\visual_run --out .\visual_profile
failure-doctor visual-runtime compare --baseline .\run_a --candidate .\run_b --out .\compare_report
failure-doctor visual-runtime adapt --source generic --input .\artifact_dir --out .\visual_run
failure-doctor visual-runtime validate --input .\visual_run --out .\validation_report

More docs: docs/P98_LIMITS.md, docs/AGENT_FRONTEND_INVOCATION.md, and docs/safety_boundary.md.

P98 master gate passed with the safety compliance pillar included.

Advanced commands include failure-doctor handoff, failure-doctor agent-bootstrap, failure-doctor propose-patch, and failure-doctor batch.

Core commands: collect / diagnose / plan / verify / run / watch / sanitize / adapt / handoff / agent-bootstrap / propose-patch / batch / visual-diagnose

Classic lifecycle: diagnose / plan / verify / run / sanitize / adapt -> diagnose -> plan -> AI handoff / patch proposal -> verify -> sanitize/share

P98 gate details: knowledge base -> coverage matrix -> trace/cross-framework/training/composite/handoff/batch/sanitize/auto-collector -> master gate

Distribution & Feedback

v6.3.2 is the current package stable baseline. It preserves the evidence-bound locator repair from v6.3.1 and hardens release identity so GitHub tags, checked-out source, package metadata, and PyPI provenance refer to the same commit. v6.3.0 introduced the lite CLI and packaged benchmark, while v6.2.0 is the earlier Local RPA Ops and browser-backend line. The v5.3 stable line includes public-safe case intake, sanitized issue packs, local benchmark/regression gates, cross-adapter support, deployment hardening, documentation/demo adoption assets, and stable API/schema/plugin ABI checks on top of the local Plugin SDK, enterprise governance, hybrid evidence reasoning, the local knowledge base, CI/CD gates, the local web console, regulated-industry mock workflow evaluation, full-chain agent evaluation, OCR/document evidence, offline visual runtime observability, and local-only safety and compliance evaluation.

Safety & Compliance Evaluation

failure-doctor safety-evaluate --project . --out .\safety_report
failure-doctor collect --project . --preset auto --out .\failure_doctor_auto_report --auto-diagnose --auto-handoff --auto-sanitize --safety-evaluate

Safety boundary: local-only, evidence-only, no upload, and no active probe. Safety boundary: no bypass challenges, no access-control evasion, no fingerprint spoofing, no signature cracking, no challenge automation, and no private remediation exposure.

• PyPI release runbook: docs/PYPI_RELEASE.md
• Active probe boundary: docs/ACTIVE_PROBE_BOUNDARY.md
• Behavioral and Client Hints boundary: docs/BEHAVIORAL_CLIENT_HINTS_BOUNDARY.md
• JavaScript integrity boundary: docs/JS_INTEGRITY_BOUNDARY.md
• Canvas fingerprint boundary: docs/CANVAS_FINGERPRINT_BOUNDARY.md
• Visual and data-quality boundary: docs/VISUAL_DATA_QUALITY_BOUNDARY.md
• 2-minute demo script: docs/DEMO_VIDEO_SCRIPT.md
• Technical article draft: docs/TECH_ARTICLE_DRAFT.md
• Real user feedback loop: docs/REAL_USER_FEEDBACK_LOOP.md

After PyPI publication, the target install command is:

pip install agent-failure-doctor

For non-technical Windows users, double-click scripts/windows/Start-FailureDoctor-Diagnosis.bat or drag a failed project folder onto it.

Advanced v3.2 commands include failure-doctor collect and failure-doctor watch.

Advanced evidence inputs include browser_runtime_report.json and input_timing_summary.json. JavaScript/request-integrity evidence can be supplied as js_integrity_report.json. Sanitized Canvas hash/session-count evidence can be supplied in browser_runtime_report.json.

Agent frontend invocation:

failure-doctor agent-bootstrap --target all --project .

This writes .failure-doctor/AGENT_ENTRYPOINT.md plus Codex, Cursor, Claude Code, VS Code/Copilot, Antigravity, OpenCode, Qoder, Trae, WorkBuddy, OpenClaw, Hermes, and generic agent workflow instructions.

Agent Failure Doctor uses a deterministic evidence-based diagnostic engine. It does not claim to solve arbitrary failures, but it provides explainable classification, evidence, fix plans, and before/after verification for known automation failure patterns.

Applied scenario demos are local-only mock workflows for commerce automation, live monitoring, content publishing, GUI data bridge, and ERP sync failure diagnosis.

Spiderbuf-inspired challenge demos are local-only mock failure packs inspired by public crawler-training challenge categories; they validate diagnosis and safe next actions without accessing spiderbuf.cn or publishing private remediation logic.

Integration commands: failure-doctor collect-playwright / failure-doctor pack-logs / failure-doctor adapt

What You Get

report/
|-- diagnosis.json
|-- diagnosis.md
|-- evidence.json
|-- input_summary.json
|-- issue_draft.md
|-- repair_suggestions.md
|-- codex_fix_prompt.md
`-- failure_doctor_report.zip

Agent Failure Doctor turns sanitized automation failure materials into a report that explains what likely failed, what evidence supports the diagnosis, what evidence is missing, and what to ask Codex or another coding assistant to change next.

One-Minute Start

Auto Capture:

failure-doctor run -- python crawler.py
failure-doctor run -- pytest tests/test_listing.py
failure-doctor run -- playwright test

This writes a local run folder under .failure-doctor/runs/<run_id>/:

.failure-doctor/runs/<run_id>/
|-- command.txt
|-- exit_code.txt
|-- stdout.log
|-- stderr.log
|-- environment.json
|-- detected_artifacts.json
|-- input_summary.json
|-- diagnosis/
|-- fix_plan/
|-- verification_hint.md
`-- shareable_failure_pack.zip

The generated safe_to_share.json defaults to safe_to_share=false; review and sanitize before sending a pack to anyone else.

Sanitize & Share Pack:

Sanitize a failed run before sharing it:

failure-doctor sanitize .\.failure-doctor\runs\<run_id> --out .\shareable_failure_pack

This writes redacted logs, redacted network summaries, trace metadata only, a redaction report, a review gate, and shareable_failure_pack.zip.

Raw trace.zip archives are not copied into the sanitized pack.

Put a failed run in a folder:

my_failed_run/
|-- error.log
|-- console.txt
|-- network.json
|-- README.txt
`-- screenshot.png

Then run:

failure-doctor diagnose .\my_failed_run --out .\report

The tool inventories inputs and uses this evidence priority:

trace.zip > log > network.json > user description > screenshot metadata

When evidence is too thin, it should downgrade to insufficient_evidence instead of guessing.

Minimal Demos

Proxy/network failure:

failure-doctor diagnose .\examples\failed_runs\proxy_failed --out .\report_proxy

Strict mode locator conflict:

failure-doctor diagnose .\examples\failed_runs\strict_mode_locator --out .\report_locator

Low-evidence screenshot-only run:

failure-doctor diagnose .\examples\failed_runs\low_evidence_screenshot_only --out .\report_low_evidence

Native Playwright trace fixture:

trace-doctor diagnose .\examples\realistic_playwright_traces\02_login_redirect_302\trace.zip --out .\report_login_trace

Before / After Report

Report structure: conclusion / evidence / why / next action / Codex fix prompt

Before:

page.goto: net::ERR_PROXY_CONNECTION_FAILED while opening https://example.test

After:

Conclusion: network/proxy setup failed before the page loaded.
Evidence: Playwright reported net::ERR_PROXY_CONNECTION_FAILED.
Next action: check proxy settings, DNS, VPN, and CI network configuration.
Codex fix prompt: add trace/log capture and make proxy configuration explicit.

Verify a Fix

failure-doctor diagnose .\failed_run --out .\report
failure-doctor plan .\report --out .\fix_plan
failure-doctor verify --before .\failed_run --after .\rerun_after_fix --out .\verification_report

verify compares before/after evidence and reports whether the original failure is resolved, unchanged, changed into another failure, or insufficiently evidenced.

AI Handoff & Patch Proposal

Turn a report into task packs that Codex, Claude Code, or Cursor can execute:

failure-doctor handoff .\report --target codex --out .\ai_handoff
failure-doctor handoff .\report --target claude_code --out .\ai_handoff
failure-doctor handoff .\report --target cursor --out .\ai_handoff

This writes:

ai_handoff/
|-- ai_handoff.json
|-- ai_handoff.md
|-- codex_task.md
|-- claude_code_task.md
|-- cursor_task.md
|-- affected_files.json
|-- validation_commands.md
|-- forbidden_actions.md
|-- token_budget_report.json
`-- ai_handoff_pack.zip

Generate a dry-run patch proposal without modifying source code:

failure-doctor propose-patch --repo . --report .\report --out .\patch_plan

This writes:

patch_plan/
|-- patch_proposal.md
|-- proposed_changes.json
|-- affected_files.json
|-- validation_commands.md
`-- patch_risk_assessment.json

propose-patch is intentionally proposal-only. It does not edit files, apply patches, run tests, or open pull requests.

v2.5 validation writes validation/ai_handoff_validation.json:

20/20 Codex task files generated
20/20 Claude Code task files generated
20/20 Cursor task files generated
18/20 patch proposals generated
20/20 required sections present
20/20 concise token budget checks pass
0 forbidden outputs

Batch Diagnosis / Fleet Mode

Diagnose many failed runs and get a fleet-level summary:

failure-doctor batch .\runs --out .\batch_report

Input:

runs/
|-- run_001/
|-- run_002/
|-- run_003/
`-- ...

Output:

batch_report/
|-- summary.json
|-- summary.md
|-- failures_by_type.csv
|-- top_root_causes.md
|-- repeated_failures.md
|-- suggested_regression_cases.md
|-- repair_priority.md
`-- reports/

Fleet mode answers which failures repeat, which root causes dominate, which runs should become regression cases, and which fixes deserve priority.

P98 Controlled Maturity

v3.0 starts the P98 controlled maturity track. This is not an ecosystem score; it does not count stars, external PRs, external issues, PyPI downloads, or long-term community adoption.

Current P98 assets:

• docs/P98_CONTROLLED_MATURITY_SCORECARD.md
• knowledge_base/
• docs/CRAWLER_FAILURE_COVERAGE_MATRIX.md
• validation/crawler_failure_coverage_matrix.json

Knowledge-base commands:

python -m tools.knowledge_base.validate_patterns
python -m tools.knowledge_base.search_patterns --query selector_drift
python -m tools.validation.run_crawler_failure_coverage_matrix

Applied Scenario Demos

Local-only mock demos show how Agent Failure Doctor can diagnose failures in:

• hot product collection
• live commerce monitoring
• ecommerce listing automation
• authorized content publishing workflow
• GUI / RPA data bridge
• ERP-to-ecommerce sync

Run:

python -m tools.validation.run_applied_scenario_validation

Spiderbuf-Inspired Challenge Demos

examples/spiderbuf_inspired_challenges/ contains local-only mock failure packs inspired by public crawler-training challenge categories:

• cookie/session required
• iframe extraction
• Ajax dynamic loading
• random CSS selector drift
• infinite scroll missing items
• rate limit 429
• API signature required
• browser fingerprint risk
• Selenium detection risk
• challenge page detected

These cases are diagnosis-only. They do not access spiderbuf.cn, do not include private remediation details, and do not include access-control defeat steps.

python -m tools.validation.run_spiderbuf_inspired_validation

Integrations

Collect Playwright test-results into a failure pack:

failure-doctor collect-playwright .\examples\mock_playwright_test_results --out .\tmp_failure_pack
failure-doctor diagnose .\tmp_failure_pack --out .\tmp_collected_report

Normalize a loose log folder:

failure-doctor pack-logs .\examples\mock_raw_logs --out .\tmp_log_pack
failure-doctor diagnose .\tmp_log_pack --out .\tmp_log_report

Normalize a Selenium, Puppeteer, Cypress, Scrapy, requests, or httpx failure log:

failure-doctor adapt .\examples\cross_framework_fixtures\selenium\no_such_element\raw --framework selenium --out .\tmp_selenium_pack
failure-doctor diagnose .\tmp_selenium_pack --out .\tmp_selenium_report
failure-doctor plan .\tmp_selenium_report --out .\tmp_selenium_fix_plan

Supported adapter frameworks:

selenium | puppeteer | cypress | scrapy | requests | httpx | auto

Playwright remains the deepest native trace backend. Cross-framework adapters normalize local logs and metadata into the same failure lifecycle; they do not run those frameworks or connect to external platforms.

See docs/INTEGRATIONS.md and docs/GITHUB_ACTION_USAGE.md.

Validation Status

Current milestone: Agent Failure Doctor v6.3.2 Release Identity Patch.

Previous stable lines: v6.3.1 Verified Repair and v6.3.0 Lite UX & Bench Core.

• 131 source-ledger records with separated real_public_issue, official_doc_pattern, and public_inspired_sanitized labels
• 50 traceable real public issue records
• 100 Playwright Trace Doctor P95 fixtures
• 100/100 Playwright trace reasonable classifications
• 100/100 Playwright trace exact subtype matches
• 62 external public reference seeds
• 20 external public reference held-out records
• 20/20 external public reference reasonable classifications
• 20/20 external public reference actionable next actions
• 12 resolution validation cases
• 12/12 resolution statuses correct
• 18 applied scenario validation cases
• 18/18 applied scenario reasonable classifications
• 18/18 applied scenario valid fix plans
• 18/18 applied scenario verification statuses correct
• Playwright collector, generic log packer, browser-use adapter, and GitHub Actions usage docs
• v2.0 Auto Capture command wrapper: failure-doctor run -- <command>
• Sanitize & Share command: failure-doctor sanitize <failed_run> --out <shareable_failure_pack>
• Cross-framework adapter command: failure-doctor adapt <input> --framework <framework> --out <failure_pack>
• 100 cross-framework P95 fixtures across Selenium, Puppeteer, Cypress, Scrapy, requests, httpx, browser-use, and generic RPA
• 100/100 cross-framework P95 reasonable classifications
• 100/100 cross-framework P95 valid fix plans
• 0 forbidden outputs in cross-framework P95 validation
• 40 training challenge P95 local-only validation cases
• 40/40 training challenge reasonable classifications
• 40/40 training challenge valid fix plans
• 40/40 training challenge verification statuses correct
• 0 forbidden outputs and 0 private remediation leaks in training challenge validation
• 160 composite P95 strict local-only validation cases
• 160/160 composite primary classifications correct
• 160/160 composite repair-order checks correct
• 160/160 composite evidence graphs generated
• 0 forbidden outputs in composite P95 strict validation
• P95 Core Triad Gate: pass
• 3 composite showcase reports under sample_reports/composite_showcase/
• 10 external held-out public-source records
• 9/10 external held-out reasonable classifications
• 10/10 external held-out actionable next actions
• 0 forbidden outputs in generated reports/prompts
• GitHub Actions green across Ubuntu, macOS, Windows, plus Windows benchmark/smoke/safety

See docs/VALIDATION_REPORT.md, docs/EXTERNAL_DATA_SOURCES.md, and validation/dashboard.md for validation metrics, limits, and boundaries.

Reproduce Validation

python -m tools.real_trace_generation.generate_real_trace_fixtures `
  --out .\examples\realistic_playwright_traces `
  --count 30 `
  --clean
python -m tools.validation.run_real_trace_validation
python -m tools.validation.run_playwright_trace_p95_validation
python -m tools.validation.run_external_public_reference_validation
python -m tools.validation.run_resolution_validation
python -m tools.validation.run_spiderbuf_inspired_validation
python -m tools.validation.run_training_challenge_validation
python -m tools.validation.run_cross_framework_p95_validation
python -m tools.validation.run_composite_diagnosis_p95_strict_validation
python -m tools.validation.run_p95_core_triage_gate
python scripts\validate_external_heldout.py

Android realtime mirror and recording

The Android adapter can use scrcpy as a visual sidecar while ADB/Appium remains the execution and locator layer:

python -m failure_doctor android mirror-check
python -m failure_doctor android mirror-start --serial <ADB_SERIAL>
python -m failure_doctor android record-screen --serial <ADB_SERIAL> --duration 10 --output outputs/mirror/run.mp4

Set RPA_SCRCPY_PATH when scrcpy is not available on PATH. Recording uses a graceful stop so the MP4 container can be finalized. The sidecar does not capture credentials or replace authorized device control.

See docs/LOCAL_CAPABILITY_PUBLICATION_AUDIT.md for the boundary between reusable public modules and local business assets.

Safety Boundary

This project is for local, sanitized failure diagnosis.

It is not:

• a challenge-solving tool
• an access-control circumvention tool
• a credential extractor
• a real-platform scraper
• a tool for unauthorized collection

For suspected platform risk cases, the intended output is identification, routing, and compliance-oriented next steps such as reducing request volume, using an official API, confirming authorization, contacting the platform, or stopping unauthorized collection.

Contributing Failure Cases

You do not need to write code. The most useful contribution is a sanitized failure case: log snippets, trace metadata, network summaries, screenshot metadata, and a short description of what happened.

Open an External failure case issue and remove secrets before posting:

• passwords
• API keys
• cookies
• tokens
• authorization headers
• private screenshots
• private data
• personal data

Accepted input types include sanitized error.log, trace.zip, console.txt, network.json, screenshot metadata, and user_description.txt.

If you allow it, a sanitized case may be assigned an EXT-YYYY-NNNN id, run once with the current released version before rule changes, and added to the external validation dashboard.

Templates and author-generated examples are not counted as external cases.

See CONTRIBUTING.md, docs/external_validation_protocol.md, docs/REAL_TRACE_CONTRIBUTION_GUIDE.md, and docs/REAL_DATA_SOURCES.md.

Commands

Run all tests:

python -m unittest discover -s tests -p "test_*.py"

Run smoke and safety checks:

scripts\smoke_test.ps1
scripts\local_safety_scan.ps1