Aumate QA Agent

Framework-agnostic auto-healing for test automation. Works with any test framework, any language, any LLM provider.

your-project/
├── agent/           ← copy from this repo (or pip install)
├── qa_heal.py       ← standalone runner
└── .qa-agent/
    └── config.yaml  ← auto-generated

Quick Start (30 seconds)

Copy into your project:

cp -r agent qa_heal.py /path/to/your-project/
cd /path/to/your-project

Run healing:

python qa_heal.py                        # heal broken tests
python qa_heal.py --apply                 # heal + apply fix to disk
python qa_heal.py --test                  # just run tests, no healing
python qa_heal.py --test-file tests/foo.spec.ts --test-name "my test"

Or install via pip:

pip install aumate-qa-agent
qa-agent init
qa-agent heal

How It Works

Your existing tests run normally
        |
        v
Failure detected (locator, timing, assertion, etc.)
        |
        v
DOM evidence collected from your project
        |
        v
Candidates generated (deterministic + optional AI)
        |
        v
Each candidate tested in isolated sandbox copy
        |
        v
PASS → Patch displayed (or auto-applied)
FAIL → Next candidate tried

No Test Code Changes Needed

The agent observes your existing tests. You don't modify a single line of test code.

Safety Guarantees

• No LLM has direct write access — the validation engine is the sole gatekeeper
• Application defects are never hidden — only automation defects (broken locators, timing) are healed
• Sandbox validation — patches are tested in an isolated copy of your project first
• Secrets are masked — no credentials, tokens, or passwords sent to LLMs

Supported Frameworks (built-in)

Adding a new adapter requires implementing one interface:

from agent.adapters.base import FrameworkAdapter

class MyAdapter(FrameworkAdapter):
    @property
    def name(self) -> str: ...
    def detect(self, project_root: str) -> AdapterDetectionResult: ...
    def run_tests(self, project_root, test_file, test_name, custom_command) -> TestRunResult: ...
    def parse_test_result(self, raw_output, exit_code, duration) -> TestRunResult: ...
    def capture_evidence(self, project_root, failure) -> FailureEvidence: ...
    def patch_code(self, source_code, old_locator, new_locator) -> str: ...

Third-party adapters can be shipped as separate pip packages:

# in their pyproject.toml
[project.entry-points."qa_agent_adapter"]
cypress = "my_package:CypressAdapter"

Supported LLM Providers

# Configure LLM
python qa_heal.py --llm-provider openai --llm-model gpt-4o-mini

# Or set env var
set GEMINI_API_KEY=your-key

Works without any LLM configured — deterministic healing runs fully offline.

Configuration

Auto-generated at .qa-agent/config.yaml on first run:

project:
  name: my-project
  framework: playwright
  language: typescript
  test_command: npx playwright test

healing:
  mode: approval          # "auto" | "approval" | "safe"
  auto_apply_threshold: 95.0
  deterministic_first: true

llm:
  primary:
    provider: gemini
    model: gemini-2.5-flash

security:
  mask_secrets: true
  strict_privacy: false

validation:
  rerun_failed_test: true
  run_related_tests: true

Healing Memory

Successful fixes are cached in ~/.qa-agent/memory.db. Recurring failures skip the LLM entirely and apply the historical fix immediately.

CLI Commands

qa-agent init          # auto-detect project + generate config
qa-agent detect        # show detected framework, language, runner
qa-agent doctor        # system health check
qa-agent config llm    # configure LLM provider
qa-agent test          # run tests through agent
qa-agent heal          # auto-heal failures
qa-agent heal --apply  # heal + write fix to source
qa-agent history       # view healing memory
qa-agent version       # show version

Project Structure

aumate-qa-agent/
├── agent/
│   ├── core/
│   │   ├── orchestrator.py      # main workflow
│   │   ├── execution_context.py # standardized data model
│   │   ├── failure_detector.py  # classify failures
│   │   ├── confidence_engine.py # score candidates
│   │   ├── validation_engine.py # sandbox testing
│   │   ├── plugins.py           # adapter discovery/registry
│   │   └── test_runner.py       # universal test execution
│   ├── adapters/
│   │   ├── base.py              # adapter interface
│   │   ├── playwright.py        # Playwright adapter
│   │   └── selenium_adapter.py  # Selenium adapter
│   ├── healing/
│   │   ├── base.py              # strategy interface
│   │   ├── locator.py           # deterministic locator healing
│   │   ├── timing.py            # timing/race condition healing
│   │   ├── assertion.py         # assertion healing (guarded)
│   │   └── ai_reasoner.py       # AI-powered healing
│   ├── ai/
│   │   ├── provider.py          # LLM base interface
│   │   ├── router.py            # multi-provider registry
│   │   └── providers/           # Gemini, OpenAI, Groq, Ollama...
│   ├── memory/healing_history.py  # SQLite persistence
│   ├── git/manager.py           # diff, patch, branch
│   ├── project/                  # detection, config, doctor
│   └── security/masking.py      # secret redaction
├── cli/main.py             # CLI entry point
├── qa_heal.py              # standalone runner (drop into any project)
├── pyproject.toml          # pip-installable package
└── ARCHITECTURE.md         # full architecture spec

Requirements

• Python 3.9+
• Your existing test framework (Playwright, Selenium, etc.)
• Node.js (for Playwright/Cypress Web projects)

License

MIT