Shan Claw: Autonomous QA Prototype

This project is a practical Python prototype inspired by the article: "From Test Automation to Autonomous QA: Are We Finally Closing the Feedback Gap?"

It demonstrates what can be implemented quickly in VS Code + Python:

• Requirement parsing from PRDs/user stories
• Coverage gap detection against existing tests
• Just-in-time test generation templates
• Flaky test self-heal suggestions for selector failures
• Failure RCA assistance based on logs and diffs
• Dependency guardrails with optional VirusTotal checks
• OpenClaw-backed execution mode (local or remote)

Quick Start

1. Create and activate a virtual environment.
2. Install dependencies:

pip install -r requirements.txt

1. Run the autonomous QA flow with sample data:

python -m shan_claw.cli run \
  --requirements examples/requirements.txt \
  --tests examples/existing_tests.txt \
  --logs examples/failure.log \
  --diff examples/code.diff \
  --deps examples/dependencies.txt

1. Run the same flow through OpenClaw provider:

python -m shan_claw.cli run \
  --llm-provider openclaw \
  --openclaw-mode local \
  --requirements examples/requirements.txt \
  --tests examples/existing_tests.txt \
  --logs examples/failure.log \
  --diff examples/code.diff \
  --deps examples/dependencies.txt

1. Use OpenClaw remote mode (cloud relay):

set OPENCLAW_API_KEY=your_api_key_here
python -m shan_claw.cli run \
  --llm-provider openclaw \
  --openclaw-mode remote \
  --openclaw-agent-id your_agent_id \
  --requirements examples/requirements.txt \
  --tests examples/existing_tests.txt \
  --logs examples/failure.log \
  --diff examples/code.diff \
  --deps examples/dependencies.txt

1. Output is written to output/report.json and generated tests under generated_tests/.

2. Generate an on-demand regression environment blueprint:

python -m shan_claw.cli provision --service payment-v2 --out-dir output/env

This creates a Docker Compose file and seed data JSON for fast test environment setup.

MCP Server (IDE + CLI)

Shan-Claw includes a Model Context Protocol server so IDEs can call QA tools over stdio.

Run from command line

shan-claw-mcp

or:

python -m shan_claw.mcp_server

Start from Shan-Claw CLI

python -m shan_claw.cli mcp-serve

VS Code MCP configuration example

Add to your VS Code MCP settings:

{
  "mcp": {
    "servers": {
      "shan-claw": {
        "command": "shan-claw-mcp"
      }
    }
  }
}

Cursor MCP configuration example

Add to ~/.cursor/mcp.json:

{
  "mcpServers": {
    "shan-claw": {
      "command": "shan-claw-mcp"
    }
  }
}

See full setup and tool catalog in docs/MCP_SERVER.md.

Article-to-Code Map

The article describes OpenClaw as the engine for collapsing the QA feedback loop. In this repository, that vision is implemented through an OpenClaw-backed provider path plus deterministic local fallbacks.

Article claim	What is implemented here	Code paths
Requirement Analysis -> Intelligent Parsing	Requirements are parsed from text artifacts, normalized, tagged, and compared against existing tests. In OpenClaw mode, the same step is delegated to an OpenClaw agent and parsed back into structured requirement objects.	shan_claw/coverage.py, shan_claw/openclaw_provider.py, examples/requirements.txt, examples/existing_tests.txt
Test Creation -> Just-in-Time Generation	Coverage gaps produce generated Playwright test files. In local mode, generation uses a deterministic template. In OpenClaw mode, the provider asks OpenClaw for test code first and falls back to the template if needed.	shan_claw/testgen.py, shan_claw/openclaw_provider.py, generated_tests/test_as-a-customer-i-can-complete-payment-with-a-saved-card.spec.ts, generated_tests/test_as-an-admin-i-can-search-orders-by-status.spec.ts
Flaky Tests -> Self-Healing Loops	Failure logs are scanned for brittle selector patterns and converted into healing actions. In OpenClaw mode, the provider can ask the agent for healing recommendations before using local heuristics.	shan_claw/self_heal.py, shan_claw/openclaw_provider.py, examples/failure.log
Environment Setup -> On-Demand Infrastructure	A regression environment blueprint can be generated from the CLI, producing a Docker Compose definition and seed data for a target service such as payment-v2.	shan_claw/infra.py, shan_claw/cli.py, output/env/docker-compose.yml, output/env/seed-data.json
Failures -> Assisted RCA + Fixes	Logs and diffs are correlated into likely causes, supporting evidence, and suggested fixes. In OpenClaw mode, the RCA step can be delegated to the agent and normalized back into the same result structure.	shan_claw/rca.py, shan_claw/openclaw_provider.py, examples/failure.log, examples/code.diff, output/report.json
Security -> Real-time Supply Chain Guardrails	Dependencies are parsed, hashed, checked against a blocklist, and optionally queried against VirusTotal. Blocked packages can be annotated with OpenClaw-suggested safer alternatives.	shan_claw/security.py, shan_claw/openclaw_provider.py, examples/dependencies.txt
From automation scripts to autonomous feedback systems	The pipeline is orchestrated as a single workflow with provider selection. The same CLI can run in local mode or through OpenClaw, which makes the repo a concrete bridge from scripted automation toward agent-backed QA feedback.	shan_claw/core.py, shan_claw/cli.py, shan_claw/openclaw_provider.py
Collapsing the feedback loop	The run command links requirement ingestion, coverage analysis, test generation, healing, RCA, and dependency screening into one pass that produces a machine-readable report.	shan_claw/cli.py, shan_claw/core.py, output/report.json

OpenClaw's role in this repository

• shan_claw/openclaw_provider.py is the main integration layer.
• shan_claw/core.py selects between local and openclaw providers.
• shan_claw/cli.py exposes --provider openclaw, --openclaw-mode local, and --openclaw-mode remote.
• In remote mode, the provider uses OPENCLAW_API_KEY or CMDOP_API_KEY plus an optional agent ID.

What is fully implemented vs. scaffolded

• Fully implemented: provider selection, OpenClaw package integration, local OpenClaw execution path, deterministic fallbacks, generated reports, regression environment blueprint generation, and dependency checks with optional VirusTotal lookups.
• Scaffolded for real enterprise rollout: direct PRD system ingestion, real DOM re-scan and auto-rewrite of selectors in browser sessions, automatic patch application back into application code, and production-grade policy/governance workflows.

Notes

• VirusTotal checks require VIRUSTOTAL_API_KEY in your environment.
• OpenClaw package is installed from PyPI and uses CMDOP under the hood.
• If OpenClaw is unavailable or returns non-JSON output, this project falls back to local deterministic logic.
• The framework is intentionally lightweight and extensible for enterprise adaptation.

Traceability Links

Architecture and design references:

• TECHNICAL_ARCHITECTURE.md
• STREAMLIT_DEMO.md

Operational runbooks and scripts:

• EXAMPLES.md
• run-examples.bat
• run-examples.sh
• run-streamlit.bat
• run-streamlit.sh
• run-audit.bat
• run-audit.sh

Release and publishing governance:

• PUBLISHING.md
• RELEASE_NOTES_v0.1.0.md
• RELEASE_NOTES_v0.2.0.md
• RELEASE_NOTES_v0.3.0.md
• RELEASE_NOTES_v0.4.0.md
• RELEASE_NOTES_v0.4.1.md
• RELEASE_NOTES_v0.5.0.md
• .github/workflows/publish-testpypi.yml
• .github/workflows/publish-pypi.yml
• .github/workflows/security-audit.yml

MCP integration references:

• MCP_SERVER.md
• shan_claw/mcp_server.py

Core implementation references:

• shan_claw/core.py
• shan_claw/openclaw_provider.py
• shan_claw/cli.py