playwright-use

Natural-language UI tests, no imperative code. Playwright + AI.

https://github.com/user-attachments/assets/1aa826fd-e05b-462b-a717-8a4b448a38ed

Setup

python -m venv .venv && . .venv/Scripts/activate   # Windows PowerShell
pip install -r requirements.txt
python -m playwright install chromium

Configure Azure OpenAI

Create a .env in project root:

AZURE_OPENAI_API_KEY=...
AZURE_OPENAI_ENDPOINT=https://<resource>.openai.azure.com/
AZURE_OPENAI_DEPLOYMENT=<deployment_name>
AZURE_OPENAI_API_VERSION=2024-08-01-preview
LLM_PROVIDER=azure-openai

Other providers

• OpenAI:

LLM_PROVIDER=openai
OPENAI_API_KEY=...
# optional
OPENAI_MODEL=gpt-4o-mini
OPENAI_BASE=https://api.openai.com/v1

• Anthropic (Claude):

LLM_PROVIDER=anthropic
ANTHROPIC_API_KEY=...
ANTHROPIC_MODEL=claude-3-haiku-20240307

• Groq:

LLM_PROVIDER=groq
GROQ_API_KEY=...
GROQ_MODEL=llama3-8b-8192

Run a goal

Headless (default):

python main.py goals/login.goal.yaml

Headed (recommended for debugging):

python main.py goals/login.goal.yaml --headed

Or, when installed as a package (see Packaging):

playwright-use goals/login.goal.yaml --headed

Artifacts are written to runs/<GoalName_Timestamp>/:

• report.html, report.json
• events.log
• trace.zip, *.webm
• step_*.png, step_fail_*.png

Goal file format

name: "Login and Checkout Flow"
url: "https://www.saucedemo.com"
steps:
  - description: "Open the site"
  - description: "Log in with username 'standard_user' and password 'secret_sauce'"
  - description: "Add 'Sauce Labs Backpack' to the cart"
  - description: "Go to cart"
  - description: "Click checkout"
  - description: "Fill in first name 'Toni', last name 'Ramchandani', zip code '411001'"
  - description: "Click Continue"
  - description: "Click Finish"
assertions:
  - "The final page shows a confirmation that order is placed"
  - "URL contains 'checkout-complete'"

Notes:

• ${var} placeholders can be used inside descriptions/assertions and are replaced from an optional vars: map.

Headed vs Headless

• Headed: launches maximized; viewport inherits OS window size for realistic layout.
• Headless: uses a deterministic 1280x800 viewport for reproducibility.

How it works (high-level)

• Each step’s natural-language description is converted into a JSON action plan by Azure OpenAI (core/planner.py).
• Actions are executed via Playwright with robust element resolution (core/healer.py) and resiliency in inputs and checkboxes (core/executor.py).
• Assertions check URL fragments or use a strict LLM oracle (core/oracle.py).
• Reports are generated by core/reporter.py.

Aliases (self-learning)

• The runner caches successful hint→selector mappings per host in fixtures/aliases.yaml.
• You can also predefine entries there. File is hot‑reloaded.

Packaging

Build and install locally:

pip install --upgrade build twine
python -m build
pip install dist/*.whl

Run via console script:

playwright-use goals/login.goal.yaml --headed

Publish to PyPI (requires credentials):

twine upload dist/*

License

This project is licensed under the MIT License. See LICENSE.

See also: ARCHITECTURE.md for internals and diagrams.