wpt-gen 0.3.0

A CLI Tool for AI-Powered Web Platform Test Generation

WPT-Gen: AI-Powered Web Platform Test Generation

WPT-Gen is an agentic CLI tool designed to increase browser interoperability by automating the creation of Web Platform Tests (WPT).

By bridging the gap between W3C Specifications and local WPT repositories, WPT-Gen uses Large Language Models (LLMs) to proactively identify testing gaps and generate high-quality, compliant test cases.

Why WPT-Gen?

Browser interoperability is critical for the web. While the W3C and WHATWG write specifications, there is often a gap between those specs and the tests that ensure browsers implement them correctly. WPT-Gen bridges this gap by:

• Reducing Manual Effort: Automating the tedious process of mapping spec assertions to existing tests.
• Ensuring High Coverage: Identifying missing edge cases and suggesting specific test scenarios.
• Standardizing Compliance: Generating tests that adhere to strict WPT style guides and directory structures.

Key Features

• Context Assembly: Automatically resolves Web Feature IDs (via web-features) to fetch specification and MDN documentation (if available), using BeautifulSoup and Markdownify to preserve critical code blocks and semantic structure.
• Deep Local Analysis: Scans your local WPT repository using WEB_FEATURES.yml metadata to identify existing tests and their dependencies.
• Gap Analysis: Compares technical requirements synthesized from specifications against current test coverage to pinpoint missing assertions.
• Test Suggestions: Brainstorms specific, actionable test scenarios (blueprints) that address identified gaps.
• Automated Generation: Produces atomic, WPT-compliant HTML and JavaScript test files based on user-approved blueprints using an autonomous agent powered by google-adk.
• Multi-Provider Support: Built-in support for Google Gemini (via google-genai and thinking models), OpenAI, and Anthropic models.

How it Works

WPT-Gen follows a structured, multi-phase agentic workflow. Each phase is designed to build upon the last, culminating in high-quality, verified test cases.

flowchart TD
    subgraph Context[Context Assembly]
        A[Web Features] --> B[Scrape Specs/MDN]
        C[Local WPT Repo] --> D[Index Existing Tests]
    end

    subgraph Analysis[Requirement & Gap Analysis]
        B & D --> E{{Requirements Extraction}}
        E --> F{{Coverage Audit}}
        F --> G[Test Blueprints]
    end

    subgraph Generation[Test Generation]
        G --> H{{Test Generation}}
    end

    classDef llm fill:#4B0082,stroke:#333,stroke-width:2px;
    class E,F,H llm;

Workflow Phases

For an in-depth explanation of the internal logic, inputs, outputs, and LLM integrations for each phase, see the Workflow Phases Documentation.

1. Phase 1: Context Assembly: Aggregates the "Source of Truth" from external documentation (W3C Specs, MDN) and identifies existing test coverage in the local WPT repository.
2. Phase 2: Requirements Extraction: Uses an LLM to synthesize specification text into structured, granular technical requirements. Supports parallel and iterative extraction modes for complex specs.
3. Phase 3: Coverage Audit: Performs a delta analysis by comparing the synthesized requirements against the local test suite. This phase outputs an audit worksheet and high-level test blueprints.
4. Phase 4: Test Generation: Translates user-selected blueprints into functional WPT-compliant code (JavaScript, Reftests, or Crashtests) using an autonomous agent powered by google-adk, which leverages specialized file-system tools and style guide instructions.

Prerequisites

• Python 3.10+
• Local WPT Repository: A local checkout of web-platform-tests/wpt.
• API Key: An API key for a supported LLM (Gemini, OpenAI, or Anthropic).

Installation

From PyPI (Recommended)

pip install wpt-gen

From Source (Development)

# Clone the repository
git clone https://github.com/GoogleChromeLabs/wpt-gen.git
cd wpt-gen

# Install the package (using a virtual environment is recommended)
python -m venv .venv
source .venv/bin/activate
make install

Configuration

WPT-Gen uses a combination of a YAML configuration file and environment variables.

1. Environment Variables

You must export the API key for your chosen provider. These are never stored on disk.

export GEMINI_API_KEY="your_gemini_api_key"
# OR
export OPENAI_API_KEY="your_openai_api_key"
# OR
export ANTHROPIC_API_KEY="your_anthropic_api_key"

2. YAML Configuration (wpt-gen.yml)

Run wpt-gen init to generate a wpt-gen.yml configuration file with sensible defaults:

wpt-gen init --wpt-path /path/to/your/local/wpt

This will create a configuration file that looks like this:

default_provider: gemini
wpt_path: /path/to/your/local/wpt  # Path to your local WPT checkout

providers:
  gemini:
    default_model: gemini-3.1-pro-preview
    categories:
      lightweight: gemini-3-flash-preview
      reasoning: gemini-3.1-pro-preview
  openai:
    default_model: gpt-5.2-high
    categories:
      lightweight: gpt-5-mini
      reasoning: gpt-5.2-high
  anthropic:
    default_model: claude-opus-4-6
    categories:
      lightweight: claude-sonnet-4-6
      reasoning: claude-opus-4-6


phase_model_mapping:
  requirements_extraction: reasoning
  coverage_audit: reasoning
  generation: lightweight

3. Managing Configuration via CLI

You can use the built-in config command group to view or modify your settings without opening the YAML file manually.

• View Configuration:

  wpt-gen config show
  

• Update Configuration: Use dot-notation to set specific values.

  wpt-gen config set default_provider "openai"
  wpt-gen config set providers.gemini.default_model "gemini-3.1-pro-preview"
  

Usage

The primary interface is the generate command, which requires a Web Feature ID (as defined in the web-features repository).

Basic Generation

wpt-gen generate font-family

Common Options

Option	Shorthand	Description
web_feature_id	(Arg)	Required. The ID of the feature (e.g., font-family, popover).
--provider	-p	Override the default LLM provider (gemini, openai, anthropic).
--wpt-dir	-w	Override the path to the local web-platform-tests repository.
--draft		Enable fetching metadata from the draft features directory.
--config	-c	Path to a custom wpt-gen.yml file.

Note: You can run wpt-gen generate --help to see a full list of all 20+ options (including manual overrides). For more detailed information, see the CLI Command Reference.

Other Commands

• wpt-gen clear-cache: Clears local cache of specs and LLM responses.
• wpt-gen version: Displays the current version.

Development

Running Tests

We use pytest for unit and integration testing. Run tests easily via:

make test

Linting & Formatting

We use ruff to maintain code quality and mypy for type checking. You can run these using make commands:

# Lint and format
make lint-fix

# Type check
make typecheck

# Run all checks (format, lint, typecheck, tests)
make check

# Prepare for PR (runs lint-fix, typecheck, and test)
make presubmit

Release Process

WPT-Gen utilizes a CI/CD pipeline via GitHub Actions. Creating and publishing a new GitHub Release (e.g., v1.0.0) automatically triggers the publish.yml workflow, which securely builds and uploads the package to PyPI. For more details, see the Contributing Guide.

AI Assistant Integration

This repository includes a GEMINI.md and a suite of AI skills in the .agents/skills/ directory to help AI assistants (like Gemini Code Assist) better understand the project's architecture, dependencies, and testing standards. You can point your AI assistant to GEMINI.md for a comprehensive overview of how to contribute accurately to the codebase.

License

Apache 2.0. See LICENSE for more information.

Source Code Headers

Every file containing source code must include copyright and license information.

Apache header:

Copyright 2026 Google LLC

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.