autowebtest 0.1.1

Record browser interactions into a JSON recording, then replay them as a repeatable, reportable test

autowebtest

Record browser interactions (clicks, form fills, navigations, and assertions against the DOM/console/network/cookies) into a JSON recording, then replay that recording headlessly (or headed) as a repeatable test, with a generated HTML report at the end.

Project layout

src/autowebtest/       package source (src-layout)
  cli.py                argparse entry point (`autowebtest` console script)
  recorder.py           drives the recording browser session, exposes bridge
                         functions the injected overlay calls into
  player.py             replays a recording.json, writes the HTML report
  browser_log.py        shared console/network buffer used by both of the above
  proxy.py               proxy passthrough for the launched browser
  assets/overlay.js      the in-page recorder/inspector UI, injected via
                         `page.add_init_script`, running in a shadow DOM
tests/                  pytest unit tests for the pure-Python logic above
                         (condition matching, CLI argument/path resolution,
                         HTML report generation) — no browser required
pyproject.toml          package metadata, build backend, pytest config

autowebtest.egg-info/ and .venv/ are local build/dev artifacts (gitignored).

Setup

pip install autowebtest
playwright install chromium

Or for local development, install the package in editable mode from a clone of this repo:

python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
playwright install chromium

Run the test suite with:

pytest

Install the extra engines you plan to use for replay (Chromium is installed by the setup steps above):

playwright install firefox webkit

Quick start

autowebtest record https://google.com recording.json   # record a session
autowebtest run recording.json                         # replay it, headed, in Chromium

A bare filename like recording.json (no directory component) is written under ./testcases/ by convention — the command above actually creates testcases/recording.json. Give an explicit path (e.g. checkout/login.json) to opt out. autowebtest run looks a bare filename up under ./testcases/ too if it isn't sitting in the current directory.

Replaying writes a pass/fail summary to the console and an HTML report under ./reports/, named after the recording (recording.json → reports/recording.html).

Recording a session

autowebtest record https://google.com recording.json

This opens a Chromium window on the given URL with a toolbar (Start / Stop) injected at the top of the page.

• Click Start to begin recording. From that point on, clicks, form fills, and page navigations are captured automatically as steps.
• Right-click anywhere on the page (once recording) to open the recorder's menu:
  • Inspect Element — opens the inspector panel (below), focused on whichever element you just right-clicked.
  • Stop Recording — same as clicking Stop in the toolbar.
• Click Stop in the toolbar (or use the menu) to end the session. The browser closes and the recording is written to the output path you gave (recording.json above).

The inspector panel

The inspector is a persistent panel docked to the bottom of the page, similar to a browser's DevTools, with four tabs: Elements, Console, Network, and Cookies. Switching tabs swaps the list on the left; the right-side builder panel adapts to whichever tab is active and is how you turn something you clicked on into a recorded assertion step.

• Drag the handle at the top edge of the panel to resize it; dragging it down past a minimum height snaps it to a minimized, header-only bar (or click the ▁ / ▔ button in the header to toggle minimize/maximize directly). The panel keeps recording while minimized — maximize it again (or re-select an element) to see the tree.
• Close the inspector with the ✕ button in its header or the Escape key.
• Every tab's builder has Save (records the condition as a step; the panel stays open so you can keep adding more) and Clear (resets the builder without saving).

Elements tab

The default tab. Shows the full page as a scrollable, expandable DOM tree, rendered like a real browser's Elements panel — actual opening tags with every attribute inline (e.g. <link rel="canonical" href="https://example.com/">), text-node content as its own indented leaf line, and a closing tag once a node is expanded.

• Click the ▸/▾ arrow to expand/collapse a node's children; click a node's opening tag to select it.

• Track pointer — a checkbox at the top of the tab. When checked, hovering any element on the page (not just in the tree) highlights it with its tag#id.class badge, and left-clicking it re-opens/re-focuses the inspector on that element instead of performing the page's normal click action — uncheck it to interact with the page normally again. Off by default; stays checked across closing/reopening the inspector within the same origin, and resets to off when you stop recording or navigate cross-origin.

• Search — a text box (like Cmd+F in DevTools) that matches against each element's rendered opening tag plus its own text, so you can search loosely (a tag name, .a-class, #an-id) or precisely with quoted attribute syntax exactly as it appears in the source, e.g. searching rel="canonical" matches only <link rel="canonical" ...>, not <link rel="stylesheet" ...>. Use the ▲/▼ buttons or Enter/Shift+Enter to cycle through matches; each match jumps to, expands, and selects that node.

• Selecting a node (by clicking it in the tree, via search, or via right-click → Inspect Element) draws a persistent (dashed) highlight around that element on the page, so you can see exactly what you're targeting even after you stop hovering — it tracks the element's position on scroll/resize and clears when you select something else.

• Selecting a node fills the builder with:

  • Tag — the element's tag, editable via a dropdown of common tags.
  • Field — a scrollable list of every attribute the element has (id, class, href, src, data-*, aria-*, etc.), plus a synthetic Text / value row (its trimmed text content, or value for form controls) which is selected by default. Click any row to build the condition against that field instead — the Value field is pre-filled from whichever field is selected and shows which attribute it's checking.
  • Condition — Exists, Does not exist (these hide the Value field — they check only that a matching element is/isn't on the page, by its recorded selector), Equal, Equal (ignore case), Greater than, Greater or equal, Smaller than, Smaller or equal, Like (SQL-style, % as wildcard, e.g. %openai%), or Custom regex.
  • Value — pre-filled from the selected field; edit as needed.
  • Choosing Custom regex reveals an extra Regex field (pre-filled from the element's id/class + tag) — the assertion then matches by regex instead of by condition, but still checks it against whichever field is selected.

  Example: select the <h1>, leave Field on Text / value, set Condition to Like, Value to %Search results% → records an assertion step that passes if any h1 matching the recorded selector contains that text. Or select a <link rel="canonical">, set Condition to Exists → records a step that fails if that link disappears from a page later.

Console tab

Lists every console message logged by the page so far. Clicking a message fills the builder with its log type and text; Log type can be narrowed to log/info/warning/error/debug or left as Any, and Condition supports the same vocabulary as Elements plus Contains, Exists, and Does not exist (the last two ignore the Value field).

Example: click a captured console.error("Failed to fetch") message, set Log type to error, Condition to Contains, Value to Failed to fetch → records a step that fails if that error is logged during replay.

Network tab

Lists every resource the page has loaded, filterable by URL substring via the text box above the list. Clicking a resource fills the builder with its URL as the URL pattern (matched by Contains or Regex) and a Check:

• Resource present — just an Exists/Does not exist toggle on whether a matching request was made at all.
• Status code — numeric condition (Equal, Greater than, etc.) against the response's HTTP status.
• HTTP method — condition against the request's HTTP method (e.g. Equal / POST), pre-filled from whichever resource you clicked.
• Response header — a header name field plus the same condition vocabulary as Console, checked against that header's value.

Example: click a POST https://api.example.com/save entry, set Check to HTTP method, Condition to Equal, Value to POST → records a step that fails if that endpoint is ever called with a different method. Or set Check to Status code, Condition to Equal, Value to 200.

Cookies tab

Lists the page's cookies (including httpOnly ones, read on the Python side via the browser context rather than document.cookie). Clicking a cookie fills the builder with its name and value; Condition works the same as Console/Network.

Example: click the session_id cookie, set Condition to Exists → records a step that fails if the user is ever logged out (cookie missing) during replay.

Recording format

{
  "url": "https://google.com",
  "startedAt": "2026-07-10T12:00:00.000Z",
  "endedAt": "2026-07-10T12:05:00.000Z",
  "steps": [
    {"type": "start", "timestamp": "...", "url": "...", "title": "Google"},
    {"type": "click", "timestamp": "...", "selector": "#hplogo", "tag": "img", "id": "hplogo", "classes": [], "text": "Google", "href": null, "actionableKind": "button"},
    {"type": "input", "timestamp": "...", "selector": "input[name='q']", "tag": "input", "value": "openai", "ariaLabel": "Search"},
    {"type": "navigation", "timestamp": "...", "url": "https://www.google.com/search?q=openai", "title": "openai - Google Search"},
    {"type": "assertion", "timestamp": "...", "tag": "div", "selector": "div.g", "condition": "like", "value": "%Search results%"},
    {"type": "assertion", "timestamp": "...", "tag": "a", "selector": "a.result-link", "attribute": "href", "condition": "like", "value": "%openai.com%"},
    {"type": "assertion", "timestamp": "...", "tag": "span", "selector": "#count", "condition": "greaterEqual", "value": "10"},
    {"type": "assertion", "timestamp": "...", "tag": "div", "selector": "div.g", "condition": "regex", "regex": "^div\\.g$", "value": "Search results"},
    {"type": "assertion", "timestamp": "...", "tag": "link", "selector": "link[rel='canonical']", "condition": "exists", "value": ""},
    {"type": "console", "timestamp": "...", "logType": "error", "condition": "contains", "value": "Failed to fetch"},
    {"type": "console", "timestamp": "...", "logType": "any", "condition": "exists", "value": ""},
    {"type": "resource", "timestamp": "...", "urlPattern": "/api/user", "matchType": "contains", "check": "exists", "condition": "exists", "value": ""},
    {"type": "resource", "timestamp": "...", "urlPattern": "\\.png$", "matchType": "regex", "check": "status", "condition": "equal", "value": "200"},
    {"type": "resource", "timestamp": "...", "urlPattern": "/api/save", "matchType": "contains", "check": "method", "condition": "equal", "value": "POST"},
    {"type": "resource", "timestamp": "...", "urlPattern": "/api/user", "matchType": "contains", "check": "header", "headerName": "content-type", "condition": "like", "value": "%json%"},
    {"type": "cookie", "timestamp": "...", "name": "session_id", "condition": "exists", "value": ""},
    {"type": "cookie", "timestamp": "...", "name": "locale", "condition": "equal", "value": "en-IN"},
    {"type": "stop", "timestamp": "...", "url": "...", "title": "openai - Google Search"}
  ]
}

• assertion steps' condition is exists/notExists (element-tab only, checked purely against the recorded selector — no tag-name fallback), or one of equal, equalIgnoreCase, greater/greaterEqual/smaller/ smallerEqual, like, or regex (which uses the separate regex field instead of selector/tag to find candidates).
• console/resource/cookie steps' condition is equal, equalIgnoreCase, contains, like, regex, greater/greaterEqual/smaller/smallerEqual, or exists/notExists (which ignore value). logType is one of any/log/info/warning/error/debug.
• resource steps' matchType is contains/regex (matched against the resource's full URL). Its check is exists, status, method, or header (header also carries headerName, looked up case-insensitively).

Replaying a recording

autowebtest run recording.json                # Chromium (default)
autowebtest run recording.json --firefox       # or --mozilla
autowebtest run recording.json --safari        # or --webkit
autowebtest run recording.json --headless      # run without a visible window

This opens the target url from the recording, then replays each step in order:

• click / input steps re-run against the saved selector (page.click / page.fill).
• assertion steps with condition: "regex" re-check the saved regex against every element's tag#id.class descriptor (and its HTML as a fallback); it passes if a matching element's checked field (its text/value, or the saved attribute if one was recorded) contains the saved value.
• assertion steps with condition: "exists"/"notExists" pass iff the saved selector does/doesn't match anything on the page — no fallback scan, so it's a precise presence check for that exact element.
• Other assertion steps look up candidates by the saved selector (falling back to every element with the saved tag), then check the saved condition against each candidate's checked field (text/value, or the saved attribute if one was recorded): equal, equalIgnoreCase (string comparison), greater/greaterEqual/smaller/smallerEqual (numeric if both sides parse as numbers, else lexicographic), or like (% treated as a wildcard, case-insensitive). It passes if any candidate satisfies the condition.
• start / stop / navigation steps are informational and skipped — any navigation triggered by a preceding click happens naturally and the player waits for the network to settle before moving on. Navigation prints the destination page's saved title ("Navigation to <title> page", falling back to the URL if no title was captured).
• click and input log lines describe what was interacted with based on the element kind (captured at record time in actionableKind/ariaLabel), falling back to the raw selector when the relevant label is empty:
  • input / select — the field's aria-label ("Click on '<aria-label>' field" / "Fill '<aria-label>' field with '<value>'").
  • anchor (<a>) — its text plus the href it points to ("Click on '<text>' link (href=<href>)").
  • button — the text of the first non-empty <span> nested inside it, falling back to the button's own text or aria-label/title for icon-only buttons ("Click on '<text>' button").
• console steps pass if any captured console message (optionally narrowed by logType) satisfies the condition against its text — or, for exists/ notExists, if any/no such message was captured at all.
• resource steps first find every network response/failure whose URL matches urlPattern (substring or regex, per matchType). For check: "exists" it passes iff a match was found (exists) or wasn't (notExists); for "status" it passes if any match's HTTP status satisfies the condition; for "method" it passes if any match's HTTP method satisfies the condition; for "header" it passes if any match's headerName response header satisfies the condition.
• cookie steps look up the named cookie via the browser context (so httpOnly cookies are visible too) and check its value against the condition, or its presence for exists/notExists.

A pass/fail line is printed per step, with a summary at the end. The process exits with a non-zero status if any step failed, so it can be used as a CI check.

--safari/--webkit uses Playwright's WebKit engine, the closest automatable stand-in for Safari — there is no true Safari automation outside Apple's own WebDriver on macOS.

HTML report

Every autowebtest run also writes a self-contained HTML report under ./reports/, named after the recording with its extension swapped to .html — e.g. testcases/recording.json → reports/recording.html, or testcases/checkout.v2.json → reports/checkout.v2.html. The console prints its path (autowebtest: HTML report written to ...) after the run summary.

The report has a header with the recording name, target URL, engine, and generation time; summary cards for overall result and passed/failed/skipped counts; and a table of every step with its action description, pass/fail/skip badge, and failure reason (for failed steps). It has no external dependencies, so it can be opened directly in a browser or attached to a CI artifact.

Known limitations (v1)

• Toolbar recording state is kept in sessionStorage, which is scoped per-origin. It's reconciled against the Python-side session on every navigation, so the recording on/off indicator stays accurate across origins. Steps already captured are unaffected either way since they're stored on the Python side. Track pointer (the checkbox in the Elements tab) follows the same per-origin sessionStorage persistence — it stays checked across closing/reopening the inspector or navigating within the same origin, but resets to off on a cross-origin navigation or when you stop recording.
• Replay is best-effort: it doesn't wait/retry beyond a fixed timeout per step, and assertion matching is a heuristic (selector/tag scan + condition or regex check against text/value), not a strict re-derivation of what was clicked during recording.
• console/resource assertions are evaluated against everything captured from page load up to that point during replay, not strictly scoped to "since the previous step" — the same best-effort philosophy as DOM assertions above.