visualcheck
Visual regression checking with:
- Figma-first baselines (optionally synced via Figma API)
- Runtime baselines (auto-create when missing; never overwrite by default)
- Pixel diff % + SSIM (with optional resize-on-mismatch + WARN)
- Ignore regions (mask dynamic areas via selectors + explicit rects)
- Self-contained HTML report +
report.json
Install
pip install visualcheck
playwright install chromium
Config (visualcheck.yaml)
project: consumer_website
suite: daily_sanity
envs:
prod: "https://example.com"
views:
desktop_profiles: ["desktop_1440x900", "macbook_1440x900"]
mobile_devices: ["iPhone 15 Pro Max", "iPhone 13 mini", "Pixel 7"]
pages:
- id: home
url: "/"
wait_for: "body"
full_page: true
# Optional flows (multi-step user journeys)
flows:
- id: search_flow
start_url: "/"
steps:
- action: click
selector: "text=Search"
- action: wait
ms: 500
snapshots:
- id: after_search
wait_for: "body"
full_page: true
baseline:
# Locked resolution order:
# figma -> runtime -> create runtime baseline (if enabled)
priority: ["figma", "runtime"]
create_if_missing: true
never_overwrite: true
on_created: "INFO" # INFO|WARN|FAIL
# Optional legacy mode: copy figma image into runtime when figma is used.
# Default behavior keeps runtime as website-captured truth.
seed_runtime_from_figma: false
compare:
resize_on_mismatch: true
mismatch_level: "WARN" # WARN|FAIL
thresholds:
max_pixel_diff_pct: 0.10
min_ssim: 0.995
# Optional per-snapshot override + region-level checks
by_snapshot:
home:
max_pixel_diff_pct: 0.15
min_ssim: 0.992
regions:
- name: header_strict
x: 0
y: 0
width: 1440
height: 120
max_pixel_diff_pct: 0.03
min_ssim: 0.998
# Optional ignore-region suggestions from diff output
suggest_ignore_regions:
enabled: true
min_area: 2500
max_regions: 5
capture:
# single channel (fallback to bundled Chromium if unavailable)
browser_channel: "chrome"
# optional cross-browser matrix in one run (view ids become <view>@<channel-slug>)
browser_channels: ["chrome", "msedge"]
extra_wait_ms: 1000
scrolls: 5
scroll_delay_ms: 700
retries: 1
retry_delay_ms: 700
# Optional page stabilization before screenshot
stability_checks: 2
stability_interval_ms: 250
stability_timeout_ms: 3000
# Strong load gate: wait for document complete + network idle best-effort
# + images/lazy-images settled before screenshot.
wait_for_full_load: true
full_load_timeout_ms: 120000
full_load_check_interval_ms: 500
full_load_stable_checks: 2
# Additional lazy-load settling rounds (scroll to bottom and settle).
lazy_load_scroll_rounds: 6
screenshot_timeout_ms: 90000
stability:
# auto-rerun unstable snapshots and choose final outcome by consensus
reruns_on_warn: 1
reruns_on_fail: 2
require_consensus: "best" # best|median|majority
# optional flaky auto-quarantine policy
auto_quarantine:
enabled: true
min_flake_score: 0.6
min_runs: 3
quarantine:
snapshots:
- "search_flow__after_search"
review_gate:
enabled: true
min_approvals: 1
required_reviewers: ["qa-owner"] # optional
require_all_changed_rows_reviewed: true
owners:
by_snapshot:
home: "web-platform-team"
"desktop_1440x900:pricing": "checkout-team"
ignore_regions:
global:
selectors: ["#cookie-banner", ".chat-widget"]
by_snapshot:
home:
rects:
- {x: 0, y: 0, width: 300, height: 120}
# Optional Figma sync (writes into visual_baseline/<project>/<suite>/figma/...)
# figma:
# token_env: FIGMA_TOKEN
# file_key: "<FIGMA_FILE_KEY>"
# frames:
# - id: home
# node_id: "123:456"
# view_id: "desktop_1440x900" # optional
Commands
Run full check
visualcheck run --env prod
# force-refresh matched figma baselines during this run:
visualcheck run --env prod --force-reset-figma-baseline
# skip browser auto-open:
visualcheck run --env prod --no-open-report
Programmatic auto-open from .env (non-CLI)
VISUALCHECK_OPEN_REPORT=true
- When enabled,
run_visualcheck(...)andVisualCheck.finalize()automatically openreport.htmlin your default browser. - CLI already has its own
--open-report/--no-open-reportoption, so env auto-open is ignored there to avoid double-opening tabs.
Outputs:
- Baselines:
visual_baseline/<project>/<suite>/figma/<view>/<snapshot>.pngvisual_baseline/<project>/<suite>/runtime/<view>/<snapshot>.png
- Run artifacts:
test_report/<project>/dashboard.html(main dashboard; latest entry point)test_report/<project>/visual_runs/<run_id>/current/...test_report/<project>/visual_runs/<run_id>/diff/...test_report/<project>/visual_runs/<run_id>/report/report.htmltest_report/<project>/visual_runs/<run_id>/report/report.jsontest_report/<project>/visual_runs/<run_id>/report/report.junit.xml
Dashboard behavior:
- Keeps full run history from
visual_runs/so older reports remain visible. - Groups runs by YAML config filename (for example
visualcheck.yaml) for easier tracking by non-technical users. - Uses stronger status colors in both dashboard and report for quicker human scanning.
- Includes dashboard filters/search/sorting (groups and runs) and report filters/search/sorting (status, text, severity, diff).
Terminal logs now include user-friendly lifecycle messages, for example:
run started ...capture started/completed ...full-load wait started ...and eitherfull-load ready ...orfull-load timeout ...with exact reason (readyState,pendingImages,pendingLazyImages) and timingno baseline found, creating runtime baseline from current screenshotusing runtime baseline .../using cached figma baseline ...- per-item final status with pixel diff and SSIM
- final report + dashboard paths
Capture only
visualcheck capture --env prod --out current
Diff only
visualcheck diff --baseline visual_baseline/myproj/mysuite/runtime --current current --out report
Sync Figma baselines
export FIGMA_TOKEN="..."
visualcheck figma-sync
# force refresh selected frames:
visualcheck figma-sync --force --only home,pricing
# auto-map page/flow snapshot ids to figma frame names:
visualcheck figma-sync --match-by-name
Compare strictly against Figma baseline
visualcheck figma-diff --env prod
Approve current run as runtime baseline (explicit)
visualcheck approve --env prod --run-id 20260207_235500
# Overwrite existing runtime baselines only with:
visualcheck approve --env prod --run-id 20260207_235500 --force
Validate config
visualcheck validate-config --config visualcheck.yaml
Cleanup old run artifacts
visualcheck cleanup --config visualcheck.yaml --keep-last 20 --dry-run
visualcheck cleanup --config visualcheck.yaml --keep-last 20
Quarantine management
visualcheck quarantine-list --config visualcheck.yaml
visualcheck quarantine-add --config visualcheck.yaml --snapshot search_flow__after_search
visualcheck quarantine-remove --config visualcheck.yaml --snapshot search_flow__after_search
Review workflow (UI-only)
- Approval is handled directly from report/dashboard UI (Approve/Reject/Needs Review buttons).
- Set reviewer name in UI once, then approve/reject rows from the same screen.
- Baseline promotion is blocked while review gate is
BLOCKED.
Baseline rules (locked)
For each snapshot + view:
- If a Figma baseline exists → use it
- visualcheck will attempt a best-effort
figma-syncwhenfigmais listed inbaseline.priorityso remote Figma baselines are consulted before falling back.
- visualcheck will attempt a best-effort
- Else if a runtime baseline exists → use it
- Else → capture and create runtime baseline (controlled by
baseline.create_if_missing)
Notes on configuration and behavior
baseline.prioritycontrols the preference order. Example:priority: ["figma","runtime"](default in examples) — visualcheck will try Figma first.- By default a Figma baseline is used directly for comparison (reported baseline path will point at
.../figma/...). - By default, Figma does not auto-create runtime baseline copies. Runtime remains website-captured truth (created from current captures when missing).
- If you want legacy behavior (copy Figma into runtime), set:
baseline:
seed_runtime_from_figma: true
- If you want Figma to become the authoritative runtime baseline (copied into the
runtime/folder and then used), set:
baseline:
figma_authoritative: true
- This avoids accidental comparisons against an old runtime image when Figma is the source of truth.
- If a figma baseline is missing locally and
figmaappears in priority, visualcheck will attempt to fetch it via the Figma API (best-effort). If the fetch fails (token/permission), it falls back to runtime perpriority. - If a matching local figma baseline already exists for snapshot+view, visualcheck does not call the Figma API again.
- Optional forced refresh from config:
baseline:
force_reset_figma_baseline: true
This is equivalent to CLI flag --force-reset-figma-baseline for a run.
Figma metadata in report
- When a snapshot resolves to a Figma frame, report rows include
figma.file_key,figma.node_id, and a directfigma.frame_url.
Report upgrades
- HTML report includes status summary cards, client-side status filters, top regressions, and an expanded split layout for faster triage.
- Each result row now provides a larger visual workspace, hover zoom cue, and one-click popup viewer with pan/zoom controls.
- Left-side mismatch region items are clickable and auto-focus the same coordinates on the diff/compare canvas.
- Auto-detected diff hotspots are listed as separate clickable lines with per-hotspot
diff%andssim. - Mismatch clicks and image popups now prioritize the
Difflayer so users see exact issue regions first. - View mode switching is built in (
baseline/current compare,diff,current,baseline) so users can inspect the exact failure context quickly. - Slider canvas opens in full-image fit by default; users can then zoom and scroll in the same canvas.
- Report now includes a Recent Run History table (status, commit, branch, counts).
- Main dashboard (
test_report/<project>/dashboard.html) lists latest runs and links to individual reports. - Rows include plain-language issue summary and recommended action to make pixel triage faster.
- Report includes flake score, root-cause hint, snapshot owner, quarantine status, and likely-introduced-after commit hint.
- History files are stored under
test_report/<project>/history/for trends, flake scoring, and change correlation. - Root-cause tags are shown per row for faster triage (
SIZE_MISMATCH,FLAKY,REGION_FAIL, etc). - A11y summary is shown per row (
CURRENT_ONLYorCOMPARED) with delta metrics. - Manual review state/comment is displayed directly from UI actions.
- Cross-browser matrix runs are supported by
capture.browser_channels.
Use from code (framework integration)
If you already navigate with Playwright/Selenium in your own framework and just want visualcheck to capture + baseline + diff + report, use the code API.
Baselines still go to
visual_baseline/<project>/<suite>/...and the HTML report goes totest_report/<project>/visual_runs/<run_id>/report/report.html.
Playwright (sync) example
from playwright.sync_api import sync_playwright
# NOTE: API object names may evolve; refer to the package docs in case of changes.
from visualcheck.api import VisualCheck
vc = VisualCheck(
project="my_project",
suite="daily_sanity",
env="prod",
base_url="https://example.com",
view_id="desktop_1440x900",
baseline_priority=["figma", "runtime"],
create_if_missing=True,
ignore_selectors=["#cookie-banner", ".chat-widget"],
)
with sync_playwright() as p:
browser = p.chromium.launch(headless=True)
page = browser.new_page(viewport={"width": 1440, "height": 900})
page.goto("https://example.com/", wait_until="domcontentloaded")
vc.check(snapshot_id="home", page=page)
page.goto("https://example.com/pricing", wait_until="domcontentloaded")
vc.check(snapshot_id="pricing", page=page)
vc.finalize()
print("Report:", vc.report_html)
browser.close()
Selenium example
from selenium import webdriver
from visualcheck.api import VisualCheck
vc = VisualCheck(
project="my_project",
suite="daily_sanity",
env="prod",
base_url="https://example.com",
view_id="desktop_1440x900",
)
driver = webdriver.Chrome()
driver.set_window_size(1440, 900)
driver.get("https://example.com/")
vc.check(snapshot_id="home", selenium_driver=driver)
driver.get("https://example.com/pricing")
vc.check(snapshot_id="pricing", selenium_driver=driver)
vc.finalize()
print("Report:", vc.report_html)
driver.quit()
Notes
- Ignore regions are masked with a solid color before diffing.
- If screenshot sizes differ and
compare.resize_on_mismatch=true, current is resized to baseline size and a warning is recorded.
License
MIT
Library & CLI usage (examples)
Programmatic API (Python)
The package exposes a simple programmatic API to integrate with Playwright or Selenium.
Example (Playwright):
from visualcheck.runner import run_visualcheck
from visualcheck.config import load_config
cfg = load_config(Path('examples/wakefit.yaml'))
result = run_visualcheck(cfg, env='prod', headed=False)
print('Report saved at', result.get('report_html'))
CLI usage
- Sync Figma baselines:
export FIGMA_TOKEN="<your_token>"
visualcheck figma-sync --config examples/wakefit.yaml
- Run full check:
visualcheck run --env prod --config examples/wakefit.yaml
Notes on FIGMA token & env handling
- The library reads Figma token from the environment variable name specified in config (defaults to
FIGMA_TOKEN). - Keep secrets out of git; use host-level secrets (e.g.
~/.openclaw/secrets.env) or a CI secret store.
Metadata
Release files for visualcheck 0.3.26
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| visualcheck-0.3.26.tar.gz | 62.6 kB | Without provenance | Details for visualcheck-0.3.26.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| visualcheck-0.3.26-py3-none-any.whl | Python 3 | none | any | Details for visualcheck-0.3.26-py3-none-any.whl |
Total release size: 122.5 kB
Release files / visualcheck-0.3.26.tar.gz
| Download URL | visualcheck-0.3.26.tar.gz |
|---|---|
| Size | 62.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
72005288b4775135d45161a95b84e0d40498de27cb95111bc7274b45909f4763
|
|
BLAKE2b-256 checksum How to use checksums |
301e70818a0a632ada9228275ca65748329b18a300ad231ae82f613040b542d7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.4
|
Release files / visualcheck-0.3.26-py3-none-any.whl
| Download URL | visualcheck-0.3.26-py3-none-any.whl |
|---|---|
| Size | 59.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
9df2ea586fef5a1d45bef6d45d040951a85544341276f6394a09612e7ffd92e6
|
|
BLAKE2b-256 checksum How to use checksums |
fa90b1b6de9f226f7b92e54408444598bd5697b37eb031bf59354afc73a46918
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.4
|
Security for visualcheck 0.3.26
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.