Pytest Plugin for Visual Testing with Playwright

As of 2025-03-22 of all of the existing packages to do simple visual regression testing using playwright are long dead. I had a bunch of updates I wanted to make to existing systems so I rewrote the plugin with a bunch of updates:

• snapshots are always written on CI to easily download them as artifacts
• ability to mask out certain elements which cause comparison brittleness
• failing on --update-snapshots to make users manually review images
• snapshot name is optional, test_name[browser][os].png is auto-generated by default
• multiple snapshots in a single test, file names are auto incremented
• updated folder structure: snapshots/file_name/test_name/test_name[browser][os].png
• ability to configure directories, etc via ini + pytest config.

You can see this implemented in a working project here.

Installation

uv add --dev pytest-playwright-visual-snapshot

Usage

This plugin provides a assert_snapshot fixture which is used to create snapshots and compare it.

def test_myapp(page, assert_snapshot: AssertSnapshot):
    page.goto("https://example.com")
    assert_snapshot(page)

Then, run pytest:

pytest

The first time you run pytest, snapshots will be created, and you will get the error:

New snapshot(s) created. Please review images

The next run, the snapshots comparison will take place. To update snapshots, run:

pytest --update-snapshots

After updating, tests will fail and you will need to review images.

In case of a mismatch, snapshot_tests_failures folder will be created with actual_.., expected_.. and diff_.. images generated.

Configuration

View all configuration options by running pytest --help. Here's a quick example:

# NOTE this runs on any pytest invocation, even if no tests are run
def pytest_configure(config: Config):
  config.option.playwright_visual_snapshots_path = Path("...")
  config.option.playwright_visual_snapshot_failures_path = Path("...")

Masking Elements

You can mask certain elements during screenshot capture to prevent them from causing comparison failures. This is useful for dynamic content like timestamps, user avatars, etc.

Configure global masks in your pytest.ini:

[pytest]
playwright_visual_snapshot_masks =
    [data-clerk-component="UserButton"]
    .timestamp
    #dynamic-content

Or directly via pytest_configure:

def pytest_configure(config: Config):
    config.option.playwright_visual_snapshot_masks = [
        '[data-clerk-component="UserButton"]',
        '.timestamp',
        '#dynamic-content'
    ]

Or specify masks directly in your test:

def test_with_custom_masks(page, assert_snapshot: AssertSnapshot):
    page.goto("https://example.com")
    assert_snapshot(page, mask_elements=[".user-avatar", "#timestamp"])

Handling Size Differences

By default, if a snapshot's dimensions change (width or height), pytest will raise an exception without generating visual comparison images. This can happen when taking snapshots of individual elements whose size changes due to visual modifications - for example, a button with updated padding or a container that dynamically adjusts its height.

To see visual diffs even when dimensions change, use the --ignore-size-diff flag:

pytest --ignore-size-diff

Or configure it globally in your pytest.ini:

[pytest]
playwright_visual_ignore_size_diff = true

Or via pytest_configure:

def pytest_configure(config: Config):
    config.option.playwright_visual_ignore_size_diff = True

When enabled, snapshots with different dimensions will still generate the actual_, expected_, and diff_ images in the failures folder, allowing you to see exactly what changed visually.

Example use-case: You're testing a UI component and update the button padding. Without --ignore-size-diff, you'd just get an exception. With it enabled, you get visual diff images showing the padding changes, making it easy to review and decide if the changes are intentional.

Disabling Visual Snapshots Locally

If CI screenshots are the source of truth, you can disable local visual assertions to keep developer runs fast and avoid creating/comparing snapshots; use pytest --disable-visual-snapshots (or set playwright_visual_disable_snapshots = true in pytest.ini). When disabled, assert_snapshot is a noop and logs a warning.

GitHub Actions Script

The CI Chrome will be slightly different than your dev chrome. You'll want to pull down screenshots from your CI run and use those for comparison. Here's a script to do that:

failed_run_id=$(gh run list --status=failure --workflow=workflow_name.yml --json databaseId --jq '.[0].databaseId')
PLAYWRIGHT_RESULT_DIRECTORY=

rm -rf ${PLAYWRIGHT_RESULT_DIRECTORY}/${failed_run_id} && \
mkdir -p ${PLAYWRIGHT_RESULT_DIRECTORY}/${failed_run_id} && \
gh run --dir ${PLAYWRIGHT_RESULT_DIRECTORY}/${failed_run_id} download $failed_run_id && \
cp -R ${PLAYWRIGHT_RESULT_DIRECTORY}/${failed_run_id}/test-results/${PLAYWRIGHT_VISUAL_SNAPSHOT_DIRECTORY}/ ${PLAYWRIGHT_VISUAL_SNAPSHOT_DIRECTORY}/

API

Fixture Parameters

• threshold - sets the threshold for the comparison of the screenshots:0 to 1. Default is 0.1

• fail_fast - If True, will fail after first different pixel. False by default
• mask_elements - List of CSS selectors to mask during screenshot capture. These will be combined with any globally configured masks.

Command Line Options

• --update-snapshots - Update existing snapshots with new screenshots
• --ignore-size-diff - Generate visual diffs even when snapshot dimensions differ (instead of raising an exception)
• --disable-visual-snapshots - Disable visual snapshot assertions

Alternatives

• https://github.com/kumaraditya303/pytest-playwright-snapshot - long dead
• https://github.com/symon-storozhenko/pytest-playwright-visual - fork of the above repo, long dead
• https://github.com/Visual-Regression-Tracker/Visual-Regression-Tracker - requires a separate server to run

This project was created from iloveitaly/python-package-template