pytest-visionspec

Pytest plugin that auto-reports test results with screenshots to VisionSpec.

Install

pip install pytest-visionspec

Setup

Set two environment variables:

VS_API_KEY=vs_your_project_api_key
VS_API_URL=https://visionspec-dev.helpfulhuman.xyz

Usage

Tag tests with @pytest.mark.vs("journey-id"):

@pytest.mark.vs("login-happy-path")
def test_login(page):
    page.goto("/login")
    page.fill("[name=email]", "user@example.com")
    page.click("button[type=submit]")
    assert page.url == "/dashboard"

The plugin automatically:

1. Captures a Playwright screenshot after each test
2. Uploads it to VisionSpec storage
3. Posts the result (pass/fail + screenshot URL) to VisionSpec

Markers

• @pytest.mark.vs("journey-id") — link test to a VisionSpec V1 journey spec
• @pytest.mark.vs_surface("mobile") — override the reported surface (default: desktop)
• @pytest.mark.vs_v2("spec-slug", jtbd="jtbd-slug") — link test to a VisionSpec V2 spec (chapters model). Results are buffered and POSTed in one batch at session end to /api/v2/results.

V2 usage (chapters model)

For V2 projects, tag tests with vs_v2 — no screenshot capture, batched reporting:

@pytest.mark.vs_v2("verdict-returns-200", jtbd="get-verdict")
def test_verdict():
    r = client.get("/verdict")
    assert r.status_code == 200

Get your project's VS_API_KEY via the V2 MCP tool get_project(project) (returns test_reporting.env_vars) or from the "CI / Test runner" tab in V2 project settings.

Results POST once per pytest session. Coverage in V2 get_doc's summary field then shows tested / passing / failing counts per spec.

Git SHA auto-detection (V2)

Each vs_v2 result is tagged with the SUT's git SHA so the vs2 dashboard can group results by build ("which SHA is safe to ship?").

Detection is automatic and deterministic:

1. git rev-parse HEAD — works locally and in any CI that checks out the .git directory (GitHub Actions does this by default).
2. GITHUB_SHA — GH Actions sets this natively.
3. Omitted — if neither works, results are recorded without a revision tag. The dashboard's build-scoped views will not include them.

No user-configurable env var (like VS_GIT_SHA) is supported — the git SHA is a single source of truth (the SUT itself). Setting it via env var risks tagging results with the wrong SHA.

Page fixture detection

By default, the plugin looks for a page fixture. Configure custom fixture names in pyproject.toml:

[tool.pytest.ini_options]
vs_page_fixtures = ["v2app", "v1app", "page"]

The plugin checks each name in order and uses the first one it finds.

Environment variables