pytest-jira-xfail 1.4.1

Plugin skips (xfail) tests if unresolved Jira issue(s) linked

pytest-jira-xfail

Plugin skips (xfail) tests linked to unresolved Jira issue(s)

1. Generate your Jira API token

You should have Jira user with API token generated

2. Add PytestJiraHelper to your pytest hook:

import pytest

from pytest_jira_xfail.jira_helper import PytestJiraHelper


@pytest.hookimpl(tryfirst=True)
def pytest_collection_modifyitems(items):
    jira = PytestJiraHelper(
        jira_url="https://company.atlassian.net",
        jira_username="[e-mail]",
        jira_api_token="my_jira_user_api_token",
    )
    jira.process_linked_jira_issues(items)

3. Link bugs to your tests

from pytest_jira_xfail.annotations import bug


@bug("MP-123")
def test_my_test_fails():
    assert False


@bug("MP-124", IndexError)
def test_my_test_broken():
    db_records = []
    assert db_records[0]


@bug("MP-124")
@bug("MP-124", IndexError)
def test_multiple_exceptions():
    db_records = []
    assert db_records[0][0] == 'active'

Skip the test entirely (never run it)

By default a test linked to an open bug is still executed and reported as XFAIL (so an unexpected pass shows up as XPASS). If the bug makes the test unsafe or pointless to run (e.g. it hangs, corrupts data, or blocks the suite), pass run=False to skip execution completely while the issue is open. The test is reported as XFAIL [NOTRUN] and its body is never executed:

from pytest_jira_xfail.annotations import bug


@bug("MP-123", run=False)
def test_not_executed_until_fixed():
    assert False  # never runs while MP-123 is open

Once the linked issue is resolved, the test runs normally again.

Match the error message, not only the error type

By default a test is treated as XFAIL whenever it raises the expected error type (raises). If the same error type can be raised for unrelated reasons, use error_contains to xfail the test only when the raised error message contains an expected substring (or one of several substrings). If the type matches but the message does not, the test is reported as a real failure so a different problem is not silently hidden:

from pytest_jira_xfail.annotations import bug


# XFAIL only if an IndexError with this exact message is raised
@bug("MP-123", IndexError, error_contains="list index out of range")
def test_single_substring():
    records = []
    assert records[0]


# XFAIL if a KeyError message contains any of the listed substrings
@bug("MP-124", KeyError, error_contains=["'user_id'", "'account_id'"])
def test_multiple_substrings():
    payload = {}
    assert payload["user_id"]

Matching is case-sensitive by default. Pass case_sensitive=False for case-insensitive matching:

# Case-sensitive (default): the case must match exactly
@bug("MP-125", ValueError, error_contains="Invalid token")
def test_case_sensitive():
    raise ValueError("invalid token")  # does NOT match -> reported as a failure


# Case-insensitive: matches regardless of case
@bug("MP-126", ValueError, error_contains="invalid token", case_sensitive=False)
def test_case_insensitive():
    raise ValueError("INVALID TOKEN")  # matches -> XFAIL

error_contains can be combined with run=False and with multiple @bug markers (each marker keeps its own expected type, substrings and case option).

Soft assertions (pytest-check)

Tests that fail via soft-assertion libraries such as pytest-check (check.equal(...), @check.check_func, ...) are handled correctly: a soft-assertion failure on a test with an open issue is reported as XFAIL deterministically, regardless of environment (IDE vs CI) or pytest-xdist. The expected exception type from raises is still enforced for hard (call-phase) exceptions.

XFAIL message format:

XFAIL The test is skipped because of open bugs:
https://company.atlassian.net/browse/MP-123

4. [Optional] Set custom resolved statuses

By default, only issues with the status "Done" and "Closed" are considered as resolved.
But you can override this and add more statuses, as following:

import pytest

from pytest_jira_xfail.jira_helper import PytestJiraHelper


@pytest.hookimpl(tryfirst=True)
def pytest_collection_modifyitems(items):
    jira = PytestJiraHelper(
        jira_url="https://company.atlassian.net",
        jira_username="[e-mail]",
        jira_api_token="my_jira_user_api_token",
        resolved_statuses=["Done", "Closed", "Released", "Declined"]
    )
    jira.process_linked_jira_issues(items)