pytest-tags 0.8.2

Overview

Framework for building pytest plugins which works on tag- (i.e. marker-) based groupings of tests.

Features

• Easy creation of plugin objects, each associated with a @pytest.mark.<...> marker used for tagging test functions, classes, and modules
• Intuitive and automatic scoping of both explicitly-labeled and anonymous tags
• Automatic grouping of tagged parametrized tests

Changelog

Refer to the Release notes.

Example

Consider the @pytest.mark.fail_fast_group marker provided by the package, which groups tests together and skips subsequent tests in a group if any earlier test in the same group failed:

from __future__ import annotations

from collections.abc import Generator
from typing import Literal

import pytest


@pytest.fixture
def good_fixture() -> int:
    return 1


@pytest.fixture
def bad_fixture() -> Generator[int, None, None]:
    raise RuntimeError('Bad fixture')
    yield 1


@pytest.mark.fail_fast_group(groupby='fixture')
@pytest.mark.parametrize('x, y', [(1, 2), (3, 4), (5, 6)])
@pytest.mark.parametrize('fixture', ['good_fixture', 'bad_fixture'])
def test_use_fixture(
    request: pytest.FixtureRequest,
    fixture: Literal['good_fixture', 'bad_fixture'],
    x: int,
    y: int,
) -> None:
    """
    This parametrized test uses a bad fixture in some subtests; with the
    `@pytest.mark.fail_fast_group` marker, all subtests using
    `good_fixture` are tagged and grouped together, while all subtests
    using `bad_fixture` end up in another group. As a result,
    only one failing test will be run in the latter group, while
    subsequent tests in the group will be skipped, citing the first
    failing test as the reason.
    """
    z = request.getfixturevalue(fixture)
    ...


@pytest.mark.fail_fast_group
def test_a_failing_test() -> None:
    """
    This test fails, causing later tests tagged with the anonymous
    `@pytest.mark.fail_fast_group` to be skipped.
    """
    raise RuntimeError('fail')


@pytest.mark.fail_fast_group
def test_skipped() -> None:
    """
    This test is skipped, because `test_a_failing_test()` is in
    the same anonymous group as it and has failed.
    """


@pytest.mark.fail_fast_group(scope='session')
def test_another_failing_test() -> None:
    """
    This test is not skipped, because while its tag is anonymous like
    `@pytest.mark.fail_fast_group` it is explicitly scoped to the entire
    session, while the implicit version is scoped to the module.
    """
    raise RuntimeError('failing again')


@pytest.mark.fail_fast_group(0)
def test_passing_test() -> None:
    """
    This test is not skipped, because its tag has an explicit label `1`;
    and is implicitly scoped to the module; thus, it does not fall in
    the same group as either `test_a_failing_test()` or
    `test_another_failing_test()`.
    """

 $ pytest -rs --tb=no -v test_module.py
============================= test sesstion starts =============================
...
test_module.py::test_use_fixture[good_fixture-1-2] PASSED                 [ 10%]
test_module.py::test_use_fixture[good_fixture-3-4] PASSED                 [ 20%]
test_module.py::test_use_fixture[good_fixture-5-6] PASSED                 [ 30%]
test_module.py::test_use_fixture[bad_fixture-1-2] FAILED                  [ 40%]
test_module.py::test_use_fixture[bad_fixture-3-4] SKIPPED (1 prior te...) [ 50%]
test_module.py::test_use_fixture[bad_fixture-5-6] SKIPPED (1 prior te...) [ 60%]
test_module.py::test_a_failing_test FAILED                                [ 70%]
test_module.py::test_skipped SKIPPED (1 prior test in the same group(...) [ 80%]
test_module.py::test_another_failing_test FAILED                          [ 90%]
test_module.py::test_passing_test PASSED                                  [100%]

=========================== short test summary info ============================
SKIPPED [2] test_module.py:20: 1 prior test in the same group(s) already failed 
({test: matched_tags}): {'test_module.py::test_use_fixture[bad_fixture-1-2]': [<
Tag "test_use_fixture(fixture='bad_fixture')">]}
SKIPPED [1] test_module.py:51: 1 prior test in the same group(s) already failed 
({test: matched_tags}): {'test_module.py::test_a_failing_test': [<Tag 'test_mod
ule.py'>]}
==================== 3 failed, 4 passed, 3 skipped in 0.08s ====================

The pytest plugin implementing @pytest.mark.fail_fast_group is built upon the rest of the package, which provides the facilities for doing such tag-/marker-based grouping.

Full documentation

Refer to the project repository: GitLab:TTsangSC/pytest-tags