pytest-multilog 1.6.1

Multi-process logs handling and other helpers for pytest

pytest-multilog

A pytest plugin to persist logs from parallel test processes (and other helpers)

Usage

To use the multilog feature, a test class just needs to inherit from the TestHelper class:

from pytest_multilog import TestHelper

class TestCustom(TestHelper):
    def test_custom(self):
        # Custom test implementation
        ...

The TestHelper class declares a logs fixture which will be automatically used by its children classes.

Behavior and attributes

Root folder

The TestHelper class provides a root_folder property, matching with the pytest root folder.

Output folder

The TestHelper class provides a output_folder property, where all files will be written. It's set to output_folder / "out" / "tests"

Test name

Each test is associated to a name (provided in TestHelper.test_name), computed from the file name, the class name and the method name.

E.g. for the snippet above, if stored in a test_custom.py file, the test name will be: custom/Custom/custom.

Test name compute behavior can be configured with following options:

• test file part is computed relatively to the root test module. Test module name is configured through the TestHelper.test_module property (default value is "tests"). Overriding the property value allow to change the computing behavior:
  • if tests are relative to another test module, just change the name returned by this TestHelper.test_module property
  • if the TestHelper.test_module property is set to None or an empty string, file part will be simply removed from the test name. In the above example, test name would become: Custom/custom
  • if the name provided by TestHelper.test_module property is not found in the current test file path, if the test file is located in a submodule, it will be simply ignored in the file part of the test name (i.e. only the test file name will be used, not its relative path)
• By default, "test_" and "Test" prefixes are removed from the different parts of the test name, in order to shorten it while keeping it unique (since all pytest detected tests files, class and methods must use these prefixes). In order to disable this replacement (i.e. restore behavior of versions < 1.5.0), just set the TestHelper.filter_test_prefixes property to False.

Current worker

In multi-process environment (pytest was invoked with -n X argument), the current worker name is provided in TestHelper.worker attribute. It's set to "master" in single-process environment.

The class also provides a TestHelper.worker_index attribute, giving the working index as an integer value (will be set to 0 for "master").

Test folder

Test logs will be written in a pytest.log file (path provided in TestHelper.test_logs), stored in each test folder (provided in TestHelper.test_folder attribute):

• While the test is running, it's set to TestHelper.output_root / "__running__" / TestHelper.worker / TestHelper.test_name.replace("/","_")
• Once the test is terminated, the folder is moved directly under the output root one (i.e.: TestHelper.output_root / TestHelper.test_name); this final folder value is stored in the TestHelper.test_final_folder property.

It means that during the test execution, it's possible to check which test is running on which worker (easing troubleshooting situations where a given worker is blocked)

Checking logs

It is possible to verify if some strings are present in the current test log, by using the TestHelper.check_logs method. It takes as input argument:

• either a simple string/Pattern or a list of strings/Patterns:
  • strings will be simply checked to be contained in the whole log
  • Patterns will be searched line by line (more flexible, but slower)
• an optional timeout
• a check_order parameter to verify patterns order (default if False)

The method will assert if all inputs are found in the log (within the expected timeout, if any).

By default, expected patterns order doesn't matter. If check_order parameter is set to True, patterns will be expected in the input order, and check will fail if the order doesn't match.