pytest-human 0.5.0

A beautiful nested HTML test report

pytest-human

A pytest plugin for generating beautiful, human-readable HTML reports for individual tests with collapsible nested logging spans and syntax highlighting. Inspired by Robot Framework and Playwright reports.

Unlike other pytest HTML report plugins, pytest-human creates a separate HTML log file for each test, aimed at helping you dive into specific parts of the test that are relevant for debugging. Works with standard python logging, no need to rewrite existing tests to get going!

Features

• Logs
  • Beautiful test logs
  • Collapsible spans
  • Syntax highlighting
  • Colored log levels
  • Support for existing native python logging
  • Streaming log writing
• Tracing
  • Automatic fixture logging
  • Automatic method call logging
  • Third-party method tracing
• Debugging
  • Deep error highlighting
  • Regex search in collapsed spans
  • Artifacts collection

Demo

https://github.com/user-attachments/assets/831b53ea-cf7f-468d-9303-335e444525e9

Example Report and test source file.

Installation

Install from PyPI:

pip install pytest-human

Quick Start

Basic Usage

1. Enable the plugin when running pytest:

   pytest --enable-html-log --log-level DEBUG --html-output-dir output/
   

   Setting the log level is important as the pytest default is high (WARNING).

2. Use the human object and the @traced decorator in your tests:

   from pytest_human.log import traced
   
   @traced()
   def insert_db(data):
       query = "INSERT INTO flowers (petals) VALUES ('{{1,2,3,4,5}}');"
       logging.info(f"executing {query=}")
       return len(data)
   
   def test_example(human):
       """This test demonstrates pytest-human logging."""
       human.log.info("Established test agent connection")
   
       with human.log.span.info("Generating sample data"):
           data = [1, 2, 3, 4, 5]
           human.log.info(f"Loaded sample data {data=} {len(data)=}", highlight=True)
           insert_db(data)
   
           with human.log.span.debug("Validating sample"):
               result = sum(data)
               human.log.debug(f"Sum {result=}", highlight=True)
   
       assert result == 15
   

3. Open the HTML test log

   At the end of individual tests you will see a similar line:

   🌎 Test test_single_stage_ui HTML log at file:///tmp/pytest-of-john.doe/pytest-2/session_logs/test_frobulator.html
   

   You can Ctrl/⌘-Click the link in most terminals to open the file.

4. Debug!

Command Line Options

Enable HTML Logging

# To enable HTML logging support and use this plugin, pass this flag

pytest --enable-html-log

Log Location

Control where HTML logs are saved:

# Save all test logs in a custom directory specified by the user.
pytest --html-output-dir /path/to/logs

# By default logs are saved in the session temp directory with the test name
# e.g. /tmp/pytest-of-user.name/pytest-446/session_logs/test_method_tracing.html
pytest --enable-html-log 

# Save in individual test temporary directories as test.html
# e.g. /tmp/pytest-of-user.name/pytest-446/session_logs/test_examplecurrent/test.html
pytest --enable-html-log --html-use-test-tmp

Log Level

Control the minimum log level:

# Use pytest's global log level.
# Opt to use this setting.
pytest --enable-html-log --log-level DEBUG

# Set log level for HTML logs specifically.
# This requires the root logger to be properly configured.
pytest --enable-html-log --html-log-level INFO

--html-log-level controls the html logger directly but might still be restricted by the python root level. The root logger level can be tweaked programatically or using the --log-level parameter. --html-log-level default to DEBUG while in pytest the root log level defaults to WARNING.

For human features to work, ensure the root logger is set to DEBUG or TRACE using --log-level.

Logger API

Fixtures

• human - Supplies a human object to the test. Includes a logger and attachment collector.
• test_log - Supplies a logger to the test, equivalent to human.log.
• human_test_log_path - Path of the html log file for the current test

Logging Methods

def test_logging_methods(human):
    # Basic logging at different levels
    human.log.trace("Trace level message")
    human.log.debug("Debug level message")
    human.log.info("Info level message")
    human.log.warning("Warning level message")
    human.log.error("Error level message")
    human.log.critical("Critical level message")
    
    # Syntax highlighting for code
    code = """
    import numpy as np

    def bark(volume: float) -> bool:
        return volume > 0.5:
    """
    human.log.info(code, highlight=True)

Direct logger access

Get the test logger programmatically, this allows to tweak the source name and is useful if you don't want to pass the human object around.

from pytest_human.log import get_logger

def test_programmatic_logger():
    logger = get_logger(__name__)
    logger.info("Custom logger")

Collapsible Spans

Create nested, collapsible sections in your HTML logs.

This allows partitioning the log into sections and diving only into the parts of the logs that are relevant to your debug session.

def test_spans(human):
    human.log.info("Starting complex operation")
    
    with human.log.span.info("Phase 1: Initialization"):
        human.log.debug("Initializing resources...")
        
        with human.log.span.debug("Loading configuration"):
            human.log.trace("Reading config file")
            config = load_config()
            human.log.debug(f"Config loaded: {config}")
        
        human.log.info("Initialization complete")
    
    with human.log.span.info("Phase 2: Processing"):
        human.log.debug("Processing data...")
        process_data()
    
    human.log.info("Operation completed")

Method Tracing

A lot of debug logging is centered around logging a function when called, returned a value or threw an error. Human supplies the @traced decorator to allow for automatic method tracing in log.

Each method call shows the function call parameters, opens a new span that includes all of the logging that happened in its scope, as well as its return value. This allows for easy segmentation of nested loggings and reduces the amount of noise encountered when debugging.

import logging
import time
from pytest_human.log import traced, get_logger

# Add the traced() decorator for automatic logging of method call/return
@traced()
def save_login(login):
    log = get_logger(__name__)
    log.info("a log inside save_login")
    return update_db(login)

@traced(log_level=logging.TRACE)
def update_db(login):
    log = get_logger(__name__)
    delay_time = 2
    log.info("delaying db update by 2 seconds")    
    time.sleep(delay_time)   
    return delay_time

def test_method_tracing(human):
    delay = save_login("hello")
    assert delay == 2

@traced supports the following parameters:

• suppress_return - do not log the return value, useful when it is overly long
• suppress_params - do not log the method parameters, useful when they are overly long
• suppress_self - do not show the self argument logging function parameters, default True
• log_level - set a log level for the method trace

Note: tracing will have some performance implications on method calls, also you should limit using @traced on frequently called functions as to reduce log noise.

Third-party method tracing

The @traced decorator is very useful for debugging but it is unfortunately restricted to code you own.

In order to log third-party methods, you can use the trace_calls and trace_public_api methods, which monkey patch third party code with human tracing.

trace_calls adds logging to a list of functions, while trace_public_api adds logging to all public methods of modules/classes.

import pytest
from playwright.sync_api import Locator, LocatorAssertionsImpl, Page
from pytest_human.log import trace_calls, trace_public_api

@pytest.fixture(autouse=True)
def log_3rdparty_methods():
    with (
        trace_calls(
            pytest.Pytester.runpytest,
            pytest.Pytester.makepyfile,
        ),
        trace_calls(Page.screenshot, suppress_return=True),
        # this skips retracing Page.screenshot as it is already defined above
        trace_public_api(Page, Locator, LocatorAssertionsImpl),
    ):
        yield

Artifacts

Sometimes you might have extra logs that are generated by subprocesses or used external resources such as Kubernetes, Docker or others. Human automatically collects stdout/stderr and allows you to collect custom logs to be attached to the log.

def test_artifacts(human):
    human.log.info("Attaching artifacts to the test log")

    print("logging something to stdout")

    log_content = """
    [10:00:01] First line of the log.
    [10:00:03] Line 2 of the log.
    [10:00:05] Line 3 of the log.
    """
    human.artifacts.add_log_text(log_content, "sample.log", description="Sample log file")

Logging

Using Standard Python Logging

pytest-human integrates with Python's standard logging system. All logs from any logger will be captured:

import logging

def test_standard_logging(human):
    # pytest-human logger
    human.log.info("Using human fixture")
    
    # Standard Python logger - also captured in HTML
    logger = logging.getLogger(__name__)
    logger.info("Using standard logger")

TRACE Logging

pytest-human adds a custom TRACE log level below DEBUG for more verbose logging:

def test_trace_logging(human):
    human.log.trace("Very detailed trace information")

Run with trace level:

pytest --enable-html-log --log-level trace

Keyboard navigation

You can use the keyboard to navigate around the log.

• Press Tab and Shift+Tab to jump between the expand buttons (+).
• Press Enter to expand and collapse a span when hovering over a button.
• Press / to jump to the search box and Esc to unfocus.
• When searching use Enter to jump to the next result and Shift+Enter to jump to the previous result.

Development

Running Tests

# Install development dependencies
pip install --group dev -e .

# Run tests
pytest

# Run with coverage
pytest --cov=pytest_human

Alternatively use tox

tox

Building Documentation

mkdocs serve

Documentation is still TBD

License

Distributed under the Apache Software License 2.0. See LICENSE for more information.

Links

• PyPI: https://pypi.org/project/pytest-human
• Repository: https://github.com/Q-cue-ai/pytest-human
• Issue Tracker: https://github.com/Q-cue-ai/pytest-human/issues