Pytest plugin for additional test reporting

Offers additional 2 reporting options:

• Skipped tests percentage
• (For Selenium tests) HTML Coverage report

These two functionalities are not related and can be used separately. At the time, it's just easier to have them in the same codebase. Both of them have the option to output the report in terminal or Confluence.

Skipped test percentage

To execute it:

$ pytest demo --track terminal
============== test session starts ===================
plugins: track-0.1.0
collected 7 items

demo/test_models.py .s.s.                       [ 71%]
demo/test_views.py .s                           [100%]

======= 4 passed, 3 skipped in 0.02 seconds ==========
Total: 4 from 7 tests not skipped (57.14%)
    test_models, 3 from 5 tests not skipped (60.00%)
    test_views, 1 from 2 tests not skipped (50.00%)

Current functionality requires only test collection so this can be used with pytest's --collect-only

To store result in Confluence, see the Confluence Configuration section and use:

$ pytest --track confluence

HTML Coverage

To have this you need both to have selenium tests and access to the HTML source for the project you want to compute coverage.

This plugin works by recording all the identifiable source code elements in the HTML source and comparing how many of them the selenium tests view and inspect.

In greater details this is doing:

1. In the plugin configure phase, read HTML files and create a simple tree with identifiable elements

   • This step can be cached (See options)
   • Identifiable elements is a tag with an id or a class or aa special HTML tag (app- for Angular)

2. While tests run, on each Selenium driver URL get, record the received HTML as viewed

3. While tests run, on each Selenium driver find_element, record the element as inspected

4. In the plugin unconfigure phase, calculate a naive % HTML seen and % HTML inspected

   $ pytest --html-cov terminal --html-cov-dir=<directory_with_HTML_source>

   To cache HTML source parse set, use

   $ pytest --html-cov --html-cov-cache

   To also report missing elements, use

   $ pytest --html-cov --html-cov-show-elements

To store result in Confluence, see the Confluence Configuration section and use:

$ pytest --html-cov confluence

Confluence reporting

To configure Confluence settings add a section to pytest.ini

[pytest_track]
track_confluence_username=<confluence_username>
track_confluence_password=<confluence_password>
track_confluence_url=<confluence_root_url>
# For skipped test percentage
confluence_report_parent_page_id=<id_of_the_parent_page>
confluence_report_page_title=<title_for_the_results_page>
# For HTML coverage
confluence_coverage_parent_page_id=<id_of_the_parent_page>
confluence_coverage_page_title=<title_for_the_results_page>

Contrib

Before PRs, only:

pre-commit install
tox

Acknowledgements

Based on the initial work of Vasilica Dumbrava.