Skip to main content

py.test plugin for coverage reporting with support for both centralised and distributed testing

Project description

This plugin produces coverage reports using the coverage package. It supports centralised testing and distributed testing in both load and each modes.

All features offered by the coverage package should be available, either through this plugin or through coverage’s own config file.

Installation

The pytest-cov pypi package may be installed / uninstalled with pip:

pip install pytest-cov
pip uninstall pytest-cov

Alternatively easy_install can be used:

easy_install pytest-cov

Usage

Centralised Testing

Running centralised testing:

py.test --cov myproj tests/

Shows a terminal report:

-------------------- coverage: platform linux2, python 2.6.4-final-0 ---------------------
Name                 Stmts   Exec  Cover   Missing
--------------------------------------------------
myproj/__init__          2      2   100%
myproj/myproj          257    244    94%   24-26, 99, 149, 233-236, 297-298, 369-370
myproj/feature4286      94     87    92%   183-188, 197
--------------------------------------------------
TOTAL                  353    333    94%

Distributed Testing

Distributed testing with dist mode set to load:

py.test --cov myproj -n 2 tests/

The results from the slaves will be combined like so:

-------------------- coverage: platform linux2, python 2.6.4-final-0 ---------------------
Name                 Stmts   Exec  Cover   Missing
--------------------------------------------------
myproj/__init__          2      2   100%
myproj/myproj          257    244    94%   24-26, 99, 149, 233-236, 297-298, 369-370
myproj/feature4286      94     87    92%   183-188, 197
--------------------------------------------------
TOTAL                  353    333    94%

Distributed testing in each mode:

py.test --cov myproj --dist=each
        --tx=popen//python=/usr/local/python265/bin/python
        --tx=popen//python=/usr/local/python27b1/bin/python
        tests/

Will produce a report for each slave:

-------------------- coverage: platform linux2, python 2.6.5-final-0 ---------------------
Name                 Stmts   Exec  Cover   Missing
--------------------------------------------------
myproj/__init__          2      2   100%
myproj/myproj          257    244    94%   24-26, 99, 149, 233-236, 297-298, 369-370
myproj/feature4286      94     87    92%   183-188, 197
--------------------------------------------------
TOTAL                  353    333    94%
--------------------- coverage: platform linux2, python 2.7.0-beta-1 ---------------------
Name                 Stmts   Exec  Cover   Missing
--------------------------------------------------
myproj/__init__          2      2   100%
myproj/myproj          257    244    94%   24-26, 99, 149, 233-236, 297-298, 369-370
myproj/feature4286      94     87    92%   183-188, 197
--------------------------------------------------
TOTAL                  353    333    94%

Distributed testing in each mode can also produce a single combined report. This is useful to get coverage information spanning things such as all python versions:

py.test --cov myproj --cov-combine-each --dist=each
        --tx=popen//python=/usr/local/python265/bin/python
        --tx=popen//python=/usr/local/python27b1/bin/python
        tests/

Which looks like:

---------------------------------------- coverage ----------------------------------------
                          platform linux2, python 2.6.5-final-0
                           platform linux2, python 2.7.0-beta-1
Name                 Stmts   Exec  Cover   Missing
--------------------------------------------------
myproj/__init__          2      2   100%
myproj/myproj          257    244    94%   24-26, 99, 149, 233-236, 297-298, 369-370
myproj/feature4286      94     87    92%   183-188, 197
--------------------------------------------------
TOTAL                  353    333    94%

Reporting

By default a terminal report is output. This report can be disabled if desired, such as when results are going to a continuous integration system and the terminal output won’t be seen.

In addition and without rerunning tests it is possible to generate annotated source code, a html report and an xml report.

The directories for annotated source code and html reports can be specified as can the file name for the xml report.

Since testing often takes a non trivial amount of time at the end of testing any / all of the reports may be generated.

Coverage Data File

During testing there may be many data files with coverage data. These will have unique suffixes and will be combined at the end of testing.

Upon completion, for –dist=load (and also for –dist=each when the –cov-combine-each option is used) there will only be one data file.

For –dist=each there may be many data files where each one will have the platform / python version info appended to the name.

These data files are left at the end of testing so that it is possible to use normal coverage tools to examine them.

At the beginning of testing any data files that are about to be used will first be erased so ensure the data is clean for each test run.

It is possible to set the name of the data file. If needed the platform / python version will be appended automatically to this name.

Coverage Config File

Coverage by default will read its own config file. An alternative file name may be specified or reading config can be disabled entirely.

Care has been taken to ensure that the coverage env vars and config file options work the same under this plugin as they do under coverage itself.

Since options may be specified in different ways the order of precedence between pytest-cov and coverage from highest to lowest is:

  1. pytest command line

  2. pytest env var

  3. pytest conftest

  4. coverage env var

  5. coverage config file

  6. coverage default

Limitations

For distributed testing the slaves must have the pytest-cov package installed. This is needed since the plugin must be registered through setuptools / distribute for pytest to start the plugin on the slave.

Acknowledgements

Holger Krekel for pytest with its distributed testing support.

Ned Batchelder for coverage and its ability to combine the coverage results of parallel runs.

Whilst this plugin has been built fresh from the ground up to support distributed testing it has been influenced by the work done on pytest-coverage (Ross Lawley, James Mills, Holger Krekel) and nose-cover (Jason Pellerin) which are other coverage plugins for pytest and nose respectively.

No doubt others have contributed to these tools as well.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pytest-cov-0.14.tar.gz (12.2 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page