py.test plugin for coverage reporting with support for both centralised and distributed testing
Project description
This plugin produces coverage reports. It supports centralised testing and distributed testing in both load and each modes. It also supports coverage of subprocesses.
All features offered by the coverage package should be available, either through pytest-cov or through coverage’s config file.
Installation
The pytest-cov package may be installed with pip or easy_install:
pip install pytest-cov easy_install pytest-cov
Uninstallation
Uninstalling packages is supported by pip:
pip uninstall pytest-cov
However easy_install does not provide an uninstall facility.
Usage
Centralised Testing
Centralised testing will report on the combined coverage of the main process and all of it’s subprocesses.
Running centralised testing:
py.test --cov myproj tests/
Shows a terminal report:
-------------------- coverage: platform linux2, python 2.6.4-final-0 --------------------- Name Stmts Miss Cover ---------------------------------------- myproj/__init__ 2 0 100% myproj/myproj 257 13 94% myproj/feature4286 94 7 92% ---------------------------------------- TOTAL 353 20 94%
Distributed Testing: Load
Distributed testing with dist mode set to load will report on the combined coverage of all slaves. The slaves may be spread out over any number of hosts and each slave may be located anywhere on the file system. Each slave will have it’s subprocesses measured.
Running distributed testing with dist mode set to load:
py.test --cov myproj -n 2 tests/
Shows a terminal report:
-------------------- coverage: platform linux2, python 2.6.4-final-0 --------------------- Name Stmts Miss Cover ---------------------------------------- myproj/__init__ 2 0 100% myproj/myproj 257 13 94% myproj/feature4286 94 7 92% ---------------------------------------- TOTAL 353 20 94%
Again but spread over different hosts and different directories:
py.test --cov myproj --dist load
--tx ssh=memedough@host1//chdir=testenv1
--tx ssh=memedough@host2//chdir=/tmp/testenv2//python=/tmp/env1/bin/python
--rsyncdir myproj --rsyncdir tests --rsync examples
tests/
Shows a terminal report:
-------------------- coverage: platform linux2, python 2.6.4-final-0 --------------------- Name Stmts Miss Cover ---------------------------------------- myproj/__init__ 2 0 100% myproj/myproj 257 13 94% myproj/feature4286 94 7 92% ---------------------------------------- TOTAL 353 20 94%
Distributed Testing: Each
Distributed testing with dist mode set to each will report on the combined coverage of all slaves. Since each slave is running all tests this allows generating a combined coverage report for multiple environments.
Running distributed testing with dist mode set to each:
py.test --cov myproj --dist each
--tx popen//chdir=/tmp/testenv3//python=/usr/local/python27/bin/python
--tx ssh=memedough@host2//chdir=/tmp/testenv4//python=/tmp/env2/bin/python
--rsyncdir myproj --rsyncdir tests --rsync examples
tests/
Shows a terminal report:
---------------------------------------- coverage ----------------------------------------
platform linux2, python 2.6.5-final-0
platform linux2, python 2.7.0-final-0
Name Stmts Miss Cover
----------------------------------------
myproj/__init__ 2 0 100%
myproj/myproj 257 13 94%
myproj/feature4286 94 7 92%
----------------------------------------
TOTAL 353 20 94%
Reporting
It is possible to generate any combination of the reports for a single test run.
The available reports are terminal (with or without missing line numbers shown), HTML, XML and annotated source code.
The terminal report without line numbers (default):
py.test --cov-report term --cov myproj tests/ -------------------- coverage: platform linux2, python 2.6.4-final-0 --------------------- Name Stmts Miss Cover ---------------------------------------- myproj/__init__ 2 0 100% myproj/myproj 257 13 94% myproj/feature4286 94 7 92% ---------------------------------------- TOTAL 353 20 94%
The terminal report with line numbers:
py.test --cov-report term-missing --cov myproj tests/ -------------------- coverage: platform linux2, python 2.6.4-final-0 --------------------- Name Stmts Miss Cover Missing -------------------------------------------------- myproj/__init__ 2 0 100% myproj/myproj 257 13 94% 24-26, 99, 149, 233-236, 297-298, 369-370 myproj/feature4286 94 7 92% 183-188, 197 -------------------------------------------------- TOTAL 353 20 94%
The remaining three reports output to files without showing anything on the terminal (useful for when the output is going to a continuous integration server):
py.test --cov-report html --cov-report xml --cov-report annotate --cov myproj tests/
Coverage Data File
The data file is erased at the beginning of testing to ensure clean data for each test run.
The data file is left at the end of testing so that it is possible to use normal coverage tools to examine it.
Limitations
For distributed testing the slaves must have the pytest-cov package installed. This is needed since the plugin must be registered through setuptools / distribute for pytest to start the plugin on the slave.
For subprocess measurement environment variables must make it from the main process to the subprocess. The python used by the subprocess must have pytest-cov installed. The subprocess must do normal site initialisation so that the environment variables can be detected and coverage started.
Acknowledgements
Holger Krekel for pytest with its distributed testing support.
Ned Batchelder for coverage and its ability to combine the coverage results of parallel runs.
Whilst this plugin has been built fresh from the ground up to support distributed testing it has been influenced by the work done on pytest-coverage (Ross Lawley, James Mills, Holger Krekel) and nose-cover (Jason Pellerin) which are other coverage plugins for pytest and nose respectively.
No doubt others have contributed to these tools as well.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
