Skip to main content

take TDD to a new level with py.test and testmon

Project description

Build Status

This is a py.test plug-in which automatically selects and re-executes only tests affected by recent changes. How is this possible in dynamic language like Python and how reliable is it? Read here: Determining affected tests

New in version 0.4+ run_variants is no longer. User run_variant_expression (see below)!

New versions usually have new dataformat, don’t forget to rm .testmondata after each upgrade.

testmon is approaching completeness. Unfortunatelly the classic console UI is reaching it’s usability limits even without testmon. With testmon it’s even a little more difficult to determine which tests are beeing executed, which are failing and why. Next step would be an implementation or integration of GUI. I don’t like any of the existing graphical test runners, so if you have some better new concept in mind, get in touch!

Usage

pip install pytest-testmon

# build the dependency database and save it to .testmondata
py.test --testmon

# list of watched project files ordered by tests which reach each specific file
py.test --by-test-count

# change some of your code (with test coverage)

# only run tests affected by recent changes
py.test --testmon

# start from scratch (if needed)
rm .testmondata

# a simple watchdog command for file changes in source tree
tmon.py

Other switches

–project-directory= only files in under this directory will be tracked by coveragepy. Default is rootdir, can be repeated

Configuration

Add testmon to the pytest.ini

[pytest]
#if you want to separate different environments running the same sources
run_variant_expression = os.environ.get('DJANGO_SETTINGS_MODULE') + ':python' + str(sys.version_info[:2])
addopts = --testmon # you can make --testmon a default if you want

Thoughts

Individual test outcomes depend on many things, so let’s write a little about some of them.

  1. executed python code inside the tested project (which presumably changes very frequently, little by little)
  2. environment variables (e.g. DJANGO_SETTINGS_MODULE), python version (the run_variant_expression config value denotes these)
  3. executed python code in all of the libraries (which presumably change infrequently)
  4. data files (txt, xml, other project assets)
  5. external services (reached through network)

testmon so far deals with incrementally running tests when faced with the 1. and 2. category of changes.

Later versions can implement some detection of other categories

libraries: we could compare pip freeze between runs, but it’s slow

data files: Probably the best bet here is a configuration where the developer would specify which files does a test depend on

Sponsors

Big thanks to Qvantel, Nick Coghlan , Abilian SAS and Infinit for beeing silver sponsors of the first release of testmon. List of all contributors to our campaing is here . Thanks a lot to all contributors.

Project details


Release history Release notifications

History Node

0.9.11

History Node

0.9.10

History Node

0.9.9

History Node

0.9.8

History Node

0.9.7

History Node

0.9.6

History Node

0.9.5

History Node

0.9.4

History Node

0.9.3

History Node

0.9.0

History Node

0.8.3

History Node

0.8.2

History Node

0.8.1

History Node

0.8.0

History Node

0.7

History Node

0.6

This version
History Node

0.5

History Node

0.4.9

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
pytest-testmon-0.5.tar.gz (12.4 kB) Copy SHA256 hash SHA256 Source None Jun 15, 2015

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page