pytest-regtest 2.2.0

pytest plugin for snapshot regression testing

About

Introduction

pytest-regtest is a plugin for pytest to implement regression testing.

Unlike functional testing, regression testing testing does not test whether the software produces the correct results, but whether it behaves as it did before changes were introduced.

More specifically, pytest-regtest provides snapshot testing, which implements regression testing by recording data within a test function and comparing this recorded output to a previously recorded reference output.

Installation

To install and activate this plugin execute:

$ pip install pytest-regtest

Use case 1: Changing code with no or little testing setup yet

If you're working with code that has little or no unit testing, you can use regression testing to ensure that your changes don't break or alter previous results.

Example: This can be useful when working with scientific data analysis scripts, which often start a long script and then are restructured into different functions.

Use case 2: Testing complex data

If your unit tests contain a lot of assert statements to check a complex data structure you can use regression tests instead.

Example: To test code which ingests data into a database one can use regression tests on textual database dumps.

Use case 3: Testing numpy arrays or pandas data frames

If code produces numerical results, such as numpy arrays or pandasdata frames, you can usepytest-regtest` to simply record such results and test later with considering relative and absolute tolerances.

Example: A function creates a 10 x 10 matrix. Either you have to write 100 assert statements or you use summary statistics to test your result. In both cases, you may get little debugging information if a test fails.