Skip to main content

Pytest fixture extending Numpy's allclose function

Project description

pytest-allclose

pytest-allclose provides the ~.allclose Pytest fixture, extending numpy.allclose with test-specific features.

A core feature of the ~.allclose fixture is that the tolerances for tests can be configured externally. This allows different repositories to share the same tests, but use different tolerances. See the “Configuration” section below for details.

Installation

To use this fixture, install with

pip install pytest-allclose

Usage

The ~.allclose fixture is used just like numpy.allclose.

import numpy as np

def test_close(allclose):
    x = np.linspace(-1, 1)
    y = x + 0.001
    assert allclose(y, x, atol=0.002)
    assert not allclose(y, x, atol=0.0005)
    assert not allclose(y, x, rtol=0.002)

Additional arguments

The ~.allclose fixture has a number of arguments that are not part of numpy.allclose. One such argument is xtol, which allows arrays that have been shifted along their first axis by a certain number of steps to be considered close.

import numpy as np

def test_close(allclose):
    x = np.linspace(-1, 1)

    assert allclose(x[1:], x[:-1], xtol=1)
    assert allclose(x[3:], x[:-3], xtol=3)
    assert not allclose(x[3:], x[:-3], xtol=1)

Refer to the ~.allclose API reference for all additional arguments.

RMSE error reporting

The ~.allclose fixture stores root-mean-square error values, which can be reported in the pytest terminal summary. To do so, put the following in your conftest.py file.

from pytest_allclose import report_rmses

def pytest_terminal_summary(terminalreporter):
    report_rmses(terminalreporter)

See the ~.report_rmses API reference for more information.

Configuration

allclose_tolerances

allclose_tolerances accepts a list of test name patterns, followed by values for any of the ~.allclose parameters. These values will override any values provided within the test function itself, allowing multiple repositories to use the same test suite, but with different tolerances.

allclose_tolerances =
    test_file.py:test_function atol=0.3  # set atol for specific test
    test_file.py:test_func* rtol=0.2  # set rtol for tests matching wildcard
    test_file.py:* atol=0.1 rtol=0.3  # set both tols for all tests in file
    test_*tion rtol=0.2  # set rtol for all matching tests in any file
    test_function[True] atol=0.1  # set atol only for one parametrization

The only special character recognized in these patterns is the wildcard character *, which matches any group of zero or more characters.

If the test is parametrized, then a pattern like test_name[param0-param1] will match specific parameter settings, and test_name* will match all parameter settings. Note that the latter will match any test that starts with test_name.

If a test has multiple ~.allclose calls, you can use multiple tolerance lines that match the same test to set different values for the first, second, third, etc. calls. If there are more ~.allclose calls than tolerance lines, the last tolerance line will be used for all remaining ~.allclose calls.

Example test file:

def test_close(allclose):
    x = np.linspace(-1, 1)
    y = x + 0.001
    assert allclose(y, x)
    assert not allclose(y, x)

Example configuration file (pytest.ini, setup.cfg):

allclose_tolerances =
    test_close atol=0.002  # affects first allclose call
    test_close atol=0.0005  # affects second allclose call

See the full documentation for the API reference.

Release History

1.0.0 (July 30, 2019)

Initial release of pytest-allclose! Thanks to all of the contributors for making this possible!

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pytest-allclose-1.0.0.tar.gz (18.6 kB view details)

Uploaded Source

Built Distribution

pytest_allclose-1.0.0-py3-none-any.whl (8.4 kB view details)

Uploaded Python 3

File details

Details for the file pytest-allclose-1.0.0.tar.gz.

File metadata

  • Download URL: pytest-allclose-1.0.0.tar.gz
  • Upload date:
  • Size: 18.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.32.2 CPython/3.6.3

File hashes

Hashes for pytest-allclose-1.0.0.tar.gz
Algorithm Hash digest
SHA256 b2f0c521fa652281400d4a105c84454db3c50b993bcfee9861380be69cc6b041
MD5 f8d65c9ec1933528a27f9d4f34c1b9e2
BLAKE2b-256 7a7c2b82fafea079feb080404b7611a4d07313ae816d2a35e7f2df084a3e002b

See more details on using hashes here.

File details

Details for the file pytest_allclose-1.0.0-py3-none-any.whl.

File metadata

  • Download URL: pytest_allclose-1.0.0-py3-none-any.whl
  • Upload date:
  • Size: 8.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.32.2 CPython/3.6.3

File hashes

Hashes for pytest_allclose-1.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 ee1a0d5dd0322c2439aa30883ca6748f18150e7e708209852a4b67444790bc97
MD5 fceeb16a03be7a567e559c4a36ee17e7
BLAKE2b-256 c295afa083bad44faa65048a9fbc018e9bc1b40045d4232383e723a555cd8b7c

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page