Skip to main content
Join the official Python Developers Survey 2018 and win valuable prizes: Start the survey!

Multi-dimensional parameterized tests for Pytest and Nose

Project description

https://travis-ci.org/akaihola/testdimensions.svg?branch=master

There are multiple ways to write parameterized tests in Python. Unittest has some support these days, Nose allows yielding test cases, Pytest has built-in parameterization support, and the excellent nose_parameterized package enhances these capabilities in most test frameworks.

testdimensions provides a convenient way to write multi-dimensional test matrices in some simple scenarios. If your function accepts multiple arguments and you want to test a cross product set of parameter combinations, testdimensions is for you.

Specify your tests as a table whose:

  • Y axis labels are values for the third-last parameter
  • X axis labels are values for the second-last paremeter
  • cell values are the expected values (last parameter)
  • columns are separated by two spaces (make sure this is true on all rows)
# test_math.py
@pytest_mark_dimensions('base,exponent,expected', '''
    # y: base
    # x: exponent
    # cell: expected

          2    3    9
     0    0    0    0
     1    1    1    1
     2    4    8  512
''')
def test_pow(base, exponent, expected):
    assert math.pow(base, exponent) == expected


@pytest_mark_dimensions('input,function,expected', '''
                round  math.floor  math.ceil
    -1.5         -2.0        -2.0       -1.0
     1.0          1.0         1.0        1.0
     1.6          2.0         1.0        2.0
''')
def test_round_floor_ceil(input, function, expected):
    assert function(input) == expected

Output:

$ pytest -v
=========================== test session starts ===============================
platform linux -- Python 3.5.2, pytest-3.0.3, py-1.4.31, pluggy-0.4.0
collected 18 items

test_math.py::test_pow[0-2-0] PASSED
test_math.py::test_pow[0-3-0] PASSED
test_math.py::test_pow[0-9-0] PASSED
test_math.py::test_pow[1-2-1] PASSED
test_math.py::test_pow[1-3-1] PASSED
test_math.py::test_pow[1-9-1] PASSED
test_math.py::test_pow[2-2-4] PASSED
test_math.py::test_pow[2-3-8] PASSED
test_math.py::test_pow[2-9-512] PASSED
test_math.py::test_round_floor_ceil[-1.5-function0--2.0] PASSED
test_math.py::test_round_floor_ceil[-1.5-function1--2.0] PASSED
test_math.py::test_round_floor_ceil[-1.5-function2--1.0] PASSED
test_math.py::test_round_floor_ceil[1.0-function3-1.0] PASSED
test_math.py::test_round_floor_ceil[1.0-function4-1.0] PASSED
test_math.py::test_round_floor_ceil[1.0-function5-1.0] PASSED
test_math.py::test_round_floor_ceil[1.6-function6-2.0] PASSED
test_math.py::test_round_floor_ceil[1.6-function7-1.0] PASSED
test_math.py::test_round_floor_ceil[1.6-function8-2.0] PASSED

============================ 18 passed in 0.03 seconds ========================

Installation

$ pip install testdimensions

Compatibility

  Py2.6 Py2.7 Py3.3 Py3.4 Py3.5 PyPy
nose no yes no no yes no
nose2 no no no no no no
py.test not tested yes not tested not tested yes not tested
unittest no no no no no no
unittest2 no no no no no no

Dependencies

Exhaustive Usage Examples

The @pytest_mark_dimensions decorator is an extension of @pytest.mark.parametrize and requires a comma-separated list of test parameters as its first argument. The second argument is a multi-line string which defines the actual tests. You can also inject values into the test globals namespace using keyword arguments.

To create higher than two-dimensional tests, just define multiple tables interspersed with values for the additional parameters.

@pytest_mark_dimensions('a,b,expected', '''
            -10   0   9  million
    -9      -19  -9   0   999991
     0      -10   0   9  million
    10        0  10  19  1000010
    ''',
    million=1000000)
def test_add(a, b, expected):
    assert a + b == expected


@pytest_mark_dimensions('operation,a,b,expected', '''
    operation = operator.sub

            -10   0    9   million
    -9        1  -9  -18  -1000009
     0       10   0   -9  -million
    10       20  10    1   -999990

    operation = operator.add

            -10   0   9  million
    -9      -19  -9   0   999991
     0      -10   0   9  million
    10        0  10  19  1000010

    operation = operator.mul

            -10   0    9   million
    -9       90   0  -81  -9000000
     0        0   0    0         0
    10     -100   0   90  10000000

    ''',
    million=1000000)
def test_arithmetic_operations(operation, a, b, expected):
    assert operation(a, b) == expected

For Nose support, you need to install nose_parameterized and use the @nosedimensions decorator:

@nosedimensions('a,b,expected', '''
            -10   0   9  million
    -9      -19  -9   0   999991
     0      -10   0   9  million
    10        0  10  19  1000010
    ''',
    million=1000000)
def test_add(a, b, expected):
    assert a + b == expected


@nosedimensions('operation,a,b,expected', '''
    operation = operator.sub

            -10   0    9   million
    -9        1  -9  -18  -1000009
     0       10   0   -9  -million
    10       20  10    1   -999990

    operation = operator.add

            -10   0   9  million
    -9      -19  -9   0   999991
     0      -10   0   9  million
    10        0  10  19  1000010

    operation = operator.mul

            -10   0    9   million
    -9       90   0  -81  -9000000
     0        0   0    0         0
    10     -100   0   90  10000000

    ''',
    million=1000000)
def test_arithmetic_operations(operation, a, b, expected):
    assert operation(a, b) == expected

Note that you still need to enumerate the test parameters just like with Pytest.

Project details


Release history Release notifications

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
testdimensions-0.0.1.tar.gz (4.7 kB) Copy SHA256 hash SHA256 Source None Nov 16, 2016

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page