pytest-oot 0.2.6

Run object-oriented tests in a simple format

pytest-oot implements a simple way to write a test step for system test engineers. It can be used for two ways:

• To directly use step() functions in a test_*.py file to reduce code lines

• To directly write a test_*.oot file to use a simple case/step language

The module is still under development. The following features will be added in:

• A hook to add new operators to enhance the conditions judgement

• A hook to add new options for test steps

• A hook to register logging function for both pass/fail steps or cases

You can download the whole package to get the examples

Install pytest-oot

pip install pytest-oot

Example for using step functions in a test_*.py

This is an easy way to use it, and options provide some tests specific functions. The step functions include step(), steps() and s(). The format of a step looks like:

obj.method(parameter) op exp-value options

In this one step, there is an action, and also a check point This one step can be translated to multiple lines of python code, or dozens lines of code if there is one option or multiple options.

How to import:

from pytest_oot.oot_step import step, steps, s

Examples (Quick Start):

1. step(“num1.add(3,4,5) == 23”)

   the same as:

   assert num1.add(3,4,5) == 23

   very simple, right?

2. step(“string1.range(1..4) !~ r’w-w’”)

   Perl-like condition, =~ means ‘contains’, and !~ means ‘not contains’. btw, regex can be used. The step is like:

   import re
   assert re.compile(r'\w\-\w').find(string1.range(1..4))

3. step(“num_async.data_sync() -t 15”)

   A little complicated, -t means timeout. In this step, a time-out timer is set to 15 seconds. It means this step is allowed to be completed in 15 seconds, otherwise, it fails. no op (==, <, >, =~, etc.) in this step, it means no assert required to check the return value

   This is implemented by forking another thread to run the step. Considering some tests require to wait for a response, but how long? this can be useful

4. step(“num_async.get_value() == 500 –repeat 20”)

   Another option –repeat (same as -r). The step means the step will be re-run every another second in total 20 seconds, until the condition comes true

   If the condition is always false in 20 seconds, then the step fails

5. step(“num2.multiple(4,5) == 460 -x True -t 12 -r 10”)

   Multiple options for one step

   -x (--expectedfail): pass if the condition is not met
   -t (--timeout): set a timeout timer
   -r (--repeat): repeat this step in 10 seconds until it comes true
      (here false actually due to -x), or timeout

6. steps(‘’’

   num1.add(4)

   num2.add(3,4,5,6) == 23

   num2.multiple(4,5) == 460 -x True -t 12 -r 10

   num3.add(3,4,var2) == 1000 –skip -t 25

   ‘’’)

   multiple steps in one shot

Example for test_*.oot file

Once the plug-in is installed, the pytest will automatically collect test_*.oot files to get cases, and run each items in the files. In a test_*.oot file, each case is a test item, and each line under it is a test step.

Example file: test_number.oot (you can get it from the source package)

# Any words after # in a line are just comments
# One file is a test suite. The test suite description
test_suite: Trial1

# Identify the test bed file, currently .py file is supported
# similar as 'import testbed.py' in test*.py file
test_bed: example.test.testbed

# A case starts from a case_idString, the description is in the bracket
# This is to define one case, just like a function or method in a .py file
# case_id1 means the function name is "id1"
case_id1 (NumberBase add function):
    # under a case, there could be multiple test steps, one step in one line
    # step format: obj.method(parameters) operator expected_result options
        # obj/methods are defined in test bed file
        # operator supports:
        #   ==(equal to), !=(not equal to), >(larger than), <(less than), >=, <=,
        #   =~(for string, contains, e.g. "hello world" =~ "llo", regex allowed
        #   !~ (not contain)
    num1.add(3,4,5,6) == 23 -t 3
    num1.add(var1, var2, var3) == 18

case_id2 (NumberBase multiple function):
    num1.multiple(2,4,5) == 200

case_id3 (NumberChange test):
    # Every line under the case line is a step of a case
    # there could be multiple lines; each line follows the format:
    #   obj.method([parameter1 [,parameter 2 [, ...]]] operator ExpectedValue -options
    # For details, see guidance ....
    # options:
    # --timeout 30 == -t 30: fail if the step could not complete in 30 seconds
    # --repeat 30 == -r 30: repeat per second if fail until pass, timeout in 30s
    # --duration 30 == -d 30: duration of the step is 30s, if completed early, just wait until 30s
    # --expectedfail == -x true == -x: If step fail, then report pass
    # --skip == -s: just skip this step
    #
    num1.add(4)
    num2.add(3,4,5,6) == 23
    num2.multiple(4,5) == 460 -x True -t 12 -r 10
    num3.add(3,4,var2) == 1000 --skip -t 25

case_id4 (Reverse String test):
    string1.range(1,4) == 'dlr' -d 6

case_async1 (To test async actions - timeout)
    num_async.addw(var100, var100) == 100
    num_async.data_sync() -t 18
    num_async.get_value() == 300

case_async2 (To test async actions - repeat)
    num_async.addw(var100, var100) >= 300
    num_async.get_value() == 500 --repeat 20

Operators & Options

Sometimes it is needed to declare the same fixtures or steps with the different names for better readability. In order to use the same step

Supported Operators by default:

==, !=, <. >, <=, >=, =~, !~

Supported Options by default:

# --timeout 30 == -t 30: fail if the step could not complete in 30 seconds
# --repeat 30 == -r 30: repeat per second if fail until pass, timeout in 30s
# --duration 30 == -d 30: duration of the step is 30s, if completed early,
  just wait until 30s
# --expectedfail == -x true == -x: If step fail, then report pass
# --skip == -s: just skip this step

Test bed

If you use step functions in a .py file, it is required to make sure the objects in the step string are in the module’s name space.

If you are using a test_*.oot file, you need to use

testbed = [module.]testbedfilename

to import all the objects defined in the testbedfilename.py file.

Hooks

pytest-oot is to support multiple hooks for operator, logs, and options next. Please send mails to [e-mail] if you have any comments or suggestions

License

This software is licensed under the MIT license.

© 2014 Steven LI

Changelog

0.2.5

• Update the license file and PYPI related documentation.

0.2.4

• The first version of the implementation of pytest_oot plugin and step functions