Run object-oriented tests in a simple format
Project description
pytest-oot implements a simple way to write a test step for system test engineers. It can be used for two ways:
To directly use step() functions in a test_*.py file to reduce code lines
To directly write a test_*.oot file to use a simple case/step language
The module is still under development. The following features will be added in:
A hook to add new operators to enhance the conditions judgement
A hook to add more options for steps
A hook to register logging for both pass/fail steps or cases
You can download the whole package to get the examples
Install pytest-oot
pip install pytest-oot
Example for using step functions in a test_*.py
This is an easy way to use it, and options provide some tests specific functions. The step functions include step(), steps() and s(). The format of a step looks like:
obj.method(parameter) op exp-value options
In this one step, there is an action, and also a check point This one step can be translated to multiple lines of python code, or dozens lines of code if there is one option or multiple options.
- How to import::
from pytest_oot.oot_step import step, steps, s
Examples (Quick Start):
step(“num1.add(3,4,5) == 23”)
It is similar as:
assert num1.add(3,4,5) == 23
very simple, right? (we do not consider advanced features here, e.g. auto-logging)
- step(“string1.range(1..4) !~ r’w-w’”)
Perl-like condition, =~ means ‘contains’, and !~ means ‘not contains’. btw, regex can be used. The step is like:
import re assert re.compile(r'\w\-\w').find(string1.range(1..4))
- step(“num_async.data_sync() -t 15”)
A little complicated, -t means timeout. In this step, a time-out timer is set to 15 seconds. It means this step is allowed to be completed in 15 seconds, otherwise, it fails. no op (==, <, >, =~, etc.) in this step, it means no assert required to check the return value
This is implemented by forking another thread to run the step. Considering some tests require to wait for a response, but how long? this can be useful
- step(“num_async.get_value() == 500 –repeat 20”)
Another option –repeat (same as -r). The step means the step will be re-run every another second in total 20 seconds, until the condition comes true
If the condition is always false in 20 seconds, then the step fails
step(“num2.multiple(4,5) == 460 -x True -t 12 -r 10”)
Multiple options for one step
-x (--expectedfail): pass if the condition is not met -t (--timeout): set a timeout timer -r (--repeat): repeat this step in 10 seconds until it comes true (here false actually due to -x), or timeout
- steps(‘’’
num1.add(4)
num2.add(3,4,5,6) == 23
num2.multiple(4,5) == 460 -x True -t 12 -r 10
num3.add(3,4,var2) == 1000 –skip -t 25
‘’’)
multiple steps in one shot
Example for test_*.oot file
Once the plug-in is installed, the pytest will automatically collect test_*.oot files to get cases, and run each items in the files. In a test_*.oot file, each case is a test item, and each line under it is a test step.
Example file: test_number.oot (you can get it from the source package)
# Any words after # in a line are just comments
# One file is a test suite. The test suite description
test_suite: Trial1
# Identify the test bed file, currently .py file is supported
# similar as 'import testbed.py' in test*.py file
test_bed: example.test.testbed
# A case starts from a case_idString, the description is in the bracket
# This is to define one case, just like a function or method in a .py file
# case_id1 means the function name is "id1"
case_id1 (NumberBase add function):
# under a case, there could be multiple test steps, one step in one line
# step format: obj.method(parameters) operator expected_result options
# obj/methods are defined in test bed file
# operator supports:
# ==(equal to), !=(not equal to), >(larger than), <(less than), >=, <=,
# =~(for string, contains, e.g. "hello world" =~ "llo", regex allowed
# !~ (not contain)
num1.add(3,4,5,6) == 23 -t 3
num1.add(var1, var2, var3) == 18
case_id2 (NumberBase multiple function):
num1.multiple(2,4,5) == 200
case_id3 (NumberChange test):
# Every line under the case line is a step of a case
# there could be multiple lines; each line follows the format:
# obj.method([parameter1 [,parameter 2 [, ...]]] operator ExpectedValue -options
# For details, see guidance ....
# options:
# --timeout 30 == -t 30: fail if the step could not complete in 30 seconds
# --repeat 30 == -r 30: repeat per second if fail until pass, timeout in 30s
# --duration 30 == -d 30: duration of the step is 30s, if completed early, just wait until 30s
# --expectedfail == -x true == -x: If step fail, then report pass
# --skip == -s: just skip this step
#
num1.add(4)
num2.add(3,4,5,6) == 23
num2.multiple(4,5) == 460 -x True -t 12 -r 10
num3.add(3,4,var2) == 1000 --skip -t 25
case_id4 (Reverse String test):
string1.range(1,4) == 'dlr' -d 6
case_async1 (To test async actions - timeout)
num_async.addw(var100, var100) == 100
num_async.data_sync() -t 18
num_async.get_value() == 300
case_async2 (To test async actions - repeat)
num_async.addw(var100, var100) >= 300
num_async.get_value() == 500 --repeat 20
Note: If it reports the testbed module could not be imported, please change the module path to make it right.
Operators & Options
Sometimes it is needed to declare the same fixtures or steps with the different names for better readability. In order to use the same step
- Supported Operators by default:
==, !=, <. >, <=, >=, =~, !~
Supported Options by default:
# --timeout 30 == -t 30: fail if the step could not complete in 30 seconds # --repeat 30 == -r 30: repeat per second if fail until pass, timeout in 30s # --duration 30 == -d 30: duration of the step is 30s, if completed early, just wait until 30s # --expectedfail == -x true == -x: If step fail, then report pass # --skip == -s: just skip this step
Test bed
If you use step functions in a .py file, it is required to make sure the objects in the step string are in the module’s name space.
If you are using a test_*.oot file, you need to use
testbed = [module.]testbedfilename
to import all the objects defined in the testbedfilename.py file.
Hooks
pytest-oot is to support multiple hooks for operator, logs, and options next. Please send mails to steven004@gmail.com if you have any comments or suggestions
License
This software is licensed under the MIT license.
© 2014 Steven LI
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.