refery 1.0.0

Functional testing tool

Refery

Refery is a functional test tool written in Python

Install

$ ./setup.py install

Usage

$ refery --help
usage: refery [-h] --exec EXEC --test-file TEST_FILE

options:
  -h, --help            show this help message and exit
  --exec EXEC           path to the tested executable
  --test-file TEST_FILE
                        path to the yaml test file

As you can see, refery takes two mandatory parameters:

• The path to an executable file
• The path to a YAML file describing the test suite. The content of this file is described in the next section

Writing tests

Test suites are described in a YAML file. Each test case is a YAML mapping accepting the following optional keys:

• arg: YAML sequence containing the arguments passed to the executable
• stdin: String passed a standard output
• ref: Path to an executable with the desired behaviour
• stdout: Expected standard output
• stderr: Expected standard error
• exit_code: Expected exit code
• stdout_mode | stderr_mode: The testing mode of the standard output and error. Can be of two kinds:
  • strict:The actual value shall be the same as the expected value. This is the default mode
  • exists: If the expected value is not empty, the actual value shall not be empty and reciprocally
• skipped: Boolean equals to true if the test case shall be ignored

If the ref is specified, it is used to test the standard output, standard error and exit code. If any of these fields is specified, its value is used instead of the ref.

For example, take the following test case:

hello:
  ref: bin/hello.sh
  exit_code: 0

The stdout and stderr are tested according to bin/hello.sh but the exit code must be equal to 0, not matter what is actually returned by bin/hello.sh.