eztest 2.0.1

eztest is a Python package and execution used for performance testing or load testing.

eztest is a Python package and execution used for performance/load testing.

Home page: https://github.com/lgt1001/eztest

Features

Test:

• Normal: Run selected cases only once.

• Continuous: Run cases [repeat] times with [interval] seconds’ sleeping.

• Simultaneous: Start [stress] threads and run cases in each thread, sleep [interval] seconds after all cases are finished, and then start testing again with [repeat] times.

• Concurrency: Start [stress] threads and each thread will continuously run cases with [interval] seconds’ sleeping.

• Frequent: Start [stress] threads per [interval] seconds. And only can have [limit] available threads running.

Report:

• Start|Stop report server, and eztest can send and save report on remote server.

• Dump failure rate and average of time taken from remote report server.

• Calculate failure rate and average of time taken for report files generated by eztest.

Control:

• Stop testing and report server.

Usage

eztest command:

$ eztest -h
usage: eztest [-h] [--version] {test,stop,calc,server,dump} ...

eztest

positional arguments:
  {test,stop,calc,server,dump}
    test                Start eztest for target cases, classes, modules.
    stop                Stop eztest and its report server.
    calc                Calculate report files generated by eztest.
    server              Start|Stop|Restart report server.
    dump                Dump data from report server.

optional arguments:
  -h, --help            show this help message and exit
  --version, -v         show program's version number and exit

usage: eztest [-h] [--target TARGET] [--classes CLASSES [CLASSES ...]]
              [--not-classes NOT_CLASSES [NOT_CLASSES ...]]
              [--cases CASES [CASES ...]]
              [--not-cases NOT_CASES [NOT_CASES ...]]
              [--mode {0,1,2,3,4,normal,continuous,simultaneous,concurrency,frequent}]
              [--stress STRESS] [--repeat REPEAT] [--interval INTERVAL]
              [--limit LIMIT] [--starts STARTS] [--duration DURATION]
              [--ends ENDS] [--mail-config MAIL_CONFIG]
              [--report-folder REPORT_FOLDER] [--noreport] [--nolog]
              [--calc CALC [CALC ...]] [--group-minutes GROUP_MINUTES]
              [--version]

eztest test command:

$ eztest test -h
usage: eztest test [-h] --target TARGET [--classes CLASSES [CLASSES ...]]
               [--not-classes NOT_CLASSES [NOT_CLASSES ...]]
               [--cases CASES [CASES ...]]
               [--not-cases NOT_CASES [NOT_CASES ...]]
               [--mode {0,1,2,3,4,normal,continuous,simultaneous,concurrency,frequent}]
               [--stress STRESS] [--repeat REPEAT] [--interval INTERVAL]
               [--limit LIMIT] [--starts STARTS] [--duration DURATION]
               [--ends ENDS] [--report-folder REPORT_FOLDER]
               [--report-server REPORT_SERVER] [--noreport] [--nolog]
               [--mail-config MAIL_CONFIG]

optional arguments:
  -h, --help            show this help message and exit

Case Group:
  Define arguments of case related.

  --target TARGET, -t TARGET
                        Folder or file path, or a module, a __init__.py file is required under that folder/module.
  --classes CLASSES [CLASSES ...], -cl CLASSES [CLASSES ...]
                        Class names to be tested. It will be considered if target is file.
  --not-classes NOT_CLASSES [NOT_CLASSES ...], -ncl NOT_CLASSES [NOT_CLASSES ...]
                        Class names to be ignored. It will be considered if target is file.
  --cases CASES [CASES ...], -c CASES [CASES ...]
                        Case names to be tested.
                        It can be whole case name or part of them(e.g.: "*a", "a*", "*a*").
  --not-cases NOT_CASES [NOT_CASES ...], -nc NOT_CASES [NOT_CASES ...]
                        Case names to be ignored.
                        It can be whole case name or part of them(e.g.: "*a", "a*", "*a*").

Test Mode Group:
  Define arguments of test mode related.

  --mode {0,1,2,3,4,normal,continuous,simultaneous,concurrency,frequent},
  -m {0,1,2,3,4,normal,continuous,simultaneous,concurrency,frequent}
                        (a)0 or normal: Run selected cases only once.
                        (b)1 or continuous: Run cases [repeat] times with [interval] seconds' sleeping.
                        (c)2 or simultaneous: Start [stress] threads and run cases in each thread,
                        sleep [interval] seconds after all cases are finished,
                        and then start testing again with [repeat] times.
                        (d)3 or concurrency: Start [stress] threads and each thread will continuously run
                        cases with [interval] seconds' sleeping.
                        (e)4 or frequent: Start [stress] threads per [interval] seconds.
                        And only can have [limit] available threads running.
  --stress STRESS, -s STRESS
                        Start [stress] threads in each round of testing. Default value is 1.
  --repeat REPEAT, -r REPEAT
                        Repeat [repeat] times of testing. Default value is 1
  --interval INTERVAL, -i INTERVAL
                        Sleep [interval] seconds after one round of testing. Default value is 0.
  --limit LIMIT, -l LIMIT
                        Only can have [limit] count of running threads.
                        No limitation if this is less than or equals to [stress].
  --starts STARTS, -st STARTS
                        Testing will be started at [starts]. It is datetime string(e.g.: "2014-01-02 03:04:05").
  --duration DURATION, -d DURATION
                        Testing will continue with [duration] minutes. Will be ignored if 'ends' is provided.
  --ends ENDS, -et ENDS
                        Testing will be stopped at [ends]. It is datetime string(e.g.: "2014-01-02 03:04:05").

Report/Log Group:
  Define arguments of report or log related.

  --report-folder REPORT_FOLDER, -rf REPORT_FOLDER
                        Report and log files will be saved under [report-folder].
  --report-server REPORT_SERVER, -rs REPORT_SERVER
                        Report server.
                        The format is "host_name:port_number" or "host_name" with default port number 8765.
  --noreport, -nr       No report file will be generated if [noreport] is clarified.
  --nolog, -nl          No log file will be generated if [nolog] is clarified.
  --mail-config MAIL_CONFIG, -mc MAIL_CONFIG
                        Mail configuration file which contains mail server information etc.
                        It should be INI format file(http://en.wikipedia.org/wiki/INI_file).
                        Will send report by mail only if mail-config is provided and report file is generated.
                        Section is "SMTP" and properties can be "server", "from_mail", "to_mails",
                        "cc_mails", "bcc_mails", "username", "password", "need_authentication" and "subject".
                        "server", "from_mail" and "to_mails" are mandatory.
                        "to_mails", "cc_mails" and "bcc_mails" can be multiple values separated by comma.
                        "need_authentication" is boolean, "username" and "password" are required if
                        "need_authentication" is True.

eztest server command:

$eztest server -h
usage: eztest server [-h] {start,stop} ...

positional arguments:
  {start,stop}

optional arguments:
  -h, --help    show this help message and exit


$eztest server start -h
usage: eztest server start [-h] [--port PORT] [--handler HANDLER]

optional arguments:
  -h, --help            show this help message and exit
  --port PORT, -p PORT  Port number.
  --handler HANDLER, -hl HANDLER
                        Custom handler.
                        The format is: "file_path:handler_class_name", or "module_name:handler_class_name".

eztest dump command:

$eztest dump -h
usage: eztest dump [-h] [--report-server REPORT_SERVER]

optional arguments:
  -h, --help            show this help message and exit
  --report-server REPORT_SERVER, -rs REPORT_SERVER
                        Report server.
                        The format is "host_name:port_number" or "host_name" with default port number 8765.

eztest calc command:

$eztest calc -h
usage: eztest calc [-h] --path PATH [PATH ...] [--group-minutes GROUP_MINUTES]

optional arguments:
  -h, --help            show this help message and exit
  --path PATH [PATH ...], -p PATH [PATH ...]
                        Report folders or files to be calculated.
  --group-minutes GROUP_MINUTES, -gm GROUP_MINUTES
                        Calculate by grouping case results with [group-minutes] minutes. Default is 60 minutes.

Examples

Test examples:

# Normal testing
$ eztest --target examples/target_is_unittest/test_case.py

# Continuous testing and repeat 100 times
$ eztest --mode continuous --target examples/target_is_unittest/test_case.py --repeat 100 --nolog

# Simultaneous testing, start 50 threads and repeat 100 times
$ eztest --mode simultaneous --target examples/target_is_unittest/test_case.py --stress 50 --repeat 100 --nolog

# Concurrency testing, start 50 threads and run 1 hour
$ eztest --mode simultaneous --target examples/target_is_unittest/test_case.py --stress 50 --duration 60 --nolog

# Frequent testing, start 50 threads and run 1 hour
$ eztest --mode frequent --target examples/target_is_unittest/test_case.py --stress 50 --duration 60 --nolog

# Ignore cases
$ eztest --target examples/target_is_unittest/test_case.py --not-cases test_hello

# Target is a module with CASES defined.
$ eztest --target examples.target_is_module

# Send and save case report to remote server.
$ eztest --target examples.target_is_module --report-server localhost:8765

# Stop testing or report server
$ eztest stop

Report related examples:

# Start report server.
$ eztest start --port 8765

# Stop report server.
$ eztest stop

# Dump testing summary from remote report server
$ eztest dump --report-server localhost:8765

# Calculate failure rate and average of time taken for report files.
$ eztest --calc "/tmp/a.csv" "/tmp/b.csv" --group-minutes 30

# Calculate failure rate and average of time taken for files under report folder.
$ eztest --calc "/tmp/reports" --group-minutes 30

Prerequisites

• C Python 2.7, 3.2 and higher.

• psutil https://pypi.org/project/psutil/

Authors

lgt

License

GNU GPL v2, see http://www.gnu.org/licenses/gpl-2.0.html