Skip to main content

Tool for black-box testing command-line programs using STDIN, STDOUT and STDERR, with multiple CPUs

Project description

This is a tool for black-box testing command-line programs simply based on STDIN, STDOUT, and STDERR, with multiple CPUs. A sempahore keeps track of which CPUs are in use; and thus, tests are run in parallel on all CPUs, and new test runs are dispatched as tests complete (or fail). test_cmd waits until all tests have completed (either successfully or failed), and prints failed tests’ STDERR output, as they fail, in real time.

Tutorial

Test cases usually consist of pairs of input and output files, as well as an optional tests.json file specifying applicable command-line arguments. The input file is piped in via STDIN. If the command being tested emits the expected output file via STDOUT, the test case passes. A file representing an expected STDERR output can also optionally be specified.

The input/output file pairs must follow this naming pattern:

test-A.in.txt  ->  test-A.out.txt
test-B.in.txt  ->  test-B.out.txt, test-B.err.txt
test-C.in.txt  ->  test-C.out.txt

The file extension (.txt here) can be anything. The file naming pattern is *.in* for input files, *.out* for expected output files, and *.err* for expected error files. The content of the *.in.* file is piped to the command being tested, and its STDOUT is compared against the *.out* file. If a *.err.* file has been provided, then the command STDERR is matched against it as well.

For an example of test_cmd in action, see the pypage project, particularly its tests folder.

Usage

usage: test_cmd.py [-h] [-b] [-d] [-u] [-t] tests_dir cmd ...

Functional Testing Utility for Command-Line Applications

positional arguments:
  tests_dir      Path to the directory containing test cases
  cmd            Path to the command to be tested
  args           The command-line arguments with an ampersand character '@' markingwhere arguments from test.json should be injected

options:
  -h, --help     show this help message and exit
  -b, --bw       black & white output
  -d, --diff     diff output
  -u, --to-unix  convert CR+LF to LF in cmd output and test files
  -t, --rtrim    ignore trailing whitespaces at the end of each line as well as trailing newlines

Command-line arguments for test cases can be specified by creating a special file named tests.json, and placing it in the directory containing your test cases. This tests.json file maps test cases to objects representing command-line arguments for that test case. If a command-line argument is a non-string value (e.g. a complex JavaScript object), the argument is stringified (with Python’s json.dumps), and passed in as JSON.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

test_cmd-2.0.6.tar.gz (9.5 kB view details)

Uploaded Source

Built Distribution

test_cmd-2.0.6-py3-none-any.whl (10.2 kB view details)

Uploaded Python 3

File details

Details for the file test_cmd-2.0.6.tar.gz.

File metadata

  • Download URL: test_cmd-2.0.6.tar.gz
  • Upload date:
  • Size: 9.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/4.0.2 CPython/3.11.5

File hashes

Hashes for test_cmd-2.0.6.tar.gz
Algorithm Hash digest
SHA256 04861d7a9e9defd8d2021abe6d96960b459cd105360f711bfb1362e2a8a5226c
MD5 15630bca248431e643ec9698a0df075f
BLAKE2b-256 aa7acefe5c04af32bd8b2f64fa5353b8ade65076b3262a9c82cd8d4a51c7697d

See more details on using hashes here.

File details

Details for the file test_cmd-2.0.6-py3-none-any.whl.

File metadata

  • Download URL: test_cmd-2.0.6-py3-none-any.whl
  • Upload date:
  • Size: 10.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/4.0.2 CPython/3.11.5

File hashes

Hashes for test_cmd-2.0.6-py3-none-any.whl
Algorithm Hash digest
SHA256 0c0615e53de9b18b5d926966cbf386194d70cf6ef86216166537b1ebfa10e2c0
MD5 20b1b922541a3818f9098ef01eced881
BLAKE2b-256 274cc643cfdd744a742d47a7e9b17f917a44a31c751adfa7a03ba974133617a6

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page