Tool for black-box testing command-line programs using STDIN, STDOUT and STDERR, with multiple CPUs
Project description
This is a tool for black-box testing command-line programs simply based on STDIN, STDOUT, and STDERR, with multiple CPUs. A sempahore keeps track of which CPUs are in use; and thus, tests are run in parallel on all CPUs, and new test runs are dispatched as tests complete (or fail). test_cmd waits until all tests have completed (either successfully or failed), and prints failed tests’ STDERR output, as they fail, in real time.
Tutorial
Test cases usually consist of pairs of input and output files, as well as an optional tests.json file specifying applicable command-line arguments. The input file is piped in via STDIN. If the command being tested emits the expected output file via STDOUT, the test case passes. A file representing an expected STDERR output can also optionally be specified.
The input/output file pairs must follow this naming pattern:
test-A.in.txt -> test-A.out.txt test-B.in.txt -> test-B.out.txt, test-B.err.txt test-C.in.txt -> test-C.out.txt
The file extension (.txt here) can be anything. The file naming pattern is *.in* for input files, *.out* for expected output files, and *.err* for expected error files. The content of the *.in.* file is piped to the command being tested, and its STDOUT is compared against the *.out* file. If a *.err.* file has been provided, then the command STDERR is matched against it as well.
For an example of test_cmd in action, see the pypage project, particularly its tests folder.
Usage
usage: test_cmd.py [-h] [-b] [-d] [-u] [-t] tests_dir cmd ... Functional Testing Utility for Command-Line Applications positional arguments: tests_dir Path to the directory containing test cases cmd Path to the command to be tested args The command-line arguments with an ampersand character '@' markingwhere arguments from test.json should be injected options: -h, --help show this help message and exit -b, --bw black & white output -d, --diff diff output -u, --to-unix convert CR+LF to LF in cmd output and test files -t, --rtrim ignore trailing whitespaces at the end of each line as well as trailing newlines
Command-line arguments for test cases can be specified by creating a special file named tests.json, and placing it in the directory containing your test cases. This tests.json file maps test cases to objects representing command-line arguments for that test case. If a command-line argument is a non-string value (e.g. a complex JavaScript object), the argument is stringified (with Python’s json.dumps), and passed in as JSON.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file test_cmd-2.0.6.tar.gz
.
File metadata
- Download URL: test_cmd-2.0.6.tar.gz
- Upload date:
- Size: 9.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/4.0.2 CPython/3.11.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 04861d7a9e9defd8d2021abe6d96960b459cd105360f711bfb1362e2a8a5226c |
|
MD5 | 15630bca248431e643ec9698a0df075f |
|
BLAKE2b-256 | aa7acefe5c04af32bd8b2f64fa5353b8ade65076b3262a9c82cd8d4a51c7697d |
File details
Details for the file test_cmd-2.0.6-py3-none-any.whl
.
File metadata
- Download URL: test_cmd-2.0.6-py3-none-any.whl
- Upload date:
- Size: 10.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/4.0.2 CPython/3.11.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0c0615e53de9b18b5d926966cbf386194d70cf6ef86216166537b1ebfa10e2c0 |
|
MD5 | 20b1b922541a3818f9098ef01eced881 |
|
BLAKE2b-256 | 274cc643cfdd744a742d47a7e9b17f917a44a31c751adfa7a03ba974133617a6 |