Test for script result.
Project description
Cli_results
Simple lib to test results or script runs from command line.
Install
Install from pypi:
pip install cli_result
Or install from github repo:
pip install git+https://github.com/ayasyrev/cli_result.git
Usage.
Main purpose test results of examples run. We run all scripts in examples folder and compare results with expected results. Check it at different python versions.
So we can be sure that all scripts work and has similar behaviors in different python versions.
It's not suitable to run script that supposed to run for a long time or resources are limited.
But it's good for quick tests, to check configs and shorts demos (examples).
Put your script in examples folder and expected results in results folder.
Arguments for tests at file name same as script name + __args.txt.
from cli_result import check_examples, Cfg
result = check_examples()
This run all scripts in examples folder with arguments from __args.txt
file and compare with results at results/
folder.
assert result is None
Examples
We can change examples folder.
cfg = Cfg(examples_path="../examples/examples_extra/")
Check examples at folder:
from cli_result import get_examples
examples = get_examples(cfg=cfg)
We got list of examples as tuple example_name, files
example = examples[0]
# name
print(example[0])
# files
print(example[1][0])
print(example[1][1])
output
example_extra_1 ../examples/examples_extra/example_extra_1.py ../examples/examples_extra/example_extra_1__alter.py</pre>
Run script
We can run script and look at result.
from cli_result import run_script
res, err = run_script(
filename=example[1][0],
args="--help",
)
print(res)
output
usage: example_extra_1.py [-h] [--echo ECHO]options: -h, --help show this help message and exit --echo ECHO </pre>
assert err == ""
Check one example.
We can check one example.
from cli_result import run_check_example
result = run_check_example(
experiment_name=example[0],
file_list=example[1],
cfg=cfg,
)
assert result is None
Alternatively we can check one as:
result = check_examples(
names=example[0], # we can send list of names as [name1, name2, ...]
cfg=cfg,
)
assert result is None
Check all examples.
Or check all examples.
result = check_examples(cfg=cfg)
assert result is None
Check errors
Lets look at example with error.
cfg = Cfg(examples_path="../tests/examples/examples_errors/")
result = check_examples(cfg=cfg)
assert result is not None
print(f"Examples with errors: {len(result)}, {examples[0][0]}: {len(result[0][1])} errors")
output
Examples with errors: 1, example_extra_1: 6 errors</pre>
Let look at one of errors.
We got file name that has error, result of run and expected result. Now we can look for what is wrong.
result[0][1][0]
output
('help', [('../tests/examples/examples_errors/exmpl_1__err_1.py', ['usage: exmpl_1__err_1.py [-h] [--e E]\n\noptions:\n -h, --help show this help message and exit\n --e E\n', 'usage: exmpl_1.py [-h] [--echo ECHO]\n\noptions:\n -h, --help show this help message and exit\n --echo ECHO\n'])])
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.