clatter 0.0.3

A Doctest-stype Command Line Application Tester

clatter is a doctest-style testing tool for command-line applications. It wraps other testing suites and allows them to be tested in docstrings.

• Free software: MIT license

• Documentation: https://clatter.readthedocs.io.

Features

• Bring testing best practices to your command line apps

• Extensible - subclassing CommandValidator is trivial using any cli testing suite

Downsides

• Security. CLI commands are dangerous, and we make no attempt to protect you. Use at your own risk.

Usage

Test command line utilities and applications by whitelisting them with app-specific testing engines:

>>> teststr = r'''
...
... .. code-block:: bash
...
...     $ echo 'Pining for the fjords'
...     Pining for the fjords
...     <BLANKLINE>
... '''
>>>
>>> tester = Runner()
>>> tester.call_engines['echo'] = SubprocessValidator()
>>> tester.validate(teststr)

Click applications

Integrate your command line app:

>>> @click.command()
... @click.argument('name')
... def hello(name):
...     click.echo('Hello %s!' % name)

This can now be tested in docstrings:

>>> teststr = '''
...
... .. code-block:: bash
...
...     $ hello Polly
...     Hello Polly!
...     <BLANKLINE>
...
...     $ hello Polly Parrot
...     Usage: hello [OPTIONS] NAME
...     <BLANKLINE>
...     Error: Got unexpected extra argument (Parrot)
...     <BLANKLINE>
...
...     $ hello 'Polly Parrot' # clitest: +NORMALIZE_WHITESPACE
...     Hello Polly Parrot!
...
... '''

Click applications can be tested with a ClickValidator engine:

>>> tester = Runner()
>>> tester.call_engines['hello'] = ClickValidator(hello)

>>> tester.validate(teststr)

Mixed applications

Your app can be combined with other command-line utilities by adding multiple engines:

>>> teststr = r'''
...
... .. code-block:: bash
...
...     $ hello Polly
...     Hello Polly!
...     <BLANKLINE>
...
...     $ echo 'Pining for the fjords'
...     Pining for the fjords
...     <BLANKLINE>
...
... Pipes/redirects don't work, so we can't redirect this value into a file.
... But we can write a file with python:
...
... .. code-block:: bash
...
...     $ python -c \
...     >     "with open('tmp.txt', 'w+') as f: f.write('Pushing up daisies')"
...
...     $ cat tmp.txt
...     Pushing up daisies
...
... '''

>>> tester.call_engines['echo'] = SubprocessValidator()
>>> tester.call_engines['python'] = SubprocessValidator()
>>> tester.call_engines['cat'] = SubprocessValidator()

>>> tester.validate(teststr)

Suppressing commands

Commands can be skipped altogether with a SkipValidator:

>>> skipstr = '''
... .. code-block:: bash
...
...     $ aws storage buckets list
...
... '''

>>> tester = Runner()
>>> tester.call_engines['aws'] = SkipValidator()

Illegal commands

Errors are raised when using an application you haven’t whitelisted:

>>> badstr = '''
...
... The following block of code should cause an error:
...
... .. code-block:: bash
...
...     $ rm tmp.txt
...
... '''

>>> tester.validate(badstr)
Traceback (most recent call last):
...
ValueError: Command "rm" not allowed. Add command caller to call_engines to whitelist.

Unrecognized commands will raise an error, even if +SKIP is specified

>>> noskip = '''
... .. code-block:: bash
...
...     $ nmake all # clitest: +SKIP
...
... '''
>>> tester.validate(badstr)
Traceback (most recent call last):
...
ValueError: Command "nmake" not allowed. Add command caller to call_engines to whitelist.

Error handling

Lines failing to match the command’s output will raise an error

>>> teststr = r'''
... .. code-block:: bash
...
...     $ echo "There, it moved!"
...     "No it didn't!"
...     <BLANKLINE>
...
... '''

>>> tester = Runner()
>>> tester.call_engines['echo'] = SubprocessValidator()

>>> tester.validate(teststr)
Traceback (most recent call last):
...
ValueError: Clatter test failed. There, it moved! != No it didn't!

+ There, it moved!

- No it didn't!

Installation

pip install clatter

Requirements

• pytest

Todo

See issues to see and add to our todos.