A function-oriented testing framework for Python 3.
Project description
Zest
.
A function-oriented testing framework for Python 3.
Written by Zack Booth Simpson, 2020
Available as a pip package: pip install zbs.zest
Motivation
Python's default unittest module is a class-oriented approach that does not lend itself well to recursive setup and teardown.
Zest uses a recursive function-based approach best demonstrated with examples.
##########################################
# some_module.py
def _say_hello():
print("Hello")
def unit_under_test(a):
if a <= 0:
raise ValueError("a should be positive")
_say_hello()
return a + 1
##########################################
# zest_some_module.py
from zest import zest
import some_module
def zest_unit_under_test():
# This is a root-level zest because it starts with "zest_"
def it_raises_on_non_positive():
def it_raises_on_negative():
with zest.raises(ValueError):
some_module.unit_under_test(-1)
def it_raises_on_zero():
with zest.raises(ValueError):
some_module.unit_under_test(0)
zest() # Note this call which tells zest to run the above two tests
def it_calls_say_hello_once():
with zest.mock(some_module._say_hello) as m_say_hello:
some_module.unit_under_test(0)
assert m_say_hello.called_once()
zest() # Same here, this will cause it_raises_on_non_positive and it_calls_say_hello_once to run
The zest() function uses stack reflection to call each function that it finds in the caller's stack-frame. However, it only calls functions that do not start with an underscore.
Two special functions are reserved: _before() and _after() which are called before/after each test function in the scope.
For example, often you may want to set up some complex state.
def zest_my_test():
state = None
def _before():
nonlocal state
state = State(1, 2, 3)
def it_raises_if_bad():
with zest.raises(Exception):
unit_under_test(state)
def it_modifies_state_on_1():
unit_under_test(state, 1)
assert state.foo == 1
def it_modifies_state_on_2():
unit_under_test(state, 2)
assert state.foo == 2
Examples
See ./zests/zest_examples.py
for more examples.
Usage
Search recursively all directories for def zest_*() functions and execute them.
$ zest
Show progress
$ zest --verbose=0 # Show no progress
$ zest --verbose=1 # Show "dot" progress (default)
$ zest --verbose=2 # Show hierarchical full progress
Search only inside the specific dirs
$ zest --include_dirs=./abc:./def
Run only tests that are in the "integration" or "slow" groups
$ zest --groups=integration:slow
Run only tests that contain the string "foobar". This will also run any parent test needed to execute the match.
$ zest foobar
Disable test order shuffling which is on by default to increase the liklihood that accidental order-dependencies are manifest.
$ zest --disable_shuffle
Helpers
Expected exceptions
def zest_foobar_should_raise_on_no_arguments():
with zest.raises(ValueError):
foobar()
Sometimes you may wish to check a property of the trapped exception
def zest_foobar_should_raise_on_no_arguments():
with zest.raises(ValueError) as e:
foobar()
assert e.exception.args == ("bad juju",)
Often you may wish to check only for a string of a property of the trapped exception in which case you can use the in_* argument to the raises.
def zest_foobar_should_raise_on_no_arguments():
with zest.raises(ValueError, in_args="bad juju") as e:
foobar()
Mocks
import unit_under_test
def zest_foobar():
with zest.mock(unit_under_test.bar) as m_bar:
# Suppose unit_under_test.foobar() calls bar()
m_bar.returns(0)
unit_under_test.foobar()
assert m_bar.called_once_with(0)
See zest.MockFunction
for a complete MockFunction API.
Gotchas
Don't forget to put the zest() call at each level of the test. If you forget, the zest runner will throw an error along the lines of: "function did not terminate with a call to zest()..."
def zest_something():
def it_foos():
foo()
def it_bars():
bar()
# WRONG! zest() wasn't called here. Error will be thrown when the test is run.
Do not mock outside of test functions:
def zest_something():
with zest.mock(...):
def it_does_something():
assert something
def it_does_something_else():
assert something
# The zest() will execute outside of the above "with" statement so
# the two tests will not inherit the mock as expected.
zest()
Rather, put the zest() inside the "with mock":
def zest_something():
with zest.mock(...):
def it_does_something():
assert something
def it_does_something_else():
assert something
# This is fine because zest() was called INSIDE the with
zest()
Don't have more than one zest() call in the same scope.
def zest_something():
with zest.mock(...):
def it_does_something():
assert something
def it_does_something_else():
assert something
# Like above example; so far, so good, but watch out...
zest()
with zest.mock(...):
def it_does_yet_another_thing():
assert something
# WRONG! A second call to zest() will RE-EXECUTE the above two tests
# (it_does_something and it_does_something_else) because this
# second call to zest() doesn't know that it is inside of a with statement.
# The "with" scope makes it look different but really the following
# call to zest() and the call to zest above are actually in the same scope.
zest()
When asserting on properties of an expected exception, be sure to do assert outside the scope of the "with" as demonstrated:
Wrong:
with zest.raises(SomeException) as e:
something_that_raises()
assert e.exception.property == "something"
# The above "assert" will NOT be run because the exception thrown by
# something_that_raises() will be caught and never get to execute the assert!
Right:
with zest.raises(SomeException) as e:
something_that_raises()
assert e.exception.property == "something"
# (Note the reference to "e.exception." as opposed to "e."
Remember that the exception returned from a zest.raises() is
not of the type you are expecting but rather of a wrapper
class called TrappedException
. To get to the properties
of interest you need to use e.exception.*
.
Wrong:
with zest.raises(SomeException) as e:
something_that_raises()
assert e.property == "something"
# Wrong! e is of type TrappedException therefore the above will not work as expected.
Right:
with zest.raises(SomeException) as e:
something_that_raises()
assert e.exception.property == "something"
# Correct, .exception reference to get original exception from the `e` TrappedException wrapper.
Development
Run in development mode
pipenv shell
pipenv sync
# Run all the example tests (which actually test the tester itself).
$ ./zest.sh
Deploy
$ ./deploy.sh
You will need the user and password and credentials for Pypi.org
TODO
- Add --rng_seed option
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.