A basic collection of tools to aid in the process of grading Python assignments while also attempting to reduce cheating through various methods.
Project description
Grading Utils
A collection of utilities for grading Python assignments.
Utilities are based off some common unit testing patterns.
Overall, syntax is modelled off JavaScript's Jest syntax,
with some modifications (hence camelCase
, for example).
Design Philosophy:
- security
- simplicity
- immutability
Matchers and Modifiers
The most common operations with this module will involve matching test values to expected values.
All matchers begin with the expect(value)
syntax, and are chained by some matcher.
For example, if we want to test $f(x) = x^2$, we write:
from grading_utils.GradingUtils import expect
expect(f(2)).toEqual(4)
In this case, we could also write:
expect(f(2)).toBe(4)
which will check for referential equality amonst the two objects.
However, use caution when using .toBe()
to compare objects for equality,
as this will not use obj.__eq__
or obj.__hash__
to compare objects.
Matchers behave similarly to Python's assert
, but provide more context on error.
Generally, matchers accept any type for which they are logically defined.
For example, .toBeApprox(2.5, eps=0.1)
works for numeric types.
Matchers with Callable Objects
Typically, we want to check the output of a function,
so we call the function and pass its result to the expectation (expect(f(x))
).
Occasionally, matchers need to track the way functions behave.
For example, let avg(l: iter) = sum(iter) / len(iter)
.
If we want to check whether []
throws ZeroDivisionError
,
we can write:
expect(avg).toThrow(ZeroDivisionError, [])
Thus, we pass the function (uncalled) to our expect
call,
then we pass toThrow
's custom argument (the exception to except
),
and then all arguments to the function to the .toThrow
call.
This is an unfortunate limitation, but this works for the time being.
In the future, some decorator workaround may be implemented.
List of Matchers
The following matchers are currently defined:
toBe
toEqual
toBeApprox
toMatchType
toMatch
toContain
toBeGreaterThan
toBeLessThan
toThrow
Modifiers
Modifiers are used to modify the behavior of matchers, by modifying the underling "Expectation" object.
Most commonly, we may want to expect that a certain condition is not met.
We can chain modifiers for thus purpose.
Modifiers use ._modifier_name
syntax, with ()
if arguments are needed.
There are currently only 3 modifiers:
._not
This modifier negates the expectation. For $$f(x) = x^2$$:
expect(f(2))._not.toBeGreaterThan(2) # throws GradingError
._exception(e)
If we want to throw a custom exception when a matcher fails.
expect(f(2))._exception(UrAnAwfulProgrammerError).toBe(4)
._maxtime(s)
This is one of 3 existing options for timing.
The _maxtime
modifier should be used similarly to a matcher with callables.
To set a 2-second time limit on slow_reducer(some_set)
,
and expect a result of True
we write:
expect(slow_reducer)._maxtime(2, some_set).toBe(True)
For all timing semantics, see the dedicated timing section.
Looking Forward
_and
and _or
are some potential future modifiers.
Timing
One critical aspect of any timer implementation is that for long running functions, we want to keep grading time down; execution should be immediately stopped when the time limit is reached, with a fail error. This avoids functions from taking needlessly long times when they will fail anyways. All times are given in seconds.
Three methods are provided for time limiting functions:
with timer(5):
expect(slow_reducer(some_set)).toBe(True)
Or, when defining functions:
@time_limit(5)
def slow_reducer(some_set):
pass # implementation here
expect(slow_reducer(some_set)).toBe(True)
Naturally, this can be easily used to wrap student solutions.
Finally, the _maxtime
modifier can be used directly:
expect(slow_reducer)._maxtime(2, some_set).toBe(True)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.