Python autograder to facilitate code output and static code checking.
Project description
pygradethis
A Python package to facilitate checking code output or static code checking
using AST analysis. It can either be used with R using the learnr
package, as
a mirror of gradethis
package, or as a standalone package for general Python
use in educational settings.
Note: This package is in early development and will undergo rapid changes.
Dependencies
- Python 3.6
- asttokens *
*We rely on asttokens for getting the text back from a source node (if possible) for static AST checking (more on that below). Python 3.8 has an improved ast module which can do away with this dependency in the future.
Features
- Simple output checking based on pass / fail conditions with feedback
- Simple static code checking (AST), with feedback on how student's code differs from solution
Since this package can't be installed via pip
yet, please place package in the directory you want to use it. To go
through how to use this as a module, refer to use_grader.py file in the root directory. Below, we
go over its output / AST checking functions.
Output checks
pygradethis
mimics the cadence to gradethis::grade_result
. For e.g., we can
check that the student supplies the mpg
dataset like so:
python_grade_result(
python_pass_if(mpg, "You also got the mpg dataframe!"),
python_fail_if(None, "")
)
Internally, these python_pass_if(output, message)
or python_fail_if(output, message)
will be checked sequentially in
the order of arguments and return on first condition we match. The None
here can be used
if you simply want to execute a condition if none of the other conditions matched.
If we match a python_pass_if
or python_fail_if
, we will present a feedback message wrapped in a convenient dict
:
dict(
message = str,
correct = True|False,
type = "auto|success|info|warning|error|custom",
location = "append|prepend|replace"
)
The message
is the feedback, the correct
is whether or not the student's solution is correct, type
is the type of feedback. When
used with learnr
the location
field here is useful for where the message is situated in the tutorial. However, for those using
this package as a standalone the location
is not an important field and it can be ignored. More on the flags here.
Internally, a random praise/encouragement message will be appended before any custom message supplied.
python_pass_if(mpg, "You also got the mpg dataframe!")
Feedback:
Bravo! You also got the mpg dataframe!
python_fail_if(None, "")
Feedback:
Try it again. You get better each time.
Code checks
For static code checking, we follow a similar cadence for gradethis::grade_code
.
When there is a solution code being supplied, grade_code(user_code, solution_code)
can be used to check the AST of
the user and solution code, making sure to standardize function calls and producing a helpful message for the student
to diagnose their issue.
Example:
grade_code("2 + sqrt(log(2))", "2 + sqrt(log(1))")
Feedback:
I expected 1, but you wrote 2 in log(2) at line 1.
Note how the expression in which the problem occurs (log(2)
) is pointed out so that the
student is aware that the 2
within the log
is incorrect not the 2
on the left operand of
the addition.
Similarly, here the feedback points out that the 2 within the log
function is incorrect, similar to the
gradethis
example.
Call Standardization
pygradethis
also knows how to take user's function call code and map positional arguments
to proper parameter names and set defaults if not supplied. This is so that you don't penalize
a student's code just because they did not explicitly spell out positional argument names, or
write the default arguments out.
For e.g. suppose a student is calling the following silly function foo
:
def foo(a, b=1):
pass
Grading the code with
grade_code(
student_code="foo(1)",
solution_code="foo(1)"
)
means that foo(1)
turns into foo(a=1, b=1)
before grading. In the example above,
the grade_code
doesn't give us a feedback message since they are equivalent expressions.
However, if the student supplies foo(2)
grade_code(
student_code="foo(2)",
solution_code="foo(1)"
)
we get back this feedback:
I expected foo(a=1, b=1), but what you wrote was interpreted as foo(a=2, b=1) at line 1.
Note how the arguments are filled in according to formal parameters, a
and b
, which can
help students be aware of how Python actually interprets their function calls.
Note: For call standardizing to work, the function definitions corresponding to function
calls must be defined and 'live' in the environment, whether that is the globals()
/locals()
,
builtins
, or custom module imports pandas
. Currently, common modules like math
is imported
for grading within grade_code.py
, but more modules will be included to serve data science
grading as well, such as pandas
or numpy
in the future. We plan to make the code more
extensible for the instructor to add them as dependencies.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pygradethis-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 40b3f995500aeaf313e957573232f30a7fd6f82b308df6a7fe5cb4ef60f98898 |
|
MD5 | a85e645db32e325fd1c24be878f6a88b |
|
BLAKE2b-256 | 8a63d8cbf027d8c39416092f18d4d251a4ea2255c211e2d48a94b112ec868ae9 |