mcautograder 0.0.5

Small multiple choice question autograding library

Multiple Choice Autograder

This repository contains a small Python-based multiple-choice question autograder inteded for use in Jupyter Notebooks. It is meant to be packaged with each assignment so that they are easier for use on third-party servers, e.g. MyBinder.

Installation

You can install mcautograder using pip.

pip install mcautograder

Usage

To use the autograder, import the mcautograder package and make sure to your tests file with your notebook. When you load the notebook dependencies, import the file and initialize the grader by creating an instance of the Notebook class (the argument to pass is the path to your tests file):

import mcautograder
grader = mcautograder.Notebook("tests.py")

If you want the autograder to score the questions, make sure to set scored=True in your Notebook call. The default behavior of the autograder is to allow students to submit answers until they get the correct one. If you want to change this behavior, you must set the max_retakes argument to an integer, the maximum number of retakes allowed. If this is the case, when students hit that ceiling, the check cells will throw an AssertionError because they've hit the retake ceiling.

An example call for a scored notebook with a retake ceiling of 5 is given below.

grader = Notebook("tests.py", scored=True, max_retakes=5)

To use the autograder to check answers, have students assign their answers to variables in the notebook; these answers can strings of length 1 or single-digit integers. Then call the Notebook.check() function; the first argument should be the question identifier in your tests file and the second should be the variable the student created.

my_answer = "A"
grader.check("q1", my_answer)

If the student's response matches the test file, then Correct. will be printed; otherwise, Try again. will be printed. If the student enters an invalid response (e.g. float, answer of > 1 character, hit retake ceiling), the grader will throw an AssertionError with a descriptive message.

To get the score on a scored autograder, simply call Notebook.score():

grader.score()

The output will contain the fraction of earned points out of possible points and the percentage.

For a more descriptive introduction to the autograder, launch our Binder.

Tests

The autograder relies on a tests file to get the answers for the questions. In this repo, the file is tests.py and it is public; in practice, I usually distribute the answers as a hidden file, .tests.py. It is unhidden here so that you can peruse its structure and contents.

In the file, we define a variable answers which is a list containing dictionaries, each of which represents a single question. Each dictionary should contain 3 keys: "identifier", "answer", and, optionally, "points". If your assignment is unscored, you can leave off the "points" key. A description of the keys' values is given below:

Key	Value Type	Value Description
"identifier"	str	a unique question identifier
"answer"	str, int	the answer to the question; specifications below
"points"	int	optional; the number of points assigned to that question

Answers must be of length 1 (i.e. a single-character string or a single-digit integer). The autograder is currently set up to throw an AssertionError if an answer of length > 1 is submitted, although we do intend to add this functionality later.

An example of a file is given below.

answers = [
	{
		"identifier": "q1",
		"answer": 3,
		"points": 1,
	}, {
		"identifier": "q2",
		"answer": 2,
		"points": 2,
	}, {
		"identifier": "q3",
		"answer": "D",
		"points": 3,
	}
]

The identifiers have no set format. This is because the identifier is passed to Notebook.check() when you call it in the notebook.

Branches

The master branch contains the current state of mcautograder as it is hosted on PyPI. The dev branch contains the next version of mcautograder in development. Do not commit directly to the master branch. Make commits in the dev branch and then PR to the master branch before uploading to PyPI.

Changelog

v0.0.4:

• Moved utils to separate file for documentation

v0.0.3:

• Changed structure of tests file to be more intuitive
• Added docstrings and better documentation