Worksheet Grading Toolkit

A collection of scripts aiming to simplify the grading process for exercise sheet grading. Supports Moodle grading worksheet import/export.

This package contains four utilities for exercise sheet grading:

• wg-import: Generates the required directory structure from a Moodle grading worksheet file
• wg-export: Writes the assigned grades to a Moodle grading worksheet file
• wg-grade: Used to grade submissions
• wg-split: Splits/merges exercise CSV files

The utilities wg-grade and wg-split can also be used without Moodle integration, but require a file structure as described in the Usage section.

A usage example and detailed guide for the worksheet-grading package can be found in the Worksheet Grading Sandbox repository and corresponding Wiki

Exercise sheet grading tool (wg-grade)

Installation

Supported platforms

Given a Python 3 installation with the required packages, the script is supported on the following platforms:

• Linux
• macOS
• Windows (WSL only, see next section)

WSL dependencies

The "Open PDF" functionality in WSL requires either wslview or xdg-open (package xdg-utils) to be installed. "Open PDF (Browser)" is not supported on Windows Systems.

Note that Windows without WSL is not supported due to dependency incompatibilities (notably readchar).

Installation using pip

To install, run:

pip install worksheet_grading

Configuration

Note: As of version 1.0.0, a configuration file corresponding to the format below is generated by wg-import

The exercise sheet that is to be graded can be configured with a JSON file with the following keys:

• sheetname (string, required): used to specify the sheet name (i.e. prefix of the used CSV files)
• exercises (dictionary, required): points of the exercises on the sheet as key-value pairs (values are arrays of sub-exercise points). Note that exercise keys must be strings (as they would otherwise violate JSON syntax) and sub-exercise points must be integer values. Keys are strings, values are arrays of integers.
• group-size (integer, optional): specifies the (maximum) number of students per group, is autodetected from the CSV files if not specified
• pdf-path (string, optional): specifies a custom path for the submission PDF files (default is pdf/)
• exercise-basename (string, optional): specifies a custom exercise basename (default is Exercise) to which the exercise number is appended (e.g. Exercise 4 -> Task 4 if exercise-basename=Task). This field is primarily intended to be used for translations.
• exercise-names (dictionary, optional): specifies custom exercise names for exercise numbers (keys and values are strings) (e.g. {"1": "A+B Problem", "2": "Depth-First-Search"}). If used, this does not have to be provided for all exercise numbers defined in exercises.

{
        "sheetname":"blatt2",
        "exercises": {
                "1": [1,1,1],
                "2": [2],
                "3": [2],
                "4": [1,6,1],
                "5": [2,1,1],
                "6": [2,2,2],
                "7": [2,5,5],
                "8": [7,3]
        },
        "group-size": 2
}

In the above example, the sheetname is blatt2 and exercise 4 consists of 3 sub-exercises with 1, 6 and 1 points, respectively.

Usage

Note: As of version 1.0.0, the CSV files below are automatically generated by wg-import

The argument -e for wg-grade specifies the range of exercises that should be graded as a comma-separated list of integers and integer ranges. For every exercise, there must be a corresponding CSV file named SHEETNAME_EXNUM.csv, where SHEETNAME is the sheet name set in the configuration section and EXNUM a two-digit decimal number for the exercise number (for example, if SHEETNAME=blatt1 and the file describes exercise 1, the file should be called blatt1_01.csv).

Using -s, a suffix for the CSV files can be used, such that only CSV files with this suffix are considered (i.e., -s _tutor1 takes only files suffixed with _tutor1, e.g. blatt1_01_tutor1.csv).

These CSV files are expected to have the following format:

"exercise","group","student_1","student_2","points","feedback"
1,"Group 1","Frank Zappa",NA,NA,""
1,"Group 2","Ruth Underwood","George Duke",3,"@1@0@correct
@2@0@correct
@3@-0.5@j should be private"

Note that quotes are only required if the column spans over multiple rows. CSV files should also contain the same "keys", i.e. student and group names. NA marks unassigned student_2 and points fields. The feedback field accepts a string matching the regular expression @([0-9]+)@(-?[0-9]*(\.[0-9]+)?)@(.*). The first number is the sub-exercise number, the second one the number of points that are deducted.

The script also supports opening PDF files for each group, these have to be contained in a folder pdf/ and must be named GROUPNAME.pdf, where GROUPNAME is the name of the group the PDF was submitted by (PDF's can be exported in this format from Moodle with the Assign Submission Download Plugin). For single-person groups, the PDF file should be named FULL NAME.pdf, where FULL NAME is the student's full name as defined in Moodle.

You can specify a custom configuration file path using the -c argument.

For accessibility and compatibility purposes, the colored terminal output can be turned off using the --no-color argument.

Since Version 1.3.0, the script supports general remarks, which adds the possibility to add general remarks for each group as well as to deduct points for errors not strictly belonging to a certain exercise. These remarks are saved to a file called SHEETNAME_remarks.csv which is structured the same way as the exercise CSV files with exercise number 0. The script will automatically detect whether this file is present. Initially, the remarks file can be generated using wg-import with the -r flag. This also works for already existing projects generated without -r in case general remarks should be added after the fact.

Saving and exporting

The script exports the same file type that can be read, using the Write to csv [w] feature in the main menu. Furthermore, after grading of every exercise and on exit with Ctrl+C (which is possible in any menu prompt) auto-saves will be created for common deductions and the currently assigned grades (files .SHEETNAME_autosave.csv and .sheetname_deductions.csv, respectively).

Limitations

There is a maximum of 15 Exercises per exercise sheet (such that they can be selected with a hex digit 1-F from the group view)

CSV split and Merge Script (wg-split)

The split script can be used to split the exercise CSV files into multiple files with given suffixes and merge them together again. This is intended as a supplement to the wg-grade script to be able to more easily split exercises upon several tutors.

For example, a file called blatt1_02.csv can be split into two files blatt1_02_tutor1.csv and blatt1_02_tutor2.csv using

$ wg-split blatt1_02.csv _tutor1 _tutor2

The resulting two files can be merged together to obtain the original file using

$ wg-split --merge blatt1_02.csv _tutor1 _tutor2

Files are split on a line-by-line basis, e.g. if there are 12 groups with subsequent numbering that should be split upon 3 tutors, groups 1,4,7,10 will be assigned to tutor 1, 2,5,8,11 to tutor 2, and 3,6,9,12 to tutor three.

Note: As of version 1.3.0 which supports a general remarks file, wg-split is not intended for splitting/merging general remarks files that contain the same groups as the intended behavior for this would deviate for the exercise CSV merging procedure. wg-split can however be used to divide the general remarks file into several groups, enabling tutors to add general remarks if they grade a subset of groups, not exercises.

Moodle import and export utilities (wg-import, wg-export)

wg-import takes a Moodle grading worksheet file as well as the number of exercises (-e) and the desired internal name of an exercise sheet (-s) and creates the necessary directory structure to use wg-grade. The generated config file (per default config.json) is supposed to be adapted to the exercise sheet structure according to the format in the Configuration section. The script also supports setting a start exercise offset with --start-exercise to start the exercise numbering at a different number than 1. Since Version 1.3.0, the script also supports generating a general remarks from a Moodle grading worksheet. This can be enabled with the -r flag and will enable wg-grade to add general remarks and deductions to a submission.

wg-export exports the comments and grades given to a Moodle grading worksheet, such that it can be uploaded and parsed by Moodle. The script allows specifying whether exercises with a single sub-exercise should be 'collapsed' in the output, meaning that the single sub-exercise is not listed explicitly to make the output more readable. This can be disabled with the --no-collapse flag.