Skip to main content

Local Gradescope autograder runner

Project description

gslocal

Run your Gradescope autograder locally against any submission, without uploading anything to Gradescope.

gslocal mirrors Gradescope's exact container workflow: it builds your autograder zip, spins up the same Docker base image Gradescope uses, mounts the submission, runs the grader, and prints a formatted score summary. Build and image caching mean repeat runs are fast.


Requirements

Installation

pip install gslocal

Quick Start

In your autograder project directory, run:

gslocal init

This walks you through creating a gslocal.toml config file. Then test a submission:

gslocal run path/to/submission.zip

How It Works

gslocal expects your autograder project to produce a Gradescope-compatible zip file via a build command. It is agnostic about your build tool - Maven, Make, a shell script, anything that outputs a zip works.

On each run, gslocal:

  1. Prepares the submission (zip, directory, or GitHub clone)
  2. Runs your build command if source files changed since the last build
  3. Generates a Dockerfile and builds a Docker image if the zip is fresh
  4. Runs the container with the submission mounted, waits for results
  5. Prints a formatted score summary

Build caching: gslocal hashes the files listed in your watch config. If nothing changed and the output zip exists, the build is skipped entirely. The Docker image is similarly only rebuilt when the zip changes.


Configuration

gslocal is configured via a gslocal.toml file in your autograder project root. Run gslocal init to generate one interactively.

[build]
cmd = "mvn clean package"       # command to produce the autograder zip
zip = "target/*.zip"            # glob pattern locating the output zip
watch = ["src/**/*", "pom.xml"] # glob patterns to watch for rebuild triggering

[docker]
setup = "src/main/resources/setup.sh"  # path to setup.sh inside the project (required)
# metadata = "src/main/resources/mock_submission_metadata.json"  # optional

[run]
timeout = 60     # autograder timeout in seconds
verbose = false  # show full build/docker output instead of spinners

Field reference:

Field Required Description
build.cmd yes Shell command to build the autograder zip
build.zip yes Glob pattern to locate the output zip
build.watch yes Glob patterns whose content is hashed to detect source changes
docker.setup yes Path to setup.sh relative to project root
docker.metadata no Path to mock submission metadata JSON; a bundled default is used if omitted
run.timeout no Container timeout in seconds (default: 60)
run.verbose no Show full build and Docker output (default: false)

gslocal locates gslocal.toml by walking up the directory tree from your current working directory, so you can run it from any subdirectory of your project.


Submission Types

gslocal accepts submissions in three formats:

# Zip file
gslocal run test-zips/solution.zip

# Local directory
gslocal run /path/to/student/project/

# GitHub URL (SSH or HTTPS)
gslocal run git@github.com:student/repo.git
gslocal run https://github.com/student/repo

GitHub submissions are cloned with --depth 1.


Commands

gslocal run <submission>

Run the autograder against a submission.

Flag Description
-r, --rebuild Force rebuild of build command and Docker image
-n, --no-build Skip build, use existing zip
-i, --interactive Drop into a shell inside the container
-c, --clean Delete temp files after the run
-t, --timeout SECS Override timeout (also: GSLOCAL_TIMEOUT env var)
-v, --verbose Show full build and Docker output

Timeout precedence: --timeout flag > GSLOCAL_TIMEOUT env var > gslocal.toml value > 60s default.

After each run, the following are available for inspection:

.gslocal/
  temp/
    submission/   # prepared submission files
    results/      # results.json
    autograder/   # extracted autograder zip contents

gslocal init

Generate a gslocal.toml in the current directory. Prompts for each field with example hints. Skipping a required field writes a placeholder value that gslocal will catch and refuse to run with.

Flag Description
--placeholders Write config with all sentinel values, no prompts

gslocal clean

Remove local gslocal state for the current project.

Flag Description
(none) Delete .gslocal/temp/ only
--image Also remove the Docker image for this project
--all Delete .gslocal/ entirely and remove Docker image (full reset)

Project State

gslocal stores all state in a .gslocal/ directory at the project root. Add it to your .gitignore (gslocal init offers to do this automatically):

.gslocal/

Adding gslocal to an Existing Autograder

  1. Install gslocal: pip install gslocal
  2. Run gslocal init in your autograder project root
  3. Fill in your build command, zip output path, and the files to watch for changes
  4. Add .gslocal/ to .gitignore
  5. Run gslocal run <submission> to test

Contributing

Contributions are welcome. To get started:

git clone https://github.com/naasanov/gslocal.git
cd gslocal
pip install -e .

The source lives in src/gslocal/. Key modules:

Module Responsibility
cli.py Argument parsing and subcommand dispatch
config.py Config loading, validation, project root resolution
commands/run.py Run orchestration
commands/init.py Interactive config generation
commands/clean.py State cleanup
build.py Build invocation and hash-based caching
submission.py Submission preparation (zip, dir, GitHub)
docker.py Dockerfile generation, image build, container execution
results.py Terminal formatting of results.json

Please open an issue before starting work on a large change.


Releasing

gslocal is released through the GitHub Actions workflow at .github/workflows/publish.yml.

Before cutting a release:

  1. Update version in pyproject.toml
  2. Add a matching section to CHANGELOG.md, for example ## [0.1.1] - 2026-05-02
  3. Merge the release-ready changes to main

Then, in GitHub:

  1. Open Actions
  2. Select Release and Publish
  3. Click Run workflow
  4. Enter the version number, such as 0.1.1

The workflow will:

  • verify the run is on main
  • verify the requested version matches pyproject.toml
  • verify CHANGELOG.md contains a section for that version
  • fail if that version is already published on PyPI
  • build the package and run twine check
  • create the v<version> tag if needed
  • create the GitHub Release from the matching changelog section if needed
  • publish the package to PyPI

If the GitHub tag or release is created but PyPI publishing fails, fix the issue and rerun the workflow for the same version. If PyPI already has that version, publish a new patch version instead.


License

MIT

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

gslocal-0.1.0.tar.gz (20.0 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

gslocal-0.1.0-py3-none-any.whl (21.3 kB view details)

Uploaded Python 3

File details

Details for the file gslocal-0.1.0.tar.gz.

File metadata

  • Download URL: gslocal-0.1.0.tar.gz
  • Upload date:
  • Size: 20.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for gslocal-0.1.0.tar.gz
Algorithm Hash digest
SHA256 ee02b15e709697e65a3ee7b906f6cc4f4e9ef95da0258406a08271777509a3fd
MD5 a3c938b783469eee3efe98888c9645ff
BLAKE2b-256 abc27c4aa57faa520ca2603966069015ce1162c3fa95bb1c89a33dd2dfe45493

See more details on using hashes here.

Provenance

The following attestation bundles were made for gslocal-0.1.0.tar.gz:

Publisher: publish.yml on naasanov/gslocal

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file gslocal-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: gslocal-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 21.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for gslocal-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 a916702757f7759d74359d578e3053cc31ef075981aa2805b03cc4742d7b9511
MD5 4b592263bfa8b1c18ab0116c941ac381
BLAKE2b-256 b48329b5bae6709451d9782942527ce525c9fdd91533ce9ca766561608d2f0df

See more details on using hashes here.

Provenance

The following attestation bundles were made for gslocal-0.1.0-py3-none-any.whl:

Publisher: publish.yml on naasanov/gslocal

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page