Opengrader

Grade paper assignments faster, right on your own computer. Print an assignment, scan the stack, and grade it question by question, without sending student work to anyone else's server.

Quick links: Install · Run it · Demo · Tests

Team

Advisor: Dr. Carlos Rojas

Project Description

Grading paper assignments is slow, and the tools that speed it up (like Gradescope) are cloud services that are expensive, rigid, and require uploading student work to a third party. Opengrader is a local, offline-first grading toolkit: it turns a markdown answer key into a printable assignment, splits and reads the scanned submissions, and gives instructors and TAs a fast question-by-question rubric grader in the terminal or the browser. Opengrader does the grading and leaves the AI to you: hand the written answers to the AI assistant you already use (Claude, ChatGPT, Codex, …) in a redacted copy that carries no student names or IDs, or connect a local or cloud model.

Prototype Scope

Works end to end today

opengrader run chains steps 3–6 into one resumable command. The desktop app (opengrader classic) gives you every step as buttons.

Incomplete or rough

• Windows and Linux installers and password-store support have been written but not yet tested on real Windows/Linux machines. macOS is the tested platform.
• Multi-grader web app: Google sign-in and shared workspaces work, but haven't been tried by a real course staff yet.
• Cloud AI providers are opt-in and work, but several hardening items remain (for example, answer crops are not yet masked before a cloud send).
• Handwritten math recognition (math extra) is experimental.

Left for the Expo

See What's Next.

System Architecture

Opengrader is a pipeline of small command-line tools that pass plain files to each other (markdown and image crops), ending in a grading app. All the core logic lives in one Python package; the terminal app, the web app, and the CLI are thin front ends over it.

flowchart LR
    subgraph Prepare
        AK["Answer key<br/>(.md / .docx / .pdf)"] -->|create-answer-key| QA["qa.md"]
        QA -->|qa-file generate-template| TPL["Printable assignment PDF"]
    end

    subgraph Ingest
        SCAN["Scanned stack PDF"] -->|scan-check + submission-splitter| SPLIT["One PDF per student"]
        SPLIT -->|"submission-extractor<br/>(OCR + optional AI)"| SUB["student.submission.md<br/>+ answer crops"]
    end

    subgraph Grade
        SUB --> APP["Grading app<br/>Terminal (Textual) · Browser (React)"]
        QA --> APP
        QA -->|automated-grading| SUB
        APP -->|grades, comments| SUB
    end

    subgraph Return
        SUB --> OUT["Annotated PDFs · CSV gradebook · Canvas"]
    end

    TPL -. printed and handed out .-> SCAN

Two ways to store grades:

• Folder mode (default): no database. Grades are saved straight into each student's .submission.md file.
• Workspace mode: a local SQLite file and a localhost-only service, so the terminal and browser apps (and several graders) can share live grading.

Prerequisites

Hardware: any recent laptop works for grading. For local AI, 16 GB of RAM is recommended (Apple Silicon or a GPU makes it much faster).

uv run tui --check-graphics

Installation

Option A: Desktop app (easiest, no terminal knowledge needed)

1. Download or clone this repository (see step 2 below).
2. Double-click Install Opengrader (Mac).command (or Install Opengrader (Windows).bat) in the project folder.
3. An Opengrader icon appears on your Desktop. Double-click it. The first launch takes a minute or two while it installs uv, Python 3.12, and dependencies; later launches are instant.

See packaging/README.md for details and uninstall steps.

Option B: Developer setup (from a clean Mac)

1. Install Homebrew (skip if you already have it):

   /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
   

2. Install the system tools:

   brew install git python@3.12 uv tesseract
   brew install ollama node      # optional: local AI and the web app
   

3. Clone the repository:

   git clone https://github.com/SJSU-CMPE-195/group-project-group-14.git opengrader
   cd opengrader
   

4. Install the Python packages:

   uv sync --all-groups --extra pydantic-ai --extra photos
   

   Extra	Adds	Needed for
   pydantic-ai	Cloud AI clients (Anthropic, OpenAI, Google, …)	Cloud AI providers; part of the test suite
   photos	iPhone HEIC photo support	photo-import with HEIC files
   math	Handwritten-math recognition (installs PyTorch, large)	Reading handwritten math as LaTeX

5. Check your setup. doctor tells you what is missing and the exact command to fix it:

   uv run opengrader doctor
   

6. (Optional) Set up an AI. Opengrader does not set one up for you. Pick one:

   • Your AI assistant (recommended): nothing to install here. See Working with your AI assistant under Running the Prototype.
   • A local model, so nothing leaves the computer. Start Ollama yourself, then choose it in Tools → AI models & providers → Open AI setup (uv run opengrader ai-readme --local prints every step):

     ollama serve &                 # start the Ollama server
     ollama pull qwen3.5:9b         # handwriting reader
     ollama pull qwen2.5vl:7b       # grade suggestions
     

   • A cloud provider with an API key: Tools → AI models & providers → Manage providers, opt-in and consented.

7. (Optional) Build the web app (needed once):

   npm --prefix src/opengrader/web ci
   npm --prefix src/opengrader/web run build
   

8. (Optional) Install the opengrader command globally so you can type opengrader … from any folder instead of uv run opengrader …:

   sh packaging/install-command.sh                                          # macOS / Linux
   powershell -ExecutionPolicy Bypass -File packaging\install-command.ps1   # Windows
   

Grading with other accounts on one computer (SSH)

Environment variables

None are required for local use. Everything below is optional. Never commit real values.

Web app sign-in (only for shared, multi-grader hosting). Copy .env.example to ~/.opengrader/server.env and fill it in:

AI and app behavior:

Running the Prototype

Try it in 30 seconds with sample data

The samples/ folder has real scanned quizzes. Copy a sample first, since grading writes into the folder:

cp -R samples/cs146-quiz1 ~/Desktop/cs146-quiz1
uv run tui --qa-file ~/Desktop/cs146-quiz1/qa.md --submissions ~/Desktop/cs146-quiz1/student-submissions/

Press enter to open a question, 1–9 to pick a rubric option, j/k for the next/previous question, and ? for every shortcut.

The desktop app (all tools, as buttons)

uv run opengrader classic

This is what the desktop icon opens. Pick a class folder (with a scan and an answer key) on the Start grading card and press Launch Grading. If the folder hasn't been processed yet, Opengrader offers to check, split, and read the scans for you first.

One command: scan in, graded work out

uv run opengrader run path/to/assignment-folder/

Point it at a folder holding the scanned PDF and the answer key (and optionally a roster and a blank template). It checks the scan, splits it, reads every answer, auto-grades what it can, and opens the grading app. Inside the folder it builds student-pdfs/ (one PDF per student; what the split noticed about each one is kept in student-pdfs/scan-notes/) and submissions/. Run the same command again and it picks up where it left off. Add --dry-run to preview the plan, or --llm to let the AI chosen in AI setup suggest grades for written answers (start Ollama first if it is a local one). If that AI can't be reached, the run stops before auto-grading and asks whether to grade without it; in a script, add --grade-without-ai or --no-auto-grade to answer up front.

opengrader run path/to/assignment-folder/

1: H2O; Water
2) B
3a. 101
4.
Paris
City of Light

uv run opengrader agent answer-key --class-folder class [--source key.docx]   # the assistant writes or fills qa.md
uv run opengrader agent use-key --folder class-ai-answer-key --to class/qa.md
uv run opengrader agent prepare --submissions class/submissions --qa-file class/qa.md            # grading
uv run opengrader agent prepare --submissions class/submissions --qa-file class/qa.md --rubrics  # drafting rubrics

# the same in a workspace: name it and the assignment instead
uv run opengrader agent prepare --workspace course --assignment "Quiz 1"
uv run opengrader agent answer-key --workspace course --assignment "Quiz 1"

uv run opengrader agent apply --submissions class/submissions --incoming class-ai-assistant/<name> --yes --delete-copy
# a workspace (accept / use-key also take --key-file update|leave):
uv run opengrader agent apply --workspace course --assignment "Quiz 1" --incoming course-ai-assistant/<name> --yes

The web app

uv run opengrader web      # starts the local service and opens your browser
uv run opengrader stop     # stops the service

From the desktop app you can also tick Show Open in browser under Tools → Browser app and use the Open in browser button.

# 0. (Optional) Draft a QA answer key from an existing key
uv run opengrader create-answer-key --input answer-key.pdf --output qa.md

# 1. Generate the printable assignment
uv run opengrader qa-file generate-template --qa-file qa.md --output-pdf template.pdf

# 2. Check and split the scanned stack
uv run opengrader scan-check --input scans/all.pdf --pages-per-submission 6
uv run opengrader submission-splitter --input scans/all.pdf --output split/ --fix-rotation

# 3. Read each student's answers (add --llm-responses to transcribe handwriting with the AI in AI setup)
uv run opengrader submission-extractor --input split/ --output student-submissions/ --qa-file qa.md

# 4. Auto-grade what can be matched (add --llm for AI suggestions on written answers)
uv run opengrader automated-grading --solutions qa.md --input student-submissions/

# 5. Grade the rest
uv run opengrader tui --qa-file qa.md --submissions student-submissions/

# 6. Send grades to Canvas (preview first)
uv run opengrader canvas-export --submissions student-submissions/ --course "CS146" --assignment "Quiz 1" --dry-run

uv run opengrader --help              # commands grouped by workflow step
uv run opengrader commands --all      # every command, including the ones the desktop app covers
uv run opengrader help <command>      # one command's options

Testing

Opengrader has ~250 test files run with pytest, plus browser tests run with Playwright. Both run automatically on every push and pull request to main through GitHub Actions.

Run the unit and integration tests (about 4 minutes):

uv sync --all-groups --extra pydantic-ai --extra photos   # make sure the extras are installed
uv run pytest

Run just one area (much faster while developing):

uv run pytest tests/unit/core            # e.g. only the core library
uv run pytest tests/unit/core/test_curve.py

Run the browser tests (needs the web app built, see Installation step 7):

uv sync --group browser
uv run --group browser playwright install chromium
uv run --group browser pytest browser_tests --browser chromium

What the tests cover:

Tests use the real scanned quizzes in samples/ as fixtures.

Demo

Demo video: Coming soon. The link will be added here.

Screenshots

Coming soon. Screenshots of the generated assignment, the scan splitter, a student submission file, and the grading app will be added here.

Technical Stack

What's Next

Before the Project Expo on December 11, we plan to:

• Test on Windows and Linux: run the installers, launchers, and password-store support on real machines and fix what breaks.
• Pilot with real course staff: grade a real quiz with several TAs using the shared web workspace and Google sign-in, and fix the rough edges they hit.
• Close the remaining AI safety gaps: mask the name/ID area out of answer crops before any cloud send, and limit the size of model responses.
• Measure accuracy: report how often handwriting reading and AI grade suggestions agree with human graders (using llm-eval and transcribe-eval).
• Polish the Expo demo: a one-click demo folder, an updated demo video, and a poster.
• Choose and add a license. MIT; see below.

License

Opengrader is released under the MIT License. The third-party packages it uses keep their own licenses, listed in THIRD_PARTY_LICENSES.

Acknowledgments

• Dr. Carlos Rojas, Advisor, for guidance and support throughout the project
• Dr. Ben Reed, for sharing ideas and experiences, and providing valuable feedback

CMPE 195A/B Senior Design Project · San José State University · 2026