assignment-evaluator 1.0.0

AI-powered assignment grading tool — auto-generates rubrics and grades student PDF submissions using Google Gemini.

Assignment Evaluator

AI-powered assignment grading tool that automatically generates a solution manual & rubric from an assignment PDF, then grades every student submission using Google Gemini vision models. Results are exported to an annotated graded PDF per student and a summary Excel spreadsheet.

---

Features

• Two-step GUI workflow – pick the assignment PDF, review the auto-generated rubric, then point to the submissions folder and click Start Evaluation.
• Automatic question & mark detection – hybrid regex + Gemini vision extraction.
• Rubric caching – generated rubrics are cached per assignment so you skip re-generation on subsequent runs.
• Graded PDF output – a cover page with per-question scores and feedback is prepended to each student's original PDF.
• Incremental re-runs – already-evaluated submissions are skipped unless you choose to re-evaluate.
• Configurable strictness & feedback style – lenient / moderate / strict and brief / balanced / detailed.

---

Requirements

Requirement	Notes
Python ≥ 3.10	Standard distribution
Poppler	PDF-to-image conversion (see below)
Google Gemini API key	Free tier available at https://aistudio.google.com/

Poppler installation

Windows – download the pre-built binaries and unzip them. The installer already ships a Release-25.12.0-0 folder next to the package source. If you use a different location, set POPPLER_BIN_PATH:

$env:POPPLER_BIN_PATH = "C:\path\to\poppler\Library\bin"

macOS

brew install poppler

Linux (Debian / Ubuntu)

sudo apt-get install -y poppler-utils

On macOS and Linux, Poppler is usually found automatically on PATH — no extra configuration needed.

---

Installation

Option A — Standalone executable (Windows, no Python required)

Download AssignmentEvaluator.exe from the Releases page and run it directly — no Python installation needed.

Poppler is still required. Download the Windows binaries from github.com/oschwartz10612/poppler-windows, unzip them, and set the environment variable:

$env:POPPLER_BIN_PATH = "C:\path\to\poppler\Library\bin"

Option B — install from source (recommended for development)

# 1. Clone or copy the project folder
cd assignment-evaluator

# 2. Create and activate a virtual environment (optional but recommended)
python -m venv .venv
# Windows
.venv\Scripts\activate
# macOS / Linux
source .venv/bin/activate

# 3. Install the package and all dependencies
pip install -e .

Option C — install dependencies only (no package install)

pip install -r requirements.txt

Option D — install from PyPI (once published)

pip install assignment-evaluator

---

Configuration

Gemini API key

When you launch the app for the first time, a dialog will ask you to enter your Google Gemini API key.

How to get a free key:

1. Go to https://aistudio.google.com/app/apikey
2. Sign in with your Google account
3. Click Create API key
4. Copy the key (starts with AIza…) and paste it into the dialog

The dialog has a "Save key locally for next use" checkbox. When ticked, the key is stored in ~/.assignment_evaluator/api_key.json (permissions set to owner-only on macOS/Linux) so you won't be asked again on subsequent runs.

Security note: The saved file is local to your machine and never transmitted anywhere other than the Google Gemini API. Never commit it to version control.

You can still supply the key via environment variable if you prefer (the env var always takes priority and skips the dialog):

# Linux / macOS
export GEMINI_API_KEY="AIza..."

# Windows PowerShell
$env:GEMINI_API_KEY = "AIza..."

# Windows CMD
set GEMINI_API_KEY=AIza...

Optional environment variables

Variable	Default	Description
GEMINI_API_KEY	(GUI prompt)	Skips the key dialog when set
POPPLER_BIN_PATH	Auto-detected	Full path to the Poppler bin/ folder

---

Usage

Launch the GUI

# If installed as a package
assignment-evaluator

# Or run as a Python module
python -m assignment_evaluator

# Or run the script directly
python assignment_evaluator/app.py

Step-by-step walkthrough

1. Step 1 – Assignment PDF

   • Browse to and select the assignment question PDF.
   • Choose the Gemini model to use for generating the rubric/solution.
   • Click Generate Rubric.

2. Review the rubric

   • The app displays the auto-generated solution manual (left) and rubric (right).
   • You can Approve and Continue, Regenerate with a different model, or Cancel.

3. Step 2 – Evaluation setup

   • Set the submissions folder, output folder, subject name, question count, and per-question marks.
   • Adjust strictness, feedback style, and models as needed.
   • Click Start Evaluation.

4. Results

   • A *_graded.pdf is written for each student in the output folder.
   • An Assignment_Evaluation_Results.xlsx spreadsheet summarises all scores.
   • Full logs and a JSONL audit trail are written to evaluation_logs/.

---

Output structure

Graded_Assignments/
├── Submission_folder_A/
│   ├── student_name_graded.pdf   ← annotated PDF with cover page
│   └── ...
├── Assignment_Evaluation_Results.xlsx
evaluation_logs/
└── assignment_eval_YYYYMMDD_HHMMSS/
    ├── run.log
    ├── session_config.json
    ├── solution_manual.txt
    ├── rubric.json
    ├── assignment_summary.txt
    ├── evaluation_audit.jsonl    ← one JSON record per submission
    └── summary.json

---

Supported Gemini models

The dropdown in the GUI lists these models (you can also type any valid model name):

Model	Notes
gemini-2.5-pro	Highest accuracy; slower
gemini-2.5-flash	Good balance of speed and accuracy
gemini-2.0-flash	Default; fast and capable
gemini-1.5-pro	Stable, widely available
gemini-2.0-flash-lite	Fastest; lower accuracy

---

Programmatic API

import os
os.environ["GEMINI_API_KEY"] = "AIza..."

from assignment_evaluator.app import main
main()   # launches the full GUI workflow

For headless / server use you can call the lower-level functions directly:

from pathlib import Path
from assignment_evaluator.app import (
    _configure_genai,
    generate_assignment_materials,
    evaluate_single_submission,
    setup_logging,
)

_configure_genai()
logger, session_dir = setup_logging("my_run")

config = {
    "subject_name": "Physics 101",
    "strictness": "moderate",
    "feedback_style": "balanced",
    "max_marks": {"Q1": 5, "Q2": 5, "Q3": 10},
    "expect_handwritten": True,
    "evaluator_notes": "",
    "page_limit": 40,
    "dpi": 150,
    "timeout_seconds": 180,
    "retry_count": 3,
    "solution_model": "gemini-2.0-flash",
    "evaluation_model": "gemini-2.0-flash",
}

assignment_pdf = Path("assignment.pdf")
materials = generate_assignment_materials(config, assignment_pdf, session_dir, logger)

result = evaluate_single_submission(
    pdf_path=Path("submissions/student_a.pdf"),
    submissions_root=Path("submissions"),
    output_root=Path("graded"),
    config=config,
    assignment_materials=materials,
    logger=logger,
)
print(result["total_score"], result["grades"])

---

Development

# Install dev extras
pip install -e ".[dev]"

# Run tests
pytest

# Build a distribution wheel
python -m build

# Upload to PyPI
twine upload dist/*

---

License

MIT — see LICENSE.