DLMentor

Standalone Deep Learning homework grading package with local-first analysis and deterministic rule-based scoring.

Why DLMentor

DLMentor is designed as its own product/package identity for grading DL submissions safely:

• static inspection only (no execution of untrusted student code)
• safe ZIP intake with limits and traversal protection
• normalized metadata output for automation
• structured rule-based scoring (0-100)
• optional Gemini feedback refinement that does not change core scoring

Installation

pip install DLMentor

Local development install:

pip install -e .

Quick Start

from DLMentor import DLMentor

mentor = DLMentor(api_key=None)
result = mentor.evaluate_dl(answer_path="path/to/submission.zip")

print(result["score"])
print(result["feedback"])
print(result["checks"])

Lesson-context usage:

from DLMentor import DLMentor

mentor = DLMentor(api_key=None)
result = mentor.evaluate_dl_lesson(
    lesson_path="qa_fixtures/dl/lesson-01/submissions/correct_notebook.ipynb",
    questions=open("qa_fixtures/dl/lesson-01/questions.md", "r", encoding="utf-8").read(),
)
print(result["score"])

DL Input Support

Supported submission formats:

• .zip
• .ipynb
• .py
• .json
• .csv
• .txt
• folder paths containing mixed project files

Recommended submission structure:

submission/
  notebook.ipynb or train.py
  README.md
  requirements.txt (or environment.yml / pyproject.toml)
  metrics.json or metrics.csv
  model.pt / model.keras / model.onnx

DL Grading Workflow

1. Validate input path/file type
2. Safely extract ZIPs with hard limits
3. Parse notebooks/scripts/metrics locally
4. Build normalized metadata schema
5. Run rule-based checks:
   • valid structure
   • model definition
   • training logic
   • dataset loading
   • evaluation/metrics
   • reproducibility artifacts
6. Optionally refine feedback with Gemini

What DLMentor Does Not Do

• Does not execute notebooks/scripts
• Does not run model training/inference jobs
• Does not install/run student dependencies
• Does not replace rule-based score with AI output

Offline DL Testing Suite

Regenerate deterministic fixtures:

python tools/generate_dl_fixtures.py

Run all offline DL tests:

pytest -m dl_offline qa_tests -q

Run fixture-only suite:

pytest qa_tests/test_dl_fixtures.py -q

Run local smoke test:

python tools/dl_smoke_test.py

Build & Publish Prep

Build distribution artifacts:

python -m build

Validate install flows:

pip install .
pip install -e .

License

MIT