grAIder

AI-powered coursework provisioning and grading for GitLab.

grAIder is a command-line tool that manages the full lifecycle of a programming course on GitLab: it provisions per-group repositories from starter templates, runs unified quality/test/coverage grading, drafts staggered AI reviews against a rubric, generates oral-exam (viva) questions grounded in each student's own project, and consolidates everything into per-group reports and a class CSV.

Students use the same tool locally to self-assess before they submit.

Status: 0.1.0 · Python 3.13+ · Full documentation →

Highlights

• Automated GitLab provisioning — create a private repo per group, push a starter template, protect main, and invite members from a CSV/XLSX roster, all from one command (with --dry-run to preview).
• Unified grading — run code-quality checks, the test suite, and coverage in one pass. graider grade --repo . for a student; --workspace <dir> grades a whole class.
• Staggered AI review — grade incrementally against milestone criteria (--up-to) using Claude, OpenAI, Gemini, or GLM. Reviews are drafted locally, then a teacher approves and publishes them to a Merge Request or Issue.
• Interview / viva questions — generate oral-exam questions grounded in a student's own code, each with the key points a correct answer must cover and red flags that betray a shaky understanding.
• Calibration — anchor the model to teacher-graded exemplars and measure agreement so AI verdicts track your grading.
• Cost & caching — per-run token-usage and cost estimates; a content-hash cache skips unchanged repositories on re-runs.
• Six starter languages — Python, Java, C++, Go, Rust, and TypeScript.
• Integrity signals — neutral commit-history metrics (cadence, contribution split, largest commit) and prompt-injection detection surfaced for triage — never used as an automatic penalty.

Installation

grAIder is a Python 3.13+ package managed with uv.

# From source (development)
git clone https://github.com/peaktwilight/grAIder.git
cd grAIder
uv sync                       # install with dev tools
uv run graider --help

# As a tool
uv tool install graider                 # core (Anthropic + Claude Code backends)
uv tool install "graider[openai]"       # add OpenAI / GLM backends
uv tool install "graider[google]"       # add Google Gemini backend

Quickstart

Teachers

# 1. Scaffold a course config (graider.toml) in the current directory
graider init --org my-course-group --template python --course swe-2026

# 2. Draft grading criteria from a syllabus (AI-assisted), then validate
graider criteria init --syllabus syllabus.pdf --out criteria/
graider criteria check criteria/

# 3. Provision GitLab repos for every group in the roster (preview first)
graider setup --roster roster.xlsx --dry-run
graider setup --roster roster.xlsx

# 4. Grade every provisioned repo, then draft AI reviews up to a milestone
graider grade --workspace ./submissions
graider review --workspace ./submissions --criteria-dir criteria/ --up-to m2

# 5. Approve and post the reviews to GitLab, then build reports
graider review publish --feedback mr
graider report --workspace ./submissions --out-dir reports/

Students

Your provisioned repo contains a .graider.yml. From its root:

graider grade            # quality + tests + coverage, exactly like the teacher runs
graider review           # local AI self-review against the released criteria
graider skills install   # optional: let the Claude Code CLI drive grAIder for you

Commands

Run graider <command> --help for the full flag set. Global options apply before any subcommand: --gitlab-url (GITLAB_URL), --token (GITLAB_TOKEN), --config, --class, and --dry-run.

AI backends

The AI commands (review, interview, criteria init, calibrate) run through a shared model abstraction, selected with --backend:

openai, glm, and gemini are text-only (no PDF syllabi) and require the graider[openai] or graider[google] extra. The default model is claude-opus-4-8; override per run with --model.

Starter templates

python · java · cpp · go · rust · typescript. Each ships a working project plus a rendered .graider.yml, .gitlab-ci.yml, qlty.toml, and a reflection prompt. Templates are stored as .tmpl / dot_ files and rendered with placeholder substitution ({{project_name}}, {{course}}, …).

Configuration

Settings are resolved in precedence order:

1. CLI flags (--gitlab-url, --token, --class, --dry-run, …)
2. Environment variables (GITLAB_URL, GITLAB_TOKEN, provider API keys)
3. Local graider.toml in the working directory (supports [class.<name>] sections)
4. Global ~/.config/graider/config.toml

Rosters are read from .csv, .xlsx, and .xlsm files with fuzzy header matching (email / group / name), per-row validation, and duplicate detection.

Documentation

Full guides live in docs/ and are published to GitHub Pages:

• Teachers Manual — setup → criteria → provisioning → grading → review → reporting.
• Students Manual — local grading, self-review, and Agent Skill setup.
• Current Implementation — technical reference for commands, config, and integrations.
• Learning Science & Topic Guides — the pedagogy behind the rubric.

Development

Managed with uv, linted/formatted with ruff, and type-checked with ty.

uv sync                     # install dependencies (incl. dev tools)
uv run graider              # run the CLI
uv run pytest               # run tests
uv run ruff check .         # lint
uv run ruff format .        # format
uv run ty check             # type check
uv run mkdocs serve         # preview the docs site locally

CI/CD

GitHub Actions runs on pushes to main, pull requests, and version tags:

• CI (ci.yml) — ruff lint, ruff format check, ty, and pytest.
• Docs (docs.yml) — builds the MkDocs site and deploys it to GitHub Pages.
• Publish (publish.yml) — on vX.Y.Z tags, builds and publishes to PyPI via Trusted Publishing (OIDC, no stored token).

Note: the .gitlab-ci.yml shipped inside the starter templates is separate — it runs in each student's provisioned GitLab repository, not for this tool.