2 projects
grader-variance
Inspect extension: isolate and decompose LLM-grader (judge) variance in a benchmark score, and give a stopping rule for how many grader repeats are needed.
rater-agreement
Chance-corrected agreement statistics for any raters, human or model — kappa, weighted kappa, Fleiss, Krippendorff's alpha, ICC, intra-rater reliability, kappa-paradox diagnostics, with bootstrap CIs.
