Skip to main content

AI Essay Evaluator

CI Status Documentation Status Test coverage percentage

uv Ruff pre-commit

PyPI Version Supported Python versions License


Documentation: https://ai-essay-evaluator.readthedocs.io

Source Code: https://github.com/markm-io/ai-essay-evaluator


A comprehensive Python framework for automated essay evaluation using OpenAI's GPT models. This tool enables educators to grade student essays at scale with customizable scoring rubrics, fine-tune models with their own grading data, and generate detailed feedback across multiple scoring dimensions.

Features

  • Automated Essay Grading - Evaluate student essays using fine-tuned OpenAI GPT-4o-mini models
  • Multiple Scoring Formats - Choose from extended (multi-dimensional), item-specific, or short scoring formats
  • Custom Model Training - Generate training datasets and fine-tune models with your own grading examples
  • Project Folder Mode - Simple folder structure for organizing essays, rubrics, and prompts
  • Cost Tracking - Built-in token usage and cost analysis for OpenAI API calls
  • Batch Processing - Grade hundreds of essays with progress tracking and async processing
  • Multi-Pass Grading - Run multiple grading passes for consistency checking
  • Rate Limit Handling - Automatic retry logic and adaptive rate limiting
  • Comprehensive Logging - Async logging for debugging and auditing

Quick Start

Installation

Install via pip:

pip install ai-essay-evaluator

Or using uv (recommended for development):

uv pip install ai-essay-evaluator

Basic Usage

  1. Set up your project folder:
my_project/
├── input.csv              # Student responses
├── question.txt           # Essay prompt
├── story/                 # Story files
│   └── story1.txt
└── rubric/                # Rubric files
    └── rubric1.txt
  1. Run the evaluator:
python -m ai_essay_evaluator evaluator grader \
  --project-folder ./my_project \
  --scoring-format extended \
  --api-key YOUR_OPENAI_API_KEY
  1. Check results in my_project/output/

Training Your Own Model

# Generate training data from graded examples
python -m ai_essay_evaluator trainer generate \
  --story-folder ./training/story \
  --question ./training/question.txt \
  --rubric ./training/rubric.txt \
  --csv ./training/graded_samples.csv \
  --output training.jsonl \
  --scoring-format extended

# Validate and fine-tune
python -m ai_essay_evaluator trainer validate --file training.jsonl
python -m ai_essay_evaluator trainer fine-tune \
  --file training.jsonl \
  --scoring-format extended \
  --api-key YOUR_OPENAI_API_KEY

For detailed documentation, visit the full usage guide.

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Mark Moreno
Mark Moreno

💻 🤔 📖

This project follows the all-contributors specification. Contributions of any kind welcome!

Credits

Copier

This package was created with Copier and the browniebroke/pypackage-template project template.

Metadata

Release files for ai-essay-evaluator 1.3.10

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for ai-essay-evaluator 1.3.10
File Size Uploaded Provenance Actions
ai_essay_evaluator-1.3.10.tar.gz 41.8 kB With provenance Details for ai_essay_evaluator-1.3.10.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for ai-essay-evaluator 1.3.10
File Interpreter ABI Platform Actions
ai_essay_evaluator-1.3.10-py3-none-any.whl Python 3 none any Details for ai_essay_evaluator-1.3.10-py3-none-any.whl

Total release size: 70.6 kB

Release files / ai_essay_evaluator-1.3.10.tar.gz

Download URL ai_essay_evaluator-1.3.10.tar.gz
Size 41.8 kB
Tags Source
SHA-256 checksum
How to use checksums
276d370c4875d39530a3fd5509249d23cff2427646346c322f75d9b4450b2f4d
BLAKE2b-256 checksum
How to use checksums
817f2bc0a1c94c62acf2f36e2789d39e68c141d8f23ea716b932ea7857c91779
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 9, 2026.

Transparency log

Release files / ai_essay_evaluator-1.3.10-py3-none-any.whl

Download URL ai_essay_evaluator-1.3.10-py3-none-any.whl
Size 28.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
d3978d24de50683a71547f8a7ce6d8b0a60f4de89332ce8b1fae59eefebb919f
BLAKE2b-256 checksum
How to use checksums
7a32a5152231983adc0dd2f3156fe4a7f16c13eb234a91be6d7829bac9471f90
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 9, 2026.

Transparency log

Security for ai-essay-evaluator 1.3.10

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for ai_essay_evaluator-1.3.10.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 9, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page