incept-eval 1.0.1

CLI tool for evaluating educational questions via Incept API

Incept Eval

CLI tool for evaluating educational questions with configurable AI-powered modules.

Features

🎯 Multiple Evaluation Modules

• V3 Evaluation - Scaffolding and DI compliance
• Answer Verification - GPT-4 powered correctness checking
• EduBench Tasks - Educational quality benchmarks (QA, EC, IP, AG)

⚙️ Configurable

• Enable/disable modules per evaluation
• Custom module combinations
• Save configurations for reuse

🚀 Easy to Use

• Simple CLI interface
• JSON input/output
• Batch processing support

Installation

pip install incept-eval

Quick Start

1. Get API Key

Get your API key at https://incept.ai/api-keys

2. Configure

incept-eval configure YOUR_API_KEY

3. Evaluate

# Generate example
incept-eval example > questions.json

# Evaluate
incept-eval evaluate questions.json --pretty

Usage

Evaluate Questions

# Full evaluation (all modules)
incept-eval evaluate questions.json --output results.json --pretty

# Only V3 evaluation
incept-eval evaluate questions.json --no-answer-verify --edubench

# Only answer verification
incept-eval evaluate questions.json --no-v3 --edubench

# Custom modules
incept-eval evaluate questions.json --edubench QA --edubench EC

Quick Single Question

incept-eval quick-eval "What is 2+2?" "4" --grade 2 --pretty

Input Format

{
  "request": {
    "grade": 3,
    "subject": "mathematics",
    "instructions": "Generate multiplication problems",
    "language": "english",
    "count": 2
  },
  "questions": [
    {
      "type": "mcq",
      "question": "What is 3 × 7?",
      "answer": "21",
      "difficulty": "medium",
      "explanation": "Multiply 3 by 7",
      "options": {"A": "18", "B": "21", "C": "24", "D": "28"},
      "answer_choice": "B"
    }
  ]
}

Authentication

Three ways to provide your API key:

1. Config file (recommended)

incept-eval configure YOUR_API_KEY

2. Environment variable

export INCEPT_API_KEY=YOUR_API_KEY

3. Command line

incept-eval evaluate questions.json --api-key YOUR_API_KEY

Commands

Command	Description
configure	Save API key to config
evaluate	Evaluate questions from file
quick-eval	Quick single question evaluation
example	Generate example input
modules	Show available modules

Evaluation Modules

V3 Evaluation

• Scaffolding quality assessment
• Direct Instruction (DI) compliance
• Pedagogical structure validation

Answer Verification

• GPT-4 powered correctness checking
• Mathematical accuracy validation
• Confidence scoring

EduBench Tasks

• QA: Question Answering evaluation
• EC: Error Correction analysis
• IP: Instructional Planning assessment
• AG: Answer Generation

Examples

Quality Assurance Pipeline

incept-eval evaluate questions.json --output qa_results.json --verbose

Fast Validation

incept-eval evaluate questions.json --no-v3 --edubench

Batch Processing

for file in questions/*.json; do
  incept-eval evaluate "$file" --output "results/$(basename $file)"
done

Requirements

• Python >= 3.11
• Incept API key

Support

• Issues: GitHub Issues
• Documentation: CLI Usage Guide
• API Keys: https://incept.ai/api-keys

License

MIT License - see LICENSE file for details.

---

Made with ❤️ by the Incept Team