CheeseBench: Evaluating Large Language Models on Rodent Behavioral Neuroscience Paradigms

A benchmark for evaluating Large Language Models (LLMs) — and Vision-Language Models (VLMs) when run with image observations — on 9 classical behavioral neuroscience paradigms, each grounded in published rodent protocols with quantitative animal baselines. The default protocol is text-only (ASCII renderings of the environment), so any chat-completion model can be evaluated; vision is supported as an optional input mode.

Key Design Principles

1. Unified Protocol: Identical system prompt for ALL tasks — no task-specific hints
2. Published Baselines: Every environment maps to a real rodent experiment with peer-reviewed success rates
3. Cognitive Taxonomy: 6 cognitive dimensions (spatial learning, navigation, working memory, instrumental conditioning, avoidance learning, associative learning) mapped to neural circuits
4. Multi-Action: The model outputs up to 8 actions per call with explicit learnings/working memory

Quick Start

# Install from PyPI
pip install cheesebench

# Run against any OpenAI-compatible endpoint
cheesebench --model gpt-oss:120b \
    --api-url http://localhost:11434/v1/chat/completions \
    --api-format openai \
    --num-trials 20

# Quick smoke test (1 trial × 1 view mode)
cheesebench --num-trials 1 --view-modes ASCII_2D

# See all options
cheesebench --help

📊 Live leaderboard: https://huggingface.co/spaces/zachz/cheesebench-leaderboard

Development install

git clone https://github.com/stef41/CheeseBench
cd CheeseBench
pip install -e .

# Re-run analysis on your results
python analysis.py results/benchmark_results.json

Project Structure

cheesebench/
├── benchmark.py           # Main benchmark runner (CLI)
├── config.py              # Centralized configuration
├── analysis.py            # Cognitive profiling & analysis pipeline
├── task_definitions.json  # Task specs with paper citations & animal baselines
├── visualize.py           # Publication-quality figures
├── environments/          # 9 behavioral paradigms
│   ├── base_env.py        # Shared engine (rendering, sessions, actions)
│   ├── morris_water_maze.py
│   ├── t_maze.py
│   ├── barnes_maze.py
│   ├── radial_arm_maze.py
│   ├── operant_chamber.py
│   ├── shuttle_box.py
│   ├── place_preference.py
│   ├── star_maze.py
│   └── dnms_task.py
└── README.md

Environments & Cognitive Taxonomy

View Modes

System Prompt (Unified — Identical for ALL Tasks)

The model receives no task-specific instructions. It must discover the goal from observation and reward feedback alone:

You are an embodied agent placed in a behavioral experiment.
Your only goal is to maximize cumulative reward.

PERCEPTION:
- ASCII rendering (top-down, FPV, or pseudo-3D)
- Position/orientation shown by arrow: ↑ ↗ → ↘ ↓ ↙ ← ↖
- Walls (#, █) block movement. Open spaces are traversable.

ACTIONS (egocentric):
- FORWARD, ROTATE_LEFT, ROTATE_RIGHT, STAY

RESPONSE FORMAT:
LEARNINGS: <working memory — position, strategy, hypotheses>
ACTIONS: <1-8 comma-separated actions>

Analysis Pipeline

The analysis module computes:

• Cognitive profiles — radar chart scores across 6 dimensions
• Learning curves — rolling-window and block-based success rates
• Strategy metrics — action entropy, forward ratio, rotation ratio, repetition rate
• Wilson score CIs — 95% confidence intervals on all success rates
• Animal comparison — model profiles overlaid with rodent baselines

python analysis.py results/benchmark_results.json
# Outputs: results/benchmark_results_analysis.json

Configuration

All parameters are in config.py and overridable via CLI or environment variables:

export CHEESEBENCH_MODEL=gpt-oss:120b
export CHEESEBENCH_API_URL=http://localhost:11434/api/chat
export CHEESEBENCH_TIMEOUT=120

Citation

If you use CheeseBench in your research, please cite:

@inproceedings{cheesebench2025,
  title={CheeseBench: Evaluating Large Language Models on Rodent Behavioral Neuroscience Paradigms},
  author={},
  booktitle={NeurIPS Datasets and Benchmarks Track},
  year={2025}
}

License

MIT