minecombat-eval 0.2.0

TCP client and bootstrap tooling for MineCombat-Evaluation (Minecraft combat benchmarks)

MineCombat-Evaluation

Paper plugin + TCP JSON protocol for combat-survival benchmarks on Minecraft 26.1 (Paper 26.1.2, plugin 0.1.0-SNAPSHOT, protocol 1).

Quick start

cp .env.example .env          # JAVA_21, JAVA_25, SERVER
python3 -m venv .venv && source .venv/bin/activate
pip install -e .              # CLI + versioned world artifact (5.5 MB zip in package)
minecombat-eval bootstrap     # Paper + plugin + mcbench_flat-v1 + config
minecombat-eval server start  # join localhost, then:
minecombat-eval run-suite l1-v1 -o results/l1-v1-ref.jsonl

Docker: docker compose up --build → join :25565 → minecombat-eval run-suite l1-v1 --port 8765

Full path: planning/run-benchmark.md · World artifact: artifacts/README.md

---

Releases (PyPI + GitHub)

Tagging v* now triggers .github/workflows/release.yml, which:

• builds Python wheel + sdist
• publishes minecombat-eval to PyPI
• creates a GitHub Release with:
  • wheel + sdist
  • world artifact zip (mcbench_flat-v1.zip)
  • manifest.json
  • SHA256SUMS.txt

One-time repository setup

1. Configure trusted publishing for PyPI project minecombat-eval (recommended), or set PYPI_API_TOKEN if you prefer token mode.
2. Ensure the GitHub Actions environment named pypi exists and is allowed for release workflow.

---

Brand Assets

• Prompt pack for polished banners/icon: assets/branding-prompts.md
• Generated files should live under assets/ (e.g. minecombat-banner-hero.png, minecombat-icon.png)

---

Scripts

From repo root (loads .env):

Script	Purpose
./scripts/run-gradle.sh …	Build plugin (JDK 21)
./scripts/run-paper.sh	Start Paper (JDK 25)
./scripts/sync-config.sh	Copy repo config.yml → server (restart Paper after)
python3 scripts/summarize_results.py	JSONL → tables / CSV

JAR output: paper-plugin/build/libs/minecombat-evaluation-0.1.0-SNAPSHOT.jar

.env example

JAVA_21=/path/to/temurin-21/Contents/Home
JAVA_25=/path/to/temurin-25/Contents/Home
SERVER=$HOME/minecraft-paper-mcbench

World: mcbench_flat (level-name in server.properties). L2 arenas are not generated from config — see planning/world-setup.md.

Python eval

Requires Paper + plugin + one player online. Stdlib only (Python 3.10+).

python3 run_eval.py --scenario ZombieRoom-v0 --episodes 3 --seed-base 0
python3 run_suite.py --suite benchmarks/l1-v1/suite.json -o results/l1-v1.jsonl

Flags and scenario ids: planning/commands-and-scenarios.md. Custom policies: planning/agent-integration.md.

Official baseline: ReferenceCombatPolicy (minecombat_eval/reference_policy.py).

Eval logs (episodes.jsonl, results/) are gitignored.