Wanderer: An AI That Plays Minecraft By Itself

Wanderer is an open-source AI agent that joins your Minecraft world, explores on its own, and permanently learns new skills as it plays — mining, crafting, building, and fighting. The longer it plays, the smarter it gets. And it never forgets.

It's a friendly, pip-installable version of the research project Voyager (NVIDIA / Stanford / UT Austin) — think of it as "Voyager, but you can actually run it."

---

What can it do?

Give Wanderer a Minecraft world and it will:

• Decide its own next task. It starts simple ("collect wood") and naturally works its way up ("craft a diamond pickaxe", "build a nether portal") — like a self-driven learner.
• Learn skills forever. Every time it figures something out, it saves the skill. Next world, next session — it still remembers. No more starting from scratch.
• Fix its own mistakes. It gets feedback from the game, notices when it fails, and tries again until it gets it right.

All you do is open a world and let it in. It takes it from there.

---

Prerequisites (download these first)

Wanderer controls a real Minecraft game, so you'll need a few things before installing:

What	Why	Where to get it	Cost
Minecraft Java Edition (version 1.19)	Wanderer plays inside a real Minecraft world	minecraft.net	Paid (~$27)
Python 3.9–3.12	Runs the Wanderer code	python.org/downloads	Free
Node.js 16.13+	Drives the in-game bot	nodejs.org	Free
An OpenAI API key	The AI's "brain" (GPT-4)	platform.openai.com/api-keys	You pay as it runs*
A Microsoft account	Lets the bot join your world	Any Microsoft / Xbox account	Free

* Cost note: Wanderer calls OpenAI's GPT-4 as it plays, billed to your API key. The project's own tests show roughly $50 USD per ~160 tasks. Great for demos — keep an eye on your OpenAI dashboard.

---

Install (2 commands, ~5 minutes)

Open a terminal (Windows: PowerShell, macOS/Linux: Terminal) and run:

pip install wanderer-kada

Then set up the bot's game controller (this installs the Minecraft helper tools automatically):

wanderer-install

That's it. You're ready to run.

Note: the package is published as wanderer-kada, but you use it in Python as wanderer.

Installing from source (for developers)

git clone https://github.com/orewamash/Wanderer
cd Wanderer
pip install -e .
wanderer-install

---

First run (15–20 minutes)

Two one-time setups, then you're watching an AI learn Minecraft.

1. Connect your Microsoft account (one time)

Wanderer logs into Minecraft using your Microsoft account. Follow the step-by-step tutorial:

Connect your Microsoft account →

(You'll create a free "app registration" on Microsoft's developer site and paste a few values into a config file — the tutorial walks you through every click.)

Also install the small Fabric mods that help Wanderer see and control the game.

2. Start learning

from wanderer import Wanderer

azure_login = {
    "client_id": "YOUR_CLIENT_ID",            # from step 1
    "redirect_url": "https://127.0.0.1/auth-response",
    "secret_value": "[OPTIONAL] YOUR_SECRET_VALUE",
    "version": "fabric-loader-0.14.18-1.19",
}
openai_api_key = "YOUR_OPENAI_API_KEY"        # from platform.openai.com

wanderer = Wanderer(
    azure_login=azure_login,
    openai_api_key=openai_api_key,
)

wanderer.learn()

Minecraft will pop open. When it does:

1. Click Singleplayer → Create New World
2. Set Game Mode to Creative and Difficulty to Peaceful
3. Once the world loads, press Esc → Open to LAN
4. Set Allow cheats: ON → Start LAN World

Your AI companion will join the world within a minute. Sit back and watch it learn.

What you'll see

The console narrates everything:

• 📋 "Proposing next task" — the curriculum agent picking what to learn next
• ⛏️ "Executing" — the bot carrying out actions in-game
• ✅ / ❌ "Task completed / failed" — it gets feedback and adapts
• 🧠 "Skill saved" — new knowledge added to its permanent skill library

---

Cost & safety (please read)

• Money: Wanderer runs on your OpenAI key. It's pay-as-you-go and adds up fast (≈$50 / 160 tasks on GPT-4). Watch your usage dashboard.
• Your world: It plays with cheats enabled on your singleplayer world. It won't touch anything outside that world, but it is an autonomous AI — keep an eye on it at first.

---

FAQ

"The bot isn't moving / nothing happens after I open to LAN." Make sure the Fabric mods are installed for the correct version (fabric-loader-0.14.18-1.19) and that the version string matches exactly. See Troubleshooting.

"It says something about access_token after I paste the login link." Re-run the login step and paste the whole link into the terminal — this is covered in the FAQ.

"Can I use a cheaper model?" Yes, but GPT-4 is strongly recommended — cheaper models make worse decisions. See the FAQ for tips.

"Can it keep learning where it left off?" Yes. Wanderer saves checkpoints as it learns. Details in the Advanced section below.

"Does it need a server or GPU?" No. All the AI runs through OpenAI's API over the internet — your computer just runs the game and the bot.

---

Advanced

Resume learning from a checkpoint

wanderer = Wanderer(
    azure_login=azure_login,
    openai_api_key=openai_api_key,
    ckpt_dir="YOUR_CKPT_DIR",
    resume=True,
)

Run a specific task with a learned skill library

wanderer = Wanderer(
    azure_login=azure_login,
    openai_api_key=openai_api_key,
    skill_library_dir="./skill_library/trial1",
    ckpt_dir="YOUR_CKPT_DIR",
    resume=False,
)

task = "Craft a diamond pickaxe"
sub_goals = wanderer.decompose_task(task=task)
wanderer.inference(sub_goals=sub_goals)

For all valid skill libraries, see Learned Skill Libraries.

---

How it works (3 components)

1. Automatic Curriculum – proposes the next task to maximize exploration
2. Ever-Growing Skill Library – stores complex behaviors as executable code that compounds over time
3. Iterative Prompting – a self-verifying loop that uses in-game feedback, errors, and critique to improve

Wanderer talks to LLMs like GPT-4 over a simple API — no model fine-tuning needed. Skills are interpretable (readable code), composable (new skills build on old ones), and permanent.

---

FAQ & Support

More questions? Check the full FAQ before opening an issue.

Credits

Wanderer is a rebranded, packaging-focused distribution of Voyager: An Open-Ended Embodied Agent with Large Language Models by Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar (NVIDIA / Stanford / UT Austin).

If you find this work useful, please consider citing the original paper:

@article{wang2023voyager,
  title   = {Voyager: An Open-Ended Embodied Agent with Large Language Models},
  author  = {Guanzhi Wang and Yuqi Xie and Yunfan Jiang and Ajay Mandlekar and Chaowei Xiao and Yuke Zhu and Linxi Fan and Anima Anandkumar},
  year    = {2023},
  journal = {arXiv preprint arXiv: Arxiv-2305.16291}
}

Disclaimer: This project is for research and educational purposes, and is not an official product from NVIDIA.